Advice
Clinical and technical evidence
Clinical and technical evidence
A literature search was carried out for this briefing in accordance with the interim process and methods statement for medtech innovation briefings. This briefing includes the most relevant or best available published evidence relating to the clinical effectiveness of the technology. Further information about how the evidence for this briefing was selected is available on request by contacting mibs@nice.org.uk.
Published evidence
Evidence from 8 studies is summarised in this briefing. The evidence base would benefit from a systematic review.
The studies in this briefing include 1 randomised controlled trial (Hollis et al. 2018), 2 objective measure studies (Edebol et al. 2012, Vogt and Shameli 2018), 1 qualitative study of user experience (Hall et al. 2017), 1 audit (Hall et al. 2016), 1 diagnostic study (Hult et al. 2018), 1 national evaluation report (East Midlands Academic Health Science Network [EMAHSN] 2022) and 1 real-world demonstrator project (NIHR Collaboration for Leadership in Applied Health Research & Care [CLAHRC] and EMAHSN 2017). The size of the population in these studies ranged from 40 to 340 people.
The clinical evidence and its strengths and limitations are summarised in the overall assessment of the evidence.
Overall assessment of the evidence
Hollis et al. (2018)
Intervention and comparators
QbTest combined with standard care attention deficit hyperactivity disorder (ADHD) assessment (QbOpen), standard care ADHD assessment (QbBlind), with 2 experienced child psychiatrists with access to clinician-completed global assessment scores blinded to group allocation.
Key outcomes
Participants whose clinicians had access to QbTest (QbOpen) were 44% more likely during the study period to receive a diagnostic decision compared with those having assessment as usual without a QbTest report (hazard ratio 1.44; 95% confidence interval [CI] 1.04 to 2.01; p=0.029). Clinicians were more likely to make a diagnostic decision about ADHD when they had access to a QbTest report (QbOpen) than when the QbTest report was withheld (QbBlind; 94/123 [76%] compared with 76/127 [60%], odds ratio 2.43; 95% CI 1.35 to 4.49; p=0.003). Compared with the index test, the sensitivity of the QbOpen clinicians' confirmed diagnosis was 86.0%, whereas the QbBlind clinicians' confirmed diagnosis had a higher sensitivity of 96.1%. Specificity was reported to be 39.4% in the QbOpen group and 36.0% in the QbBlind group. Statistical analysis reported no significant differences in sensitivity and specificity between the groups (p=0.64). Cost analysis of using QbTest reports suggested an incremental cost-effectiveness ratio of £1.72. Health economic analysis dominated standard care; however, cost savings were small, suggesting that the impact of providing the QbTest report within this trial can best be viewed as cost neutral.
Strengths and limitations
This is a well-designed comparator study of high methodological quality. The clinical setting and population are relevant to the NHS. The outcome measures reported are appropriate. There is limited detail about the method of randomisation; however, the study does reference use of a web-based system. Power calculations are reported. The study reports time to diagnosis as the primary outcome for the economic evaluation; the study does not report the subsequent treatment and clinical outcome after treatment. A limitation is that the health economic analysis was based on a 6‑month time horizon and discounting was not applied to costs or outcomes. As such, it was not possible to determine longer-term costs associated with cases awaiting diagnostic determination (which was more common when clinicians did not have access to the QbTest report).
Edebol et al. (2012)
Study size, design and location
Objective measures study evaluating behaviour manifestations in adults with ADHD and in adults with or without other conditions. The study consisted of 306 participants belonging to 4 groups: diagnosed with ADHD (n=53), either bipolar 2 disorder or borderline personality disorder (n=45), assessed for but disconfirmed diagnosis of ADHD (n=29) and the adults without any of these conditions, described in the 'adult normative group' (n=179).
Intervention and comparators
Evaluation of 2 psychometric instruments derived from QbTest, summarised into a Weighted Core Symptoms Scale.
Key outcomes
The Weighted Core Symptoms Scale separated ADHD and normative participants from each other as well as the other 2 clinical reference groups. The highest level of core symptoms reported were in the ADHD group and the lowest level in the normative group. Analyses with prediction of ADHD yielded 85% specificity for the normative group, 87% sensitivity for the ADHD group, 36% sensitivity for the bipolar 2 and borderline group and 41% sensitivity for the group with disconfirmed diagnosis of ADHD. The results of this study helped in the objective assessment of adult ADHD.
Strengths and limitations
The study has a sufficient study size and provides a combination of prospective design, proper scientific approach and a study hypothesis. It also provides both comparison to a general population and a differential diagnosis. Sensitivity for QbTest was lower in complex clinical groups with other conditions and in those with disconfirmed diagnosis. The study did have some limitations in that most participants had ADHD in combined form, which may have caused generalisation of data. Also, the group with ADHD was tested with QbTest by their clinical contact, which may have created sample biases or affected the generalisation of the study.
Vogt and Shameli (2018)
Intervention and comparators
Two groups of ADHD assessments were compared: the first group without any objective measures within the assessment and the second group with objective measures included.
Key outcomes
The study showed objective measures improve differentiation between ADHD and other conditions where symptoms overlap with ADHD. The study results stated a reduction in the risk of unidentified ADHD (p<0.0035), as measured by subsequent rates of revised diagnosis over a 12‑month period. The study states that the introduction of an objective measure into clinical assessment of ADHD will aid in clinical diagnosis and strengthen clinical decision making.
Hall et al. (2017)
Key outcomes
Qualitative findings through semi-structured interviews reported that QbTest was considered valuable and facilitated communication between clinicians, families and schools. In the survey, all clinicians reported that QbTest was helpful to evaluate treatment effects; however, only 39% of families felt it helped them understand the decision.
Hall et al. (2016)
Key outcomes
Significantly fewer clinician consultations (mean 2.18 compared with 3.05; p<0.02) were required to confirm the diagnosis of ADHD when the QbTest was used to augment assessment in comparison to standard assessment.
Strengths and limitations
One limitation of this study was that ADHD diagnoses were not independently verified, and it is not known whether QbTest helped exclude ADHD diagnosis in non-ADHD cases referred for ADHD diagnostic assessment, limiting the comprehensiveness of the findings. Findings were limited to 1 NHS trust, which is a small sample size. Strengths of this audit include the similar composition of children in each group and the random selection.
Hult et al. (2018)
Intervention and comparators
QbTest in children with ADHD and in children with other clinical diagnoses.
Key outcomes
Only QbTest parameters for inattention and hyperactivity differentiated between ADHD and other clinical diagnoses at the p≤0.01 level, not for measures of impulsivity. Sensitivity ranged from 47% to 67% and specificity from 72% to 84%. The positive predictive value ranged from 41% to 86%, and negative predictive value from 43% to 86%. Area under the curve varied from 0.70 to 0.80. The study states that analysing QbTest performances in different clinical groups (including ADHD) might give valuable information on clinical presentation that might explain more than the broad diagnostic categories.
Strengths and limitations
Strengths of the study include that the study group was representative of patients who would receive the test in practice, and both the study and comparison groups received the same reference tests. A limitation in the study is that, although the ADHD was not based on results from the QbTest, the results were known to some clinicians, whose information could have contributed to the final diagnosis. Also, no inter-rater reliability tests about diagnoses were carried out.
EMAHSN (2022)
Study size, design and location
Focus ADHD national programme evaluation report 2022. A total of 549 pre-implementation cases and 549 post-implementation cases were analysed. Evaluation was carried out in NHS sites across England. The national programme was supported by all 15 regional AHSNs in England and Qbtech Ltd.
Key outcomes
The evaluation found implementation of QbTest in addition to standard care reduced the number of clinical appointments needed to reach a diagnostic decision. There was a reduction of 17% in the number of school observations that were conducted pre- and post-implementation and a 5% increase in patients having ADHD ruled out as a diagnosis. There was a reduction in the mean number of clinical appointments from 3.22 pre-QbTest to 2.85 post-QbTest implementation, though this is not statistically significant. This change translates to an 11.5% release of clinical consultations. There was an increase in number of days to reach a diagnostic decision (10.3%) and from initial referral to diagnosis (12.2%), with the former being statistically significant.
Strengths and limitations
Sites were responsible for their own audit collection data, so it was difficult to ascertain consistency with what the evaluation requested. Although checked, the evaluation team had no way of determining accuracy of the data. There was also the possibility of non-response bias from sites that did not participate, as well as geographical bias as most responses came from London and South East England. This evaluation was also done during COVID-19, which would have limited the generalisability of the findings.
NIHR CLAHRC East Midlands and the EMAHSN (2017) – AQUA randomised controlled trial and the Transforming ADHD Demonstrator Project (2017)
Study size, design and location
ADHD care in the East Midlands. The NIHR CLAHRC East Midlands-funded 10-site, 18-month long randomised controlled trial. The EMAHSN funded and led the 12-month, 7-site real-world demonstrator project. The CLAHRC's 'AQUA' randomised controlled trial explored the clinical use of QbTest for ADHD diagnosis alongside standard practice in CAMHS and community paediatric services. The EMAHSN Transforming ADHD Care Project used the QbTest diagnostic tool across 3 counties to prove the impact on patient experience, efficiency and time to diagnosis.
Key outcomes
There was a reduction from the first appointment to diagnosis by an average of 146 to 201 days depending on the model implementation. Eighty-five percent of patients surveyed stated the test helped them better understand their symptoms. There was a release of between 20% and 33% of clinician workforce time. EMAHSN demonstrator projects also saw cost reductions between 9% and 39% depending on model implementation used and a return of investment between £14,300 and £93,000 when QbTest was used in the assessment process. It was also found to maintain clinical accuracy despite speeding up diagnosis. The study found QbTest to be acceptable and feasible for implementation.
Recent and ongoing studies
A nurse-led model of care for ADHD. Status: recently completed. Indication: ADHD. Outcomes: reduced waiting list and prescription costs achieved through using QbTest.
Although there are ongoing studies, the company has suggested they will not conclude for some time and are therefore not appropriate to include.