Evidence generation plan for HTE10063 Digital technologies to support asthma self-management
Open for comments Open until Request commenting lead permission
We have updated this service so that members of the same organisation can now collaborate on a joint online response.
Read our blog to learn more.
3 Approach to evidence generation
3.1 Evidence gaps and ongoing studies
Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment group's (EAG's) report; evidence not meeting the scope and inclusion criteria is not included. The table shows the evidence available to the committee when the guidance was published.
Evidence gap | Asthmahub | Asthma hub for parents | AsthmaTuner | BreathSmart/Respi.me (RDMP) | Digital Health Passport | Luscii | myAsthma | Smart Asthma |
|---|---|---|---|---|---|---|---|---|
Clinical outcomes | Limited evidence | Limited evidence | Limited evidence | Limited evidence | Limited evidence | Limited evidence | Limited evidence | Limited evidence |
Uptake and attrition rates | Limited evidence (Ongoing study) | No evidence | Limited evidence (Ongoing study) | Limited evidence (Ongoing study) | Limited evidence (Ongoing study) | No evidence | Limited evidence (Ongoing study) | No evidence |
Impact on condition management | Limited evidence (Ongoing study) | Limited evidence | Limited evidence (Ongoing study) | Limited evidence (Ongoing study) | Limited evidence (Ongoing study) | Limited evidence | Limited evidence (Ongoing study) | Limited evidence |
Healthcare resource use | Limited evidence | No evidence | Limited evidence (Ongoing study) | Limited evidence | Limited evidence (Ongoing study) | No evidence | Limited evidence | No evidence |
Generalisability | Good evidence (Ongoing study) | Limited evidence | Limited evidence | Limited evidence | Limited evidence (Ongoing study) | No evidence | Limited evidence (Ongoing study) | No evidence |
Barriers and facilitators | No evidence | No evidence | Limited evidence (Ongoing study) | Limited evidence (Ongoing study) | Limited evidence (Ongoing study) | No evidence | Limited evidence (Ongoing study) | Limited evidence |
The EAG identified multiple ongoing studies across the technologies, summarised in Table 17 of the EAG's report. These may provide additional evidence on asthma control, medication use, adherence and quality of life. But the studies will not address all the uncertainties, such as the lack of robust comparative evidence, limited UK data and the absence of reliable information on uptake, engagement and attrition.
3.2 Data sources
Most of the data needed for this evaluation, particularly outcomes about engagement or attrition patterns, is best collected as primary data within the technologies themselves. Some additional outcomes may be available through existing NHS data sources, but generally these will need to be linked to the primary dataset to ensure completeness and accuracy.
Several data collections have different strengths and weaknesses that could potentially support evidence generation. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question. Potential data sources include:
Clinical Practice Research Datalink (CPRD)
Hospital Episode Statistics (HES)
Hospital Admitted Patient Care Activity data
Hospital Accident and Emergency Activity data (covering acute asthma presentations)
the UK Severe Asthma Registry, which provides detailed clinical information for people with severe asthma managed in specialist centres
National Respiratory Audit Programme (NRAP).
The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach to ensure good-quality data with broad coverage.
3.3 Evidence collection plan
Prospective real-world comparative cohort study
A prospective real-world comparative cohort study should be done across NHS sites where the digital technologies for supporting asthma self-management are offered and compared with similar sites where they are not yet in use. People with asthma in both groups should be followed from the point when they would typically be offered the technology, reflecting routine referral pathways in primary and secondary care. This includes groups of people with different levels of asthma control (controlled, partly controlled and uncontrolled).
The comparison group should include people from similar services with comparable asthma pathways, clinical structures and patient populations without access to the digital technology. Ideally, the study should be done across multiple centres to reflect the diversity of the NHS service provision.
Non-random assignment to interventions introduces a risk of confounding bias. So, appropriate methods, such as matching or adjustment (for example, propensity score methods), should be used to minimise selection bias and balance confounding factors between groups. High-quality data on patient characteristics will be essential to support these methods. The identification of key confounders should be informed by expert input during protocol development.
Qualitative survey
A qualitative study should be undertaken to understand the experiences of people using the technologies to support asthma self-management, as well as the views of parents or carers (for children) and relevant clinicians. Evidence should be collected through semistructured interviews, structured feedback and focus groups with a diverse sample of users across different NHS sites and clinical severity groups. The robustness of the findings will depend on:
broad and inclusive recruitment across eligible users
the sample of respondents being representative of the population of potential users (for examples, variation in asthma control, socioeconomic status or ethnicity)
capturing and documenting the reasoning for usability, barriers and facilitators, changes in asthma regimens and perceived benefits.
3.4 Data to be collected
Demographic and baseline characteristics
Age, sex and ethnicity
Asthma severity and control
Long-term conditions (such as COPD, anxiety or depression)
Postcode deprivation index
Clinical outcomes
Exacerbations (mild, moderate, severe, emergency department visits)
Lung function measurements, ideally using spirometry (for technologies that offer external devices to measure lung function, data on cost effectiveness and impact on quality of life is expected to be submitted)
Asthma control
Rescue versus controller medication use
Number of asthma attacks
Time to exacerbation
Treatment step-up or step-down
Smoking status
Quality of life questionnaire (EQ-5D-5L or EQ-5D-Y for children) at baseline and at 3, 6, 12 or 18 months or, ideally, up to 2 years
Adverse effects (such as anxiety)
Asthma Quality of Life Questionnaire (AQLQ)
Ideally, missed school or work days
Uptake and attrition rates
Date the technology was first used
Number of logins per month
Duration of use (days between logins and inactivity periods)
Discontinuation date and reason
Completion rates of app tasks or information material
Percentage of active users at 1, 2, 3, 6, 12 or 18 months or, ideally, up to 2 years
Engagement reported by asthma control at baseline
Impact on condition management
Understanding inhaler technique steps
Changes in symptom recognition scores
Completing and updating personalised asthma action plan
Self-reported ability to interpret triggers, warning signs and deterioration
Adherence to preventative medication
Qualitative findings on improved understanding or self-management
Healthcare resource use
Number of GP visits
Number of specialist visits
Number of emergency department attendances
Number of hospital admissions and length
Use of out of hours services
Courses of corticosteroids prescribed in primary or secondary care
Generalisability to UK guidelines
Setting (UK or outside UK)
Baseline asthma severity data (controlled, partly controlled or uncontrolled)
Reporting of medication regimen
Reporting of alignment with NHS care pathway
Barriers and facilitators
User-reported technical issues
Qualitative data on satisfaction and acceptability
Patient and clinician views on usefulness of modules
Accessibility needs (support with visual, cognitive or language needs)
Reporting barriers and facilitators when used in clinical practice
Data collection should follow a predefined protocol, and quality assurance processes should be in place to ensure the integrity and consistency of data collection. NICE's real-world evidence framework provides guidance on the planning, conduct and reporting of real-world evidence studies.
3.5 Evidence generation period
This will be 3 years to allow for setting up, implementing the test, data collection, analysis and reporting.
3.6 Following best practice in study methodology
Following best practice in conducting studies is paramount for ensuring the reliability and validity of the research findings. Adherence to rigorous guidelines and established standards is crucial for generating credible evidence that can ultimately improve patient care. The NICE real-world evidence framework details some key considerations.
Within the context of an early value assessment, a key factor to consider as part of the informed consent process is to ensure that patients (and their carers, as appropriate) understand that data will be collected to address the evidence gaps identified in section 2. Where applicable, this should take account of NICE guidance about shared decision making.
How are you taking part in this consultation?
You will not be able to change how you comment later.
You must be signed in to answer questions