3 Approach to evidence generation

3.1 Evidence gaps and ongoing studies

Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment group's report. Information on the studies in the table can be found in the supporting documents. The table shows the evidence available to the committee when the guidance was published.

**Table 1 Evidence gaps and ongoing studies**
Evidence gap	mymobility	GoWellHealth	BPMpathway	Physitrack
Condition	Hip or knee replacement surgery	Hip replacement surgery	Knee replacement surgery	Knee replacement surgery
Comparative effectiveness versus NHS current practice	Limited evidence; ongoing study	Limited evidence	Limited evidence; ongoing study	Limited evidence
Adverse events	No evidence	No evidence	No evidence	No evidence
Health-related quality of life	Good evidence; ongoing study	Limited evidence	Limited evidence	Good evidence
Healthcare resource use	Good evidence; ongoing study	Limited evidence	Limited evidence	Limited evidence
Patient engagement and adherence	Limited evidence	Limited evidence	Limited evidence; ongoing study	No evidence

3.2 Data sources

Some of the data needed for a future evaluation, particularly outcomes about engagement, is best collected as primary data through the technologies themselves.

There are several data collections that have different strengths and weaknesses that could potentially support evidence generation. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question. Potential data sources include:

the National Joint Registry, which is useful to identify data on procedure type and long-term outcomes
Hospital Episode Statistics, which is helpful to identify hospital admissions, length of stay, same-day discharge cases and readmissions
primary care data sources such as the Clinical Practice Research Datalink (CPRD) and OpenSafely, which can be useful to gather information on GP appointments, prescriptions for analgesics and use of related community services
sub-national NHS England Secure Data Environment service.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach of ensuring good-quality data with broad coverage.

3.3 Evidence collection plan

Prospective real-world comparative cohort study

A prospective real-world comparative cohort study is suggested across NHS sites that offer digital platforms to support preparation before and rehabilitation after primary elective hip or knee replacement surgery compared with similar NHS sites that do not offer the digital platforms. People having surgery should be followed from the point at which they would typically enter the elective joint replacement pathway.

It is critical that the people in the cohorts are sub-grouped by either hip or knee replacement surgery. Data from the cohorts using the digital platforms should be compared with cohorts having NHS current practice, such as written information with no face-to-face physiotherapy, or face-to-face physiotherapy without access to the digital platforms. Ideally, the study should be done across multiple NHS centres to reflect variation in service delivery, rehabilitation pathways and patient demographics across the NHS.

Non-random assignment to interventions introduces a risk of confounding bias. So, appropriate methods, such as matching or adjustment (for example, propensity score methods), should be used to minimise selection bias and balance confounding factors between groups. High-quality data on patient characteristics will be essential to support these methods. The identification of key confounders should be informed by expert input during protocol development.

Data analysis should assess the relationship between digital platform use and clinical outcomes such as hospital length of stay. This will help inform whether these approaches are associated with shorter admissions and faster time to recovery, including in people who had same-day discharge.

Qualitative survey

A qualitative study should be done to understand the experiences of people using the technologies to support rehabilitation before and after surgery, as well as the views of carers (where relevant), physiotherapists, orthopaedic teams and service managers. Evidence should be collected through semi-structured interviews, structured feedback and focus groups with a diverse sample of users across different NHS sites and type of surgical group. Advice on the conduct of qualitative studies is available in the NICE real-world evidence framework.

3.4 Data to be collected

When possible and appropriate, the following outcomes should be collected at baseline, 6 weeks, 3 months, 6 months and ideally 12 months.

Study population and baseline characteristics

age at surgery
sex
ethnicity
socioeconomic deprivation index
type of surgery (total hip replacement, hip resurfacing, total knee replacement or partial knee replacement)
baseline digital literacy and access to smartphone or tablet
baseline functional status
baseline pain severity.

Comparative effectiveness versus NHS current practice

technology used
start date and duration of use
standard care in the comparable NHS site
number and type of healthcare professional contact in each arm
use of additional support.

Healthcare resource use

length of stay (days)
number of same-day discharge cases
length and type of appointments (for example, phone call follow-up, app responses)
number of physiotherapy sessions successfully completed
number of face-to face reviews escalated for further support
number of GP appointments related to rehabilitation
number of emergency department attendances
number of readmissions and root cause
number of analgesic prescriptions.

Patient engagement and adherence

number of patients offered the technology and sign-up registrations
number of logins per week
number of exercises provided and number of exercises completed
percentage of adherence to programme and reasons for engagement
percentage of discontinuation to programme and reasons for discontinuation
patient satisfaction with the technology
adherence and discontinuation (presented by age, sex, deprivation status, type of surgery and digital literacy).

Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies.

3.5 Evidence generation period

The evidence generation period will be 3 years, to allow for enough time to set up and implement the technology, collect the necessary data, and analyse and report it.

3.6 Following best practice in study methodology

Following best practice in conducting studies is paramount to ensuring the reliability and validity of the research findings. Adherence to rigorous guidelines and established standards is crucial for generating credible evidence that can ultimately improve patient care. The NICE real-world evidence framework details some key considerations.

Within the context of an early value assessment a key factor to consider as part of the informed consent process is to ensure that patients (and their carers, as appropriate) understand that data will be collected to address the evidence gaps identified in section 2. Where applicable this should take account of NICE guidance about shared decision making.

How are you taking part in this consultation?

Question on Consultation

Question on Consultation

Question on Consultation

Question on Consultation

Question on Document

Evidence generation plan for HTG10887 Digital platforms to support preparation before and rehabilitation after primary elective hip or knee replacement surgery: early-use assessment