3 Evidence

NICE commissioned an external assessment group (EAG) to review the evidence submitted by the company. This section summarises that review. Full details of all the evidence are in the project documents on the NICE website.

Clinical evidence

The clinical evidence comprises 11 studies, 1 of which is a randomised controlled trial

3.1

The EAG's review included 11 studies: 1 randomised controlled trial, 4 diagnostic studies, 1 cross-sectional design study, 1 case report, and 4 publications that included a mix of before-and-after and case-series study design. Six of the studies included a total of 656 people with hip or knee osteoarthritis. Four of the studies included a total of 242 people at risk of falls. One study was done in a healthy population of 136 people with no existing gait problems. For full details of the clinical evidence, see section 3 of the assessment report in the supporting documentation.

There are 4 relevant studies in people at risk of falls

3.2

The relevant evidence for people at risk of falls included:

  • a before-and-after study of 121 people who had had an injurious fall and were in community care (Rodgers et al. 2020)

  • a before-and-after study of 46 people at risk of falls from 2 GP surgeries (Hodgins and Newby 2023a)

  • a before-and-after study of 46 people at risk of falls in community care (Hodgins and Newby 2023b)

  • a case series that reported on how gait parameters in a healthy older population differed from 18 older people with gait and balance issues (Hodgins and McCarthy 2015). The EAG considered that this study had limited applicability because it used a healthy control group and did not use the vGym app to generate the personalised exercise plan.

There are 3 relevant studies in people having a hip or knee replacement, including a randomised controlled trial

3.3

The study most clinically relevant to people having hip or knee replacements was the parallel group randomised controlled trial (McNamara et al. 2023). This compared GaitSmart with standard-care rehabilitation after surgery in 44 people who had total knee or hip arthroplasty, but whose rehabilitation goals had not been met. Other relevant studies included:

  • a cross-sectional study in 74 people who had or were having a knee replacement, in which outcomes were compared between 4 groups who were each at different timepoints relating to surgery (Rahman et al. 2015)

  • a case series that reported gait differences in a group of 55 people with hip arthrosis and included comparisons with a control group of people with no health issues (Hanly et al. 2016).

    The EAG considered that these 2 studies had limited applicability because they both used a healthy control group and did not evaluate the GaitSmart exercise programme. There were also 3 diagnostic studies in relevant populations, which compared GaitSmart with optical tracking systems (IMI‑APPROACH, McCarthy et al. 2013, Zügner et al. 2019). Outcomes from these studies suggested that GaitSmart measurements correlate with other comprehensive gait analysis systems.

There is a high degree of heterogeneity in the evidence, which reflects the variation in the care pathway

3.4

There was a high degree of heterogeneity in terms of the comparisons made and outcomes reported in the studies. So, a meta-analysis was not done. The clinical experts noted that the care pathways are extremely variable for people at risk of falls, and for people who have had a hip or knee replacement. This may make it difficult to identify appropriate comparators. The EAG also noted that there was poor reporting of study designs and recruitment methods in the included studies. There was also a lack of long-term follow up for clinical outcomes and data on adherence to the intervention protocol.

Results from the studies indicate that GaitSmart has the potential to improve gait parameters and patient-reported outcomes

3.5

The relevant evidence for changes in gait parameters and patient-reported outcomes was based on comparative studies that included control groups and single-arm studies. The clinical evidence was primarily generated in settings that are generalisable to the NHS. There were strengths in the available studies in that they used validated tools to measure patient-reported outcomes and function. Also, consideration was given to whether observed changes were clinically significant. Based on the limited clinical evidence, the EAG considered that the case for adoption of GaitSmart was potentially supported, but that further evidence generation would be beneficial.

No adverse events are reported in the literature

Cost evidence

The company's cost model for people at risk of falls finds GaitSmart to be cost saving compared with standard care

3.7

The company developed a decision-tree model from an NHS perspective with a time horizon of 1 year, which compared GaitSmart with individual physiotherapy. Clinical inputs were taken from the single-arm Rodgers et al. (2020) study. In the company's model, each intervention was applied only after a fall that had resulted in an injury. People who had an injurious fall had medical attention through ambulance call-out, a GP visit or attendance at an emergency department. The model used the probability of falls among community-dwelling adults aged over 65. It used the average probability of recurrent falls by fear of falling from Berry and Miller (2008), Tinetti and Williams (1998) and Arfken et al. (1994). The falls risk reduction was assumed to be 0% with standard care and 1.77% with GaitSmart. The response rate of each intervention was not considered in the company's model. The company's base case showed a cost saving of £2.90 per person using GaitSmart. For full details of the cost evidence, see section 4 of the assessment report in the supporting documentation.

The company's cost model for hip and knee rehabilitation finds GaitSmart to be cost saving compared with standard care

3.8

The company developed a decision-tree model from an NHS perspective with a time horizon of 17 weeks. It compared GaitSmart with self-managed home exercise, or group or individual physiotherapy. It was assumed that 20% of people carry out self-managed rehabilitation, and 80% have group or individual physiotherapy. Clinical inputs were taken from the McNamara et al. (2023) randomised controlled trial. This included estimates of the response rate of each intervention and the reduction in falls risk. The company's model considered the change in falls risk of each intervention through the observed change in gait speed. It did this using individual-level data from McNamara et al. (2023), and a risk ratio between gait speed and falls risk of 1.069/10 cm/s decrease from Verghese et al. (2009). A response rate of 0.79 was assumed for the standard-care arm and of 0.93 for the GaitSmart arm. This rate was defined as any improvement in gait speed. The probability of falls was estimated as 0.4. This was calculated using the proportion of people who had falls after arthroplasty from Smith et al. (2016) or because of symptomatic osteoarthritis from Doré et al. (2015). The company's base case showed a cost saving of £450.56 per person using GaitSmart. For full details of the cost evidence, see section 4 of the assessment report in the supporting documentation.

The company's cost model for people at risk of falls is appropriate, but there are EAG changes to the model structure and parameters

3.9

The EAG stated that the company's falls model was flawed because of the time point when the intervention was provided. It meant that each intervention was applied only after an injurious fall. The company's model did not model further outcomes after an intervention was given for people who had an injurious fall. The EAG's falls model started with people having either GaitSmart or standard care. At the end of each branch, people had either falls or no falls. The falls outcomes were modelled following a fall. The EAG was concerned that the probability of falls needing medical treatment in the company's model was taken from an Australian study, Watson et al. (2011). This may not be generalisable to the NHS setting. In the EAG's model, the probability of injurious falls and medical treatment after a fall were taken from Craig et al. (2013). This was used to populate the return on investment tool developed by Public Health England for falls prevention programmes for older people in the community. The EAG increased the total cost for all GaitSmart sessions per patient from £40.00 to £82.00. This included the total staff costs for the intervention. The total cost for standard care was calculated by the EAG to be £102.71 rather than £765.00. This large decrease was primarily because the number of physiotherapy sessions was reduced from 30 to 8. Also, sessions after the initial appointment were assumed to be group rather than individual physiotherapy. The EAG made small changes to the cost of events after a fall, which were:

  • GP visit: from £36 to £42

  • ambulance call-out: from £257 to £282

  • emergency department visit with no admission: from £166 to £118

  • inpatient stay: from £1,609 to £1,950.

The company's cost model for people having hip or knee replacements is appropriate, but there are some EAG changes to model parameters

3.10

The EAG did not agree with the company's calculation of the response rate for each intervention in the rehabilitation model. In the EAG's model, the falls risk ratio for each intervention was calculated using the approach detailed in Verghese et al. (2009). This study specified that there was a change in the risk of falls of 1.069 per 10 cm/s decrease in gait speed. The risk ratio was then applied to the probability of baseline falls to yield the falls probability of each intervention. The EAG also separated the falls risk ratio for people who did or did not have a response to the intervention in the rehabilitation model. The EAG used updated Personal Social Services Research Unit cost data and inflated it for 2021/2022. This increased the total cost for all GaitSmart sessions per patient from £67.00 to £82.00. The total cost for all group or individual physiotherapy sessions per patient was calculated by the EAG to be £198.44 rather than £643.98. This decrease was primarily because consultant time was excluded.

GaitSmart is cost saving compared with standard care in the EAG's base-case models

3.11

In the EAG's base-case model for people at risk of falls, GaitSmart remained cost saving by £28.70 compared with individual physiotherapy. In the sensitivity analysis, the EAG selected a variety of comparator options. All included an initial 45‑minute assessment by a band 5 physiotherapist, followed by a variety of group or 1:1 interventions. The sensitivity analysis also used a risk ratio ranging from 0.5 (50% reduction in falls) to 1 (no reduction in falls) to 1.5 (50% increase in falls). A two-way analysis was done to identify the point of cost neutrality when factoring in the risk ratio for standard care. Cost neutrality was likely to lie between £70.00 and £110.00 per person for standard care, with GaitSmart costing £82.00 per person to deliver. When using the EAG's base-case model for rehabilitation, GaitSmart remained cost saving by £80.39 per person compared with standard care. The EAG did a series of one-way sensitivity analyses for several key parameters. GaitSmart was found to be cost saving across the range of results for each parameter. One EAG scenario varied standard care by substituting a band 6 physiotherapist for a band 4 therapy assistant for physiotherapy sessions. This yielded a change in the cost saving from £80.39 to £24.45. Combining this scenario with an increase in the proportion of people having group physiotherapy sessions (from 50% to 75%) resulted in incurred costs of £17.32.

Costs of GaitSmart and standard care are the key cost drivers in the economic models

3.12

For the falls model, 70% of the base-case cost difference was because of the relative costs of the interventions. So, by far the most important economic input to the model was the cost of the comparator. For GaitSmart to be cost neutral or cost saving, the cost of comparator needed to be very close to the cost of GaitSmart or higher. Falls can have a significant impact on people who have them and on the NHS. But GaitSmart resulted in a relatively small reduction in the number of falls (11%) in the model, so the modelled impact of falls on cost saving was small. In the rehabilitation model, overall cost saving was also dominated by the cost difference between interventions, while a marginal number of falls were prevented by GaitSmart. The impact of falls in the model was limited by the short duration.

  • National Institute for Health and Care Excellence (NICE)