Genedrive MT-RNR1 ID Kit for detecting a genetic variant to guide antibiotic use and prevent hearing loss in babies: early value assessment

3.3 Evidence on time to antibiotic treatment from the PALOH study (McDermott et al. 2022) showed no statistically significant difference between standard care and when using the Genedrive test. The PALOH study was a prospective observational implementation trial based in 2 large specialist neonatal intensive care units. But most of the data (94% of babies recruited) was from 1 of these units. A clinical expert said that it was uncertain if the result from the PALOH study was generalisable to other settings, such as smaller non-specialist centres or outside of a neonatal intensive care unit (on a labour ward, for example). There may be differences in how wards are set up and the experience of staff in implementing new technologies, or there may be more demands on staff time. The committee noted that one of the centres in the PALOH study now routinely tests all babies on admission to the neonatal intensive care unit. But it said that individual centres should decide which babies to test depending on local knowledge of what proportion of babies go on to receive aminoglycosides. The clinical expert explained that time to antibiotic treatment is very important for the efficacy of the antibiotics, but it is often a challenge in standard care to ensure that antibiotics are administered within 1 hour. However, another clinical expert said that, of those babies that were prescribed antibiotics based on clinical suspicion of infection (10% to 12% of all babies), only a small proportion would benefit from this treatment being delivered within the hour (around 1%). This is because in a lot of cases, with hindsight, the antibiotics were not needed because there was no infection. However, they are given because the consequences of missing the small proportion who benefit would be severe. The committee agreed that further evidence in alternative settings was needed, but it also agreed the risk was low if the test was introduced while further evidence was generated on time to antibiotics. It concluded that the benefits of using the test and preventing aminoglycoside-induced hearing loss in susceptible babies outweighed this risk.

3.4 A clinical expert explained that the reason gentamicin is the first-choice antibiotic is because it is active against a wide range of bacteria (Gram-negative bacteria and Staphylococcus aureus) and has a low risk of increasing antibiotic resistance. They said that other antibiotics such as cefotaxime (a cephalosporin) are equally effective and have a better safety profile but are not the first choice because using them can lead to an increase in antibiotic resistance. In the PALOH study, babies that needed antibiotics and were found to have the m.1555A>G variant were treated with cefotaxime. A clinical expert said that if the test was implemented, failed tests and false positive results could lead to more widespread use of alternative antibiotics that may promote antibiotic resistance. The committee said that this would not be a concern if the failure rate and false positive rates remained low, but noted that there was some uncertainty in this data (see the sections on test failure rate and diagnostic accuracy). The committee said that this should be monitored closely and alternative antibiotics should be used when the test is positive. When the test fails and there is no time for a second test to be done, NICE's guideline on neonatal infection should be followed alongside local prescribing protocols. The committee concluded that, although alternative antibiotics are associated with increased antibiotic resistance, a relatively small number of babies would be treated with them if the real-world false positive rate was low. Therefore, if the Genedrive test was implemented, it should not lead to increased antibiotic resistance in neonatal units. The committee also concluded that the alternative antibiotics were likely to be equally effective, so there were no concerns around treating babies with an alternative if they have the m.1555A>G variant.

3.5 The PALOH study reported an initial failure rate of 17.1% for the Genedrive test. This was reduced to 5.7% after modifications to the test buffer. The committee said that it had some concerns about the test being modified during the study and whether the reported improvement in the failure rate would be replicated in real-world use. A clinical expert said that if the test failed, clinicians may decide to use an alternative antibiotic. In many cases, this would be unnecessary and could promote the spread of antibiotic resistance. They said that failed tests could also lead to delays to antibiotic treatment. However, the company said the PALOH study was an implementation study and the results were used to update the test. The updated test is currently in use at Saint Mary's Hospital in Manchester and is the commercially available Genedrive MT‑RNR1 ID Kit. Comments from the Manchester Centre for Genomic Medicine said that the current failure rate in real-world use since PALOH is 1.81% (95% confidence interval [CI] 0.6% to 5.18%). The committee noted that this was promising but was based on data from a large specialist neonatal intensive care unit and may not generalise to all settings where the test may be used. It said that further data should be collected on the real-world failure rate of the Genedrive test in different settings. The committee concluded that, although there was some uncertainty around the failure rate of the test, this did not present a risk to the babies being tested and the test could reduce the risk of aminoglycoside-induced hearing loss. It also concluded that data on the failure rate could be collected as part of real-world evidence generation.

3.6 No false negative results were reported in the PALOH study, but only 3 babies were identified with the m.1555A>G variant (true positives). So, although the reported sensitivity estimate was high at 100%, there was still considerable uncertainty about the value (95% CI 29.2% to 100%). The committee noted that it would be difficult to reduce this uncertainty without a very large study because of the low prevalence of the m.1555A>G variant in the population (around 0.19%). A clinical expert explained that hearing loss can occur soon after exposure to aminoglycosides, but in some babies it might be after they have passed the newborn hearing test. For babies that develop hearing loss later, it can be more difficult to link it to their exposure to aminoglycosides, so it may be difficult to follow up and identify babies that had a false negative result in a real-world setting. But the clinical experts said that they were not concerned about false negative results because at the moment there is no testing at all. So even with a risk of false negatives, any testing improves the chance of babies with the m.1555A>G variant being identified and reduces the risk of them receiving aminoglycosides. The PALOH study also reported 5 false positive results. The authors said that this was corrected with an updated test cartridge design. The committee said that false positive results did not pose any particular risk to babies because they would be treated with an alternative, equally effective, antibiotic. However, it reiterated its concerns about the test being updated during the study and noted that it was uncertain whether changes made to reduce the false positive rate could affect the sensitivity. The committee concluded that there was some uncertainty in the estimated sensitivity of the Genedrive test. It said that post-market surveillance could be used to follow up babies that test negative on the Genedrive test but later go on to develop hearing loss. Retrospective laboratory testing of these babies, in addition to routine confirmatory testing of babies with a positive result, should be done to help improve the precision of the sensitivity estimate and confirm the real-world false positive rate.

3.7 The external assessment group (EAG) said that there was clear evidence that the m.1555A>G variant is a risk factor for aminoglycoside-induced hearing loss. However, the evidence is from case-control studies that may overestimate this risk, so the precise level of risk is uncertain. The committee said that it would be difficult to use an alternative study design because of ethical concerns, so it would likely always be uncertain. Other variants in the MT‑RNR1 gene are also associated with a risk of aminoglycoside-induced hearing loss. Therefore, it considered that babies who test negative for the m.1555A>G variant but still go on to develop hearing loss should be followed up with laboratory testing to determine if they have had a false negative result. Depending on developments in laboratory testing, in the future it may also be possible to determine if they have a different MT‑RNR1 variant. A clinical expert said that mitochondrial whole genome sequencing could also be considered to look for different variants if clinically indicated. The committee concluded that, although there was some uncertainty around the risk and severity of hearing loss, further evidence generation would be unlikely to address this.

3.8 Implementing the Genedrive test with real-world data collection would incur upfront costs that would be lost to the NHS (that is, sunken costs) if it is later shown not to be value for money. The committee considered that these upfront costs included the Genedrive MT‑RNR1 ID system (£4,995) and Bluetooth printer (£400). A clinical expert said that centres may need more than 1 Genedrive system to successfully implement the test, and the company noted that Saint Mary's Hospital in Manchester has 2 systems in use. The EAG noted that this would not substantially affect the cost per test because of the high volume of testing and expected lifespan of the equipment. However, the committee concluded that the upfront costs of implementing the Genedrive test should be carefully considered by commissioners. It noted that purchase options not involving large capital investment costs should be explored for any conditional recommendation and real-world data collection.

3.9 The EAG's early economic model made a number of key assumptions. It assumed no effect on time to antibiotic treatment when using the Genedrive test, so the consequences of any delay were not included in the economic analysis (see the section on time to antibiotic treatment). The EAG explained that it made a pragmatic decision to assume no effect on the time to antibiotics (as reported in the PALOH study). It also said that the data on time to antibiotics and outcomes was uncertain. The committee said that any future model for a full diagnostics evaluation and guidance should include time to antibiotic treatment and explore how any delays to antibiotics affect clinical outcomes. The model used the high diagnostic accuracy estimates reported in the PALOH study, which the committee recalled had some uncertainty around them (see the section on diagnostic accuracy). The committee noted that the early model also assumed that all babies with the m.1555A>G variant treated with aminoglycosides had severe or profound hearing loss. It said that the true risk and severity of deafness was difficult to estimate and would likely always be uncertain because it would be difficult to study (see the section on risk of aminoglycoside-induced hearing loss). The committee concluded that, overall, the model assumptions were reasonable, but that any future model to be used for full guidance recommendations should include time to antibiotic treatment and its associated outcomes.

3.10 The committee noted the high cost of identifying 1 baby with the m.1555A>G variant because it is relatively uncommon in the population. But the EAG's early economic model showed that the Genedrive test has the potential to be cost effective because over the lifetime of the baby, it is cheaper and more effective than standard care. The committee said that this was based on a number of assumptions, as previously mentioned, but it understood that hearing loss is associated with substantial healthcare costs and can have a major impact on quality of life. It also noted that the sensitivity analysis showed that changing these assumptions was unlikely to change the model conclusions. A patient expert highlighted the wider societal costs and educational impact of hearing loss. The committee noted that these costs were not captured in the model. It said that, although these are important considerations, they are outside of the NICE reference case. The committee also considered the potential additional benefits of avoiding aminoglycoside exposure in the future but noted that these were not included in the cost-effectiveness analysis. The committee concluded that, based on the early economic model results, the Genedrive test had the potential to be cost effective over a lifetime.

3.11 The prevalence of the m.1555A>G variant varies between different ethnicities, so the committee considered that any further evidence generation should ensure that the Genedrive test is implemented in centres with babies from different patient demographics, for example with different proportions of patients from diverse ethnicities (see the section on equality considerations).

3.12 The committee said that further evidence generation should include smaller, non-specialist centres and other settings outside of neonatal intensive care units where the test may be used. This is to ensure that the evidence generated can help assess if the time to antibiotic treatment, test failure rate and diagnostic accuracy estimates reported in the PALOH study are generalisable to other settings. This will also reveal what effect test implementation has on antibiotic prescribing decisions in different centres and settings.

3.13 The committee said that if the test was implemented for further evidence generation, this should be done in a wide range of geographical regions to ensure equal access to patients from different socioeconomic groups.