Guidance
3 Clinical evidence
Summary of clinical evidence
3.1 More detailed information on the clinical outcomes and evidence considered by the Committee is in the assessment report overview.
3.2 The clinical outcomes for VibraTip presented in the decision problem were:
-
sensitivity and specificity in assessment of vibration perception and/or light touch
-
sensitivity and specificity in assessment of grade of neuropathy
-
inter‑rater agreement of assessment of grade of neuropathy
-
accuracy of risk assessment in ulcer formation
-
ulcer formation and amputation
-
time taken for sensory testing
-
quality of life
-
device-related adverse events.
3.3 The sponsor's submission identified 9 studies: 4 journal papers, 2 conference abstracts, 2 unpublished studies and a technical study. The sponsor excluded the technical study and therefore presented 8 studies that were relevant to the scope. A literature search by the External Assessment Centre identified 2 additional studies (Bracewell et al. 2011, Baker 2012), as well as the 9 published studies presented by the sponsor. The External Assessment Centre considered that 6 of the 11 studies presented unique patient data relevant to the scope: 4 papers (Levy 2010, Bowling et al. 2012, Bracewell et al. 2012, Nizar et al. 2014) and 2 abstracts (Urbancic‑Rovan et al. 2012, Garbas et al. 2013). The External Assessment Centre excluded the other 5 studies from further consideration: Bracewell et al. (2011) was a conference abstract which overlapped with the Bracewell et al. (2012) study; Baker (2012) was a commentary on Bowling et al. (2012) and Bracewell et al. (2012); and Horsfield and Levy (2013) and Horsfield and Levy (unpublished) were technical papers with no information of direct relevance to the scope. However, they contain potentially useful information on the device's battery life and likely useable lifetime in clinical practice which the External Assessment Centre summarised in an appendix to the assessment report. Finally, a study by Levy and Greenwood was excluded because the intervention was outside the scope (VibraTip was used by patients in their own homes). All 6 relevant studies were diagnostic accuracy studies.
3.4 Bowling et al. (2012) was a cross‑sectional diagnostic accuracy study that compared 2 index tests (VibraTip and the Ipswich touch test) with each of 2 reference standards: a neurothesiometer (vibration perception threshold ≥25 V) or the Neuropathy Disability Score, which is a composite outcome derived from pain sensation, vibration sensation, temperature sensation and ankle reflex. People (n=83) attending diabetes outpatient clinics in hospital and community settings in the UK were assessed using all 4 methods. The results showed that VibraTip had good agreement with the vibration perception threshold in the neurothesiometer (Cohen's kappa=0.973, p<0.001) and with the Neuropathy Disability Score (Cohen's kappa=0.921, p<0.001). The External Assessment Centre calculated that relative to the neurothesiometer, VibraTip's sensitivity was 1.00 (95% confidence interval [CI] 0.93 to 1.00) and its specificity was 0.97 (95% CI 0.82 to 1.00).
3.5 Bracewell et al. (2012) was a cross-sectional diagnostic accuracy study that compared 4 index tests (VibraTip, NeuroTip [a neurological examination pin which can exert a calibrated force], 10 g monofilament and 128 Hz tuning fork) with a neurothesiometer as a reference standard. It also attempted to establish the number of insensate sites that optimised accuracy for each test. The study population was 141 people with diabetes type 1 or 2 in secondary care in the UK, with a reported prevalence of diabetic peripheral neuropathy of 41%. The first part of the study tested intra‑rater reliability of VibraTip in a population of 18 people with diabetes and at high risk of diabetic peripheral neuropathy (note: inter‑rater reliability was the outcome specified in the scope). Results from successive readings, taken 2‑3 weeks apart, demonstrated good intra‑rater reliability (Cronbach's alpha = 0.88, no CI given). The main part of the study tested the comparative diagnostic accuracy of the 4 index tests compared with the neurothesiometer. Of the 141 people recruited from a secondary care setting, 89% reported having no history of foot ulcers. The authors performed a receiver‑operator characteristic analysis to find the optimum number of insensate sites which gave the best diagnostic accuracy for each test, and found that 2 or more out of 10 were optimal for VibraTip, 10 g monofilament and NeuroTip, while 1 or more was optimal for the tuning fork. From the results provided, the External Assessment Centre calculated VibraTip's sensitivity as 0.79 (95% CI 0.69 to 0.90) and its specificity as 0.82 (95% CI 0.74 to 0.90). Results for the 10 g monofilament were sensitivity 0.84 (0.75 to 0.94) and specificity 0.83 (0.75 to 0.91), and for the 128 Hz tuning fork were sensitivity 0.69 (0.57 to 0.81) and specificity 0.90 (0.84 to 0.97).
3.6 Levy (2010) was a cross‑sectional study that compared 3 diagnostic devices in 100 people with diabetes having their annual review in a hospital or podiatry clinic in the UK. The aim of the study was to measure the level of agreement between VibraTip, a 10 g monofilament and a 128 Hz tuning fork. Agreement data between the tests were reported and the External Assessment Centre analysed the results, which showed no statistically significant difference between the tests.
3.7 Nizar et al. (2014) compared 2 index tests (VibraTip and a tuning fork) with a neurothesiometer as the reference standard. The study reported tests on 100 people with type 1 or 2 diabetes attending specialist clinics. Although the authors described the study as a 'cross‑sectional diagnostic' design, it was based on a case‑control design in which the researchers had prior knowledge of the patients' diabetic peripheral neuropathy status, and recruited them accordingly to make the prevalence of diabetic peripheral neuropathy exactly 50%. Results from the study show that the sensitivity of VibraTip was 0.92 (95% CI 0.81 to 0.98) and its specificity was 0.94 (95% CI 0.83 to 0.99). The authors concluded that VibraTip is comparable to the neurothesiometer and superior to the tuning fork in the detection of peripheral neuropathy, and that it could therefore be a useful screening tool in clinical practice.
3.8 Urbancic-Rovan et al. (2012) was a small pilot study reported as a conference abstract, which compared 5 index tests (VibraTip, 128 Hz tuning fork, 10 g monofilament, Tip Therm [which detects impaired skin temperature sensation] and Neuropad [described as a simple and cheap diagnostic tool for the evaluation of sweat gland function]) in 42 people attending diabetes outpatient clinics in Slovenia. The results suggested that the 10 g monofilament had a much poorer sensitivity (positive in only 14.3% tests) compared with the other tests, including VibraTip (positive in 47.6% of tests).
3.9 Garbas et al. (2013) was a follow-up to the pilot study by Urbancic‑Rovan et al. (2012) and is reported in a conference poster with few details. Based in a university medical centre in Slovenia, this large study (n=496) compared 2 index tests (VibraTip and 128 Hz tuning fork). The results indicate that there was no statistically significant inter-foot variability for either of the index tests and that the tuning fork was shown to be statistically significantly more sensitive than VibraTip in detecting impairment of vibration sensation.
3.10 The sponsor found no adverse event reports relating to VibraTip. No alerts have been issued and no information found in a search of the Medicines and Healthcare Products Regulatory Agency website.
3.11 The External Assessment Centre used the QUADAS‑2 tool (revised Quality Assessment of Diagnostic Accuracy Studies) to critique the 6 diagnostic accuracy studies and noted a number of limitations:
-
The risk of bias in all studies was high (poor description of the test procedures, lack of evidence of test application randomisation, lack of evidence that the testers were blinded to the patients' reference test results, biases in study populations).
-
Three of the studies (Levy 2010, Urbancic‑Rovan et al. 2012 and Garbas et al. 2013) lacked a reference test to diagnose diabetic peripheral neuropathy, making calculation of diagnostic accuracy parameters impossible.
-
The population in 1 study (Bowling et al. 2012) had diabetic peripheral neuropathy of varying severity already diagnosed, and its target condition was 'at risk' feet.
-
The study by Nizar et al. (2014) used a different reference standard to that of most other diabetic peripheral neuropathy studies (a neurothesiometer set at a threshold of 20 V rather than the widely applied 25 V). The same study reported diagnostic accuracy results for the tuning fork (frequency of the fork was not stated) which were not consistent with other published studies.
3.12 The External Assessment Centre considered that the Bracewell et al. (2012) study was the highest quality study and most closely matched the decision problem. It noted the results showed that there were no statistically significant differences in diagnostic accuracy between VibraTip and the 10 g monofilament or the 128 Hz tuning fork in the detection of peripheral neuropathy. The External Assessment Centre considered the optimisation of the thresholds for each device a limitation of this study, because it is not clear how generalisable these thresholds are to clinical practice. It also highlighted that it is unclear whether the study was sufficiently statistically powered to reliably conclude non‑inferiority.
Committee considerations
3.13 The Committee noted the clinical evidence base for VibraTip was 6 diagnostic accuracy studies. It agreed with the External Assessment Centre's opinion that the studies were of relatively low methodological quality and had a high risk of bias, but it recognised that the general quality of evidence in this clinical area is low. The Committee agreed with the External Assessment Centre that Bracewell et al. (2102) was the study with most relevance to the scope. It also agreed that, although it would appear that VibraTip has a diagnostic accuracy comparable with that of the 10 g monofilament and the tuning fork, there remained some uncertainties. The Committee judged that these uncertainties were important, because even small differences in diagnostic accuracy might have serious consequences concerning post-diagnosis outcomes for these patients. The Committee concluded that further evidence based on a high quality diagnostic accuracy study was needed to assess the clinical effectiveness of this technology.
3.14 The Committee discussed the comparators specified in the scope and received expert advice that the appropriate comparators for VibraTip are the 10 g monofilament and the calibrated tuning fork. Expert advice indicated the 10 g monofilament was routinely used in primary care, but practice varied in secondary care and could in some cases involve dual modalities measuring both touch and vibration sensation in testing for diabetic peripheral neuropathy. Clinical experts also agreed there was no accepted standard on the number or location of the sites to be tested on the foot. The Committee also heard expert advice that the neurothesiometer is the reference standard and that the appropriate threshold voltage varied with the patient's age. The Committee concluded that the variability in the use of the devices in the diagnosis of diabetic peripheral neuropathy was an additional challenge to the collection of high quality diagnostic accuracy information.
3.15 The Committee questioned the equivalence of touch sensation and vibration perception in the diagnosis of diabetic peripheral neuropathy. It was aware the foot sensory nerve function assessment, which is part of the annual foot examination described in NICE's guideline on foot problems in type 2 diabetes, recommends either a touch sensation or vibration perception assessment. The clinical experts explained that testing vibration perception and touch sensation could be used separately or together to explore different conditions relevant to diabetic peripheral neuropathy and the diabetic foot at risk of ulceration, and that touch and vibration involved different nerve pathways. The Committee considered that research investigating the different diagnostic testing methodologies (VibraTip, the 10 g monofilament and the calibrated tuning fork) using nerve conduction measurements was feasible and had the potential to define the relative importance of impaired touch sensation and/or vibration perception in the progression of neuropathy.
3.16 The Committee considered the lack of evidence for clinical outcomes and patient benefits associated with the use of VibraTip. It noted that serious adverse consequences of diabetic peripheral neuropathy, such as ulcer formation and limb amputation, take several years to manifest and studies to directly measure these outcomes would be difficult to conduct.