Recent studies have shown that more of us than ever before are turning to online symptom checkers and digital tools to manage and improve our health – but just how accurate are these new digital tools? We’ll explain new research undertaken by Methods Analytics, and how available symptom checkers match up.
- Overall test results showed significant room for improvement, with tested symptom checkers giving the correct diagnosis listed first in just 38% of cases.
- When compared against NICE guidelines, non-Doctorlink symptom checkers included in the survey recommended a higher level of care than necessary in over 50% of cases.
- Where self-care was the most appropriate solution, in 52% of cases symptom checker tools recommended patients use a healthcare service – compared to Doctorlink’s 12.5%.
- Study concluded that with the exception of high performing outliers like Doctorlink, symptom checker performance was significantly lacking.
Why use a symptom checker over a GP appointment?
Unsurprisingly, COVID-19 has played a part in accelerating digital adoption across the globe. A recent survey by McKinsey & Co highlighted that overall, digital adoption in Europe has accelerated by up to three years as a result of the pandemic. Further analysis also indicates that telehealth usage has since stabilised to be 38 times higher than the pre-COVID-19 baseline.
Source: McKinsey & Company
The rise of online symptom checkers
One of the ways digital tools can help is through the use of symptom checkers. These tools have the potential to help reduce the growing workload of clinicians by managing finite resources more effectively. A study in 2019 predicted that by 2028/29, there will be a shortfall of 11,500 general practitioners (GPs) in the UK[1] – posing a serious threat to the sustainability of primary care services.
These web or app-based tools can also help patients understand the likely causes of their symptoms. Some go further by also triaging patients – directing patients to the most appropriate care setting for their symptoms i.e. GP appointment, self-care at home, trip to A&E. The usage of symptom checker tools is also growing – and proving popular with patients. In fact, 80% of us see symptom checker tools as useful – and over 90% of us said we would use them again.[2]
Despite the growing number of these tools, only two studies have looked at tools that handle multiple conditions and multiple presentations of symptoms – and one of these studies was conducted as far back as 2015.[3]
Understanding performance
A new peer-reviewed study conducted by specialist independent data consultancy, Methods Analytics, set out to test symptom checker tools already available on the market. The research aimed to analyse:
- The performance of symptom checkers as diagnostic and triage tools.
- The potential impact of symptom checkers on clinical capacity and utilisation.
- The variations in performance across selected symptom trackers available today.
Researchers ran a series of 50 clinical cases through twelve publicly available symptom checkers to test their efficacy. The overall result showed significant room for improvement, with the correct diagnosis, listed first in just 38% of cases.
Doctorlink emerges best-of-breed in symptom checkers
One online symptom checker from Doctorlink, part of the HealthHero family of companies, was one of only two symptom checkers that proved to be consistently accurate – with the correct diagnosis listed first in 68% of cases and in the top 5 diagnoses in 86% of cases, versus the average of 51%.
Doctorlink continued to outperform other providers when measured for triage accuracy and safety. On the accuracy of its triage advice, Doctorlink ranked first – recommending the right level of care in 90% of cases compared to an average of 58% across symptom checkers. On the safety of its triage advice, Doctorlink scored 100% compared to an average of 83%.
And when reviewing the relationship between triage accuracy and patient safety, Doctorlink also scored highest. (Figure 1)
This relationship between triage accuracy and safety is a particularly important measurement, as a good digital triage or symptom checker tool needs to be both reliable and provide clinically appropriate advice with necessary safeguards in place. And the outcome of this study shows that Doctorlink has successfully struck the right balance.
Additionally, evidence from the study shows that Doctorlink was the most efficient in ensuring patient safety – without having a negative impact on healthcare resources. When compared against NICE guidelines, other symptom checkers included in the survey recommended a higher level of care than necessary in over 50% of cases. Doctorlink, however, only recommended a higher level of care than necessary in just 18% of cases. Furthermore, where self-care was the most appropriate solution, in 52% of cases symptom checker tools recommended patients use a healthcare service – compared to Doctorlink’s 12.5%.
Understanding whether online symptom checkers are accurately recommending the right level of care is key in analysing the impact digital tools have on the healthcare system. With clinical workloads continuing to increase, referring minor conditions to the healthcare system only places a further strain on finite resources.
Overall, the study concluded that with the exception of high performing outliers like Doctorlink, symptom checker performance was significantly below what would normally be expected of medical devices in any other field. And because they are typically risk averse and the accuracy varies considerably, the benefits symptom checkers bring to clinicians remains unclear in terms of how much they really mitigate capacity challenges.
A Need For Greater Rigor
Despite its own strong performance, Doctorlink is committed to championing greater rigour and assessment of tools like online symptom checkers to ensure that all such tools consistently deliver a safe service to patients, and prove a valuable tool for clinicians.
Dr Ben Littlewood-Hillsden, Chief Medical Officer of Doctorlink commented: “Healthcare is a high stakes industry, with clinical teams being responsible for the lives of their patients. This is an argument in itself for why we must mandate the use of proper tools that provide consistently accurate results. These have the potential to be transformational, but they simply aren’t at the moment because many are just not sophisticated enough to be truly fit-for- purpose.”
These findings also echo an ORCHA (Organisation for the Review of Care and Health Apps) report published earlier this year, which stated 80% of health apps available don’t meet standards for quality.
To learn more about Doctorlink and the performance of its online symptom checker, get in touch with us at enquires@healthhero.com.
[1] Beech J, Bottery S, Charlesworth A, Evans H, Gershlick B, Hemmings N, et al. Closing the Gap: Key Areas for Action on the Health and Care Workforce. London: The Health Foundation, The King’s Fund & Nuffield Trust, 2019 [Internet] KingsFund [cited 2020 Apr 12]. Available from: https://www.kingsfund.org.uk/publications/closing-gap-health-care-workforce
[2] Meyer AND, Giardina TD, Spitzmueller C, Shahid U, Scott TMT, Singh H. Patient Perspectives on the Usefulness of an Artificial Intelligence-Assisted Symptom Checker: Cross-Sectional Survey Study. J Med Internet Res. 2020;22(1):e14679. Published 2020 Jan 30. pmid:32012052
[3] Studies referenced include: Semigran HL, Linder JA, Gidengil C, Mehrotra A. Evaluation of symptom checkers for self diagnosis and triage: audit study. BMJ. 2015;351:h3480. pmid:26157077 and Gilbert S, Mehl A, Baluch A, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020;10(12):e040269. Published 2020 Dec 16. pmid:33328258
