Incident: Inaccurate Diagnoses and Advice on NHS Choices Symptom Checker

Published Date: 2015-07-09

Postmortem Analysis
Timeline 1. The software failure incident happened in July 2015.
System 1. Symptom checker software algorithms [38021]
Responsible Organization 1. The software algorithms used in the symptom checkers evaluated by the Harvard Medical School experts were responsible for causing the software failure incident [38021].
Impacted Organization 1. Patients seeking medical advice online [38021]
Software Causes 1. Inaccurate software algorithms used by the symptom checkers, leading to missing emergencies and providing wrong advice to patients [38021].
Non-software Causes 1. The study conducted by experts at Harvard Medical School found that dangerous diseases and health emergencies were often missed on 'symptom checker' websites, indicating a potential issue with the accuracy of the medical information provided [38021]. 2. The research highlighted that online programs often give patients the wrong advice about what action to take, suggesting a problem with the guidance provided by these websites [38021]. 3. The study revealed that there was a great deal of variation between different symptom checkers, with none being without limitations, indicating inconsistencies in the performance and reliability of these tools [38021].
Impacts 1. The software failure incident led to up to one in eight emergencies being missed by the flagship NHS Choices symptom checker website, casting doubt on the safety of relying on the internet for diagnosing illnesses [38021]. 2. The study found that dangerous diseases and health emergencies were often missed on 'symptom checker' websites, indicating a potential risk to users who rely on online programs for medical advice [38021]. 3. The software algorithms studied listed the correct diagnosis first in only 34 percent of cases, highlighting a significant failure in accurately diagnosing medical conditions [38021]. 4. Overall, the NHS Choices site correctly identified 87 percent of emergencies but still missed 13 percent, showing that even the best-performing site had limitations in providing accurate medical advice [38021]. 5. The incident emphasized the importance of caution when using online symptom checkers, as users should not take the information received from these platforms as definitive medical advice [38021].
Preventions 1. Implementing more rigorous testing procedures for the software algorithms used in the symptom checker websites to ensure accurate diagnoses and appropriate advice [38021]. 2. Conducting regular reviews and updates of the symptom checker software based on clinical evidence and national guidance recommendations to enhance accuracy and reliability [38021]. 3. Enhancing the training and education of the software developers to improve the design and functionality of the symptom checker websites [38021].
Fixes 1. Implement regular reviews and updates of the symptom checker software based on clinical evidence and national guidance to improve accuracy and reliability [38021]. 2. Enhance the software algorithms to increase the percentage of correct diagnoses listed first, in the top three, and in the top 20 to provide more accurate results [38021]. 3. Focus on improving the advice given by the symptom checker software to ensure patients receive the correct guidance on whether and how quickly to seek medical attention [38021]. 4. Address limitations in the existing symptom checker software by learning from tools with more accurate diagnoses and better recommendations for care, such as Isabel, iTriage, Mayo Clinic, Healthychildren.org, Steps2Care, and Symptify [38021].
References 1. The study conducted by experts at Harvard Medical School in the US [38021] 2. The research published in the British Medical Journal [38021] 3. Professor Ateev Mehrotra, who led the research [38021] 4. NHS Choices website [38021]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) The software failure incident having happened again at one_organization: The NHS Choices symptom checker website, run by the British health service, was found to have missed up to one in eight emergencies according to a study conducted by experts at Harvard Medical School [38021]. The study evaluated 23 leading health websites and found that dangerous diseases and health emergencies were often missed on 'symptom checker' websites, including the NHS Choices site. The site correctly identified 87% of emergencies but still missed 13% of them, giving the correct advice in only 52% of cases [38021]. (b) The software failure incident having happened again at multiple_organization: The study conducted by Harvard Medical School evaluated 23 leading health websites, including the NHS Choices site, and found that online programs often give patients the wrong advice about what action to take. The software algorithms studied listed the correct diagnosis first in only 34% of cases, indicating a common issue across various health websites [38021].
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase can be seen in the article where it discusses the study conducted by experts at Harvard Medical School evaluating 23 leading health websites, including the NHS Choices site. The study found that dangerous diseases and health emergencies were often missed on 'symptom checker' websites, indicating a failure in the design of these online programs to accurately diagnose illnesses [38021]. (b) The software failure incident related to the operation phase is evident in the article where it mentions that the NHS Choices site correctly identified 87% of emergencies but still missed 13%. Overall, the site gave the correct advice in only 52% of cases, indicating a failure in the operation or functioning of the system when providing advice to users based on their symptoms [38021].
Boundary (Internal/External) within_system (a) within_system: The software failure incident related to the NHS Choices symptom checker website missing up to one in eight emergencies was primarily due to factors within the system. The study conducted by experts at Harvard Medical School evaluated 23 leading health websites, including the NHS Choices site, and found that dangerous diseases and health emergencies were often missed by the symptom checker websites [38021]. The study revealed that the software algorithms used by the symptom checkers listed the correct diagnosis first in only 34 percent of cases, indicating a failure within the system in accurately identifying emergencies and providing correct advice to users. Additionally, the researchers highlighted that there was a great deal of variation between the different symptom checkers, suggesting internal issues within the software systems [38021]. (b) outside_system: The software failure incident related to the NHS Choices symptom checker website missing up to one in eight emergencies was not primarily due to factors originating from outside the system. The study conducted by experts at Harvard Medical School focused on evaluating the performance of various health websites, including the NHS Choices site, in accurately identifying emergencies and providing appropriate advice to users [38021]. The research did not highlight external factors as the primary cause of the software failure incident but rather emphasized the limitations and variations within the software algorithms and the need for caution when relying on online symptom checkers for medical advice.
Nature (Human/Non-human) unknown (a) The software failure incident occurring due to non-human actions: The software failure in this case was not due to non-human actions but rather due to the limitations and inaccuracies of the software algorithms used in the symptom checkers. The study conducted by experts at Harvard Medical School evaluated 23 leading health websites and found that dangerous diseases and health emergencies were often missed by these 'symptom checker' websites. The software algorithms listed the correct diagnosis first in only 34 percent of cases, indicating a failure in accurately identifying emergencies without any direct non-human actions involved [38021]. (b) The software failure incident occurring due to human actions: The software failure in this case was not directly attributed to human actions but rather to the limitations of the software algorithms used in the symptom checkers. The study conducted by experts at Harvard Medical School evaluated 23 leading health websites and found that these online programs often gave patients the wrong advice about what action to take. While human actions were involved in creating and maintaining these symptom checkers, the failure was more related to the accuracy and effectiveness of the software algorithms rather than specific human actions leading to the failure [38021].
Dimension (Hardware/Software) software (a) The articles do not mention any software failure incident occurring due to contributing factors originating in hardware. Hence, there is no information available regarding a hardware-related failure incident. (b) The software failure incident mentioned in the articles is related to online symptom checkers used for diagnosing illnesses. The study conducted by experts at Harvard Medical School evaluated 23 leading health websites, including the NHS Choices site, and found that dangerous diseases and health emergencies were often missed by these 'symptom checker' websites. The software algorithms used by these websites listed the correct diagnosis first in only 34% of cases, and the correct advice about what action to take was often incorrect. The study highlighted the limitations and inaccuracies of these online symptom checkers, emphasizing the importance of caution when relying on internet advice for medical diagnoses [38021].
Objective (Malicious/Non-malicious) non-malicious (a) The articles do not mention any software failure incident related to malicious intent or hacking. [38021] (b) The software failure incident discussed in the articles is non-malicious. It pertains to the limitations and inaccuracies of online symptom checkers in diagnosing illnesses and providing appropriate medical advice. The failure is attributed to the algorithms and design of the software rather than any malicious intent. The study conducted by experts at Harvard Medical School highlighted the challenges and risks associated with relying solely on internet-based symptom checkers for medical diagnosis. The incident underscores the importance of caution when using such tools and the need for continuous improvement in their accuracy and effectiveness.
Intent (Poor/Accidental Decisions) poor_decisions (a) The intent of the software failure incident related to poor decisions can be inferred from the article. The failure of the NHS Choices symptom checker website to accurately identify emergencies and provide correct advice can be attributed to poor decisions in the design and implementation of the software algorithms. The study conducted by experts at Harvard Medical School highlighted that the online programs often gave patients the wrong advice about what action to take, indicating a flaw in the decision-making process behind the software [38021].
Capability (Incompetence/Accidental) development_incompetence (a) The software failure incident related to development incompetence is evident in the study conducted by experts at Harvard Medical School, which evaluated 23 leading health websites, including the flagship NHS Choices symptom checker website. The study found that up to one in eight emergencies were missed by the NHS Choices site, and overall, the site gave the correct advice in only 52% of cases [38021]. (b) The software failure incident related to accidental factors is highlighted in the findings of the study conducted by Harvard scientists, where they input standardized lists of symptoms into 23 different symptom checkers to test their accuracy. The study revealed that the software algorithms listed the correct diagnosis first in only 34% of cases, indicating a significant number of inaccuracies in the results provided by the symptom checkers [38021].
Duration unknown The software failure incident discussed in the articles does not directly relate to a temporary or permanent failure of the software itself. Instead, the focus is on the accuracy and reliability of online symptom checkers in diagnosing illnesses and providing appropriate advice to users. Therefore, the articles do not provide information on the duration of the software failure incident in terms of being temporary or permanent.
Behaviour crash, omission, value, other (a) crash: The software failure incident related to the NHS Choices symptom checker website can be categorized as a crash. The study conducted by experts at Harvard Medical School found that up to one in eight emergencies were missed by the NHS Choices symptom checker website, indicating a failure of the system to perform its intended function correctly [38021]. (b) omission: The incident also involves omission as a behavior of the software failure. The study revealed that dangerous diseases and health emergencies were often missed on 'symptom checker' websites, including the NHS Choices site, leading to the omission of critical information that should have been provided to users [38021]. (c) timing: The timing behavior is not explicitly mentioned in the articles provided. (d) value: The software failure incident can be associated with the value behavior as well. The study found that online programs, including the symptom checkers, often gave patients the wrong advice about what action to take, indicating a failure in providing correct and valuable information to users [38021]. (e) byzantine: The byzantine behavior is not described in the articles provided. (f) other: The software failure incident can also be categorized under the "other" behavior as it involves providing incorrect advice to users, not always listing the correct diagnosis first, and not always offering the best advice for care despite accurate diagnoses in some cases [38021].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence theoretical_consequence (a) death: The articles do not mention any deaths resulting from the software failure incident [38021]. (b) harm: The software failure incident did not directly result in physical harm to individuals [38021]. (c) basic: There is no mention of people's access to food or shelter being impacted by the software failure incident [38021]. (d) property: The software failure incident did not lead to any direct impact on people's material goods, money, or data [38021]. (e) delay: The software failure incident did not cause any delays in activities [38021]. (f) non-human: The software failure incident did not have any impact on non-human entities [38021]. (g) no_consequence: The articles do not mention any real observed consequences of the software failure incident [38021]. (h) theoretical_consequence: The articles discuss potential consequences of the software failure incident, such as missing emergencies and giving incorrect advice, but do not mention any actual occurrences of these consequences [38021]. (i) other: There are no other consequences of the software failure incident described in the articles [38021].
Domain information, health (a) The failed system was related to the health industry. The software failure incident discussed in the articles pertains to the NHS Choices symptom checker website, which is a tool designed to help users diagnose their health symptoms and determine the appropriate course of action [38021]. The study conducted by experts at Harvard Medical School evaluated 23 leading health websites, including the NHS Choices site, to assess their accuracy in identifying emergencies and providing correct advice to users [38021]. The research highlighted that up to one in eight emergencies were missed by the NHS Choices symptom checker website, indicating a failure in accurately diagnosing critical health conditions [38021].

Sources

Back to List