| Recurring |
one_organization, multiple_organization |
(a) The software failure incident having happened again at one_organization:
- North Tyneside council had to drop TransUnion's system used for checking housing and council tax benefit claims due to erroneous predictive analytics causing delays in welfare payments [91514].
- Hackney council in east London also dropped Xantura's system from a project to predict child abuse, stating that it did not deliver the expected benefits [91514].
(b) The software failure incident having happened again at multiple_organization:
- Sunderland city council did not renew a £4.5m data analytics contract for an "intelligence hub" provided by Palantir due to reasons not disclosed [91514].
- Concerns have been raised about the reliability and effectiveness of the software systems provided by various companies like Experian, TransUnion, Capita, and Palantir to local authorities for automated decision-making on benefit claims, child abuse prediction, and other welfare issues [91514]. |
| Phase (Design/Operation) |
design, operation |
(a) The software failure incident related to the design phase can be seen in the article where North Tyneside council dropped TransUnion's system used for housing and council tax benefit claims. The system's "predictive analytics" erroneously identified low-risk claims as high risk, leading to welfare payments being wrongly delayed. The council report highlighted that the system provided no reason for cases meeting a high-risk category, and the reasons for being high risk could not be established, causing delays in benefit claims [91514].
(b) The software failure incident related to the operation phase is evident in the case of Hackney council dropping Xantura from a project to predict child abuse. The council stated that the system did not deliver the expected benefits due to issues of variable data quality, indicating a failure in the operation or utilization of the system for providing useful insights [91514]. |
| Boundary (Internal/External) |
within_system |
(a) The software failure incident reported in the articles can be categorized as within_system. The incidents mentioned, such as the erroneous identification of low-risk claims as high risk by TransUnion's system leading to delayed welfare payments [91514], the inability of Xantura's system to provide sufficiently useful insights due to variable data quality [91514], and the lack of reliable predictive models due to issues with accessing regular updates of source data [91514], all point to failures originating from within the system itself. These issues highlight the challenges and limitations of the algorithms and machine-learning systems being used by councils for decision-making processes. |
| Nature (Human/Non-human) |
non-human_actions, human_actions |
(a) The software failure incident occurring due to non-human actions:
The article reports incidents where software systems provided erroneous results without human involvement. For example, in the case of North Tyneside council using TransUnion's system, the "predictive analytics" erroneously identified low-risk claims as high risk, leading to wrongly delayed welfare payments [91514]. This indicates a failure caused by non-human actions, specifically the inaccurate predictions made by the software system.
(b) The software failure incident occurring due to human actions:
The article also highlights instances where human actions contributed to software failures. For instance, Hackney council in east London dropped Xantura from a project to predict child abuse due to the system not delivering the expected benefits [91514]. This decision to discontinue the project was a result of human actions taken based on the system's performance, indicating a failure influenced by human decisions. |
| Dimension (Hardware/Software) |
hardware, software |
(a) The software failure incident related to hardware:
- The article mentions that North Tyneside council experienced a software failure incident where welfare payments to an unknown number of people were wrongly delayed due to the computer's "predictive analytics" erroneously identifying low-risk claims as high risk. This incident was attributed to TransUnion's system, which automatically processed data about claimants for housing and council tax benefit to determine the likelihood of fraud, indicating a hardware-related failure [91514].
(b) The software failure incident related to software:
- The article highlights that Hackney council in east London dropped Xantura from a project to predict child abuse and intervene before it happens, stating that Xantura's system did not deliver the expected benefits. This failure was attributed to the software provided by Xantura, indicating a software-related failure [91514]. |
| Objective (Malicious/Non-malicious) |
non-malicious |
(a) The software failure incident described in the articles does not seem to be malicious, as there is no indication that the failure was due to contributing factors introduced by humans with the intent to harm the system. The incidents mentioned, such as delays in benefit claims and lack of expected benefits in predicting child abuse, appear to be a result of non-malicious factors like unreliable algorithms, lack of understanding of the systems, and issues with data quality [91514]. |
| Intent (Poor/Accidental Decisions) |
poor_decisions, accidental_decisions |
(a) The intent of the software failure incident related to poor decisions can be seen in the article where it mentions that some councils invested in software contracts without fully understanding how the systems work. Gwilym Morris, a management consultant, highlighted that the complexity of the systems meant that the leadership of local authorities "don’t really understand what is going on" [Article 91514]. This lack of understanding raises questions about how citizens' data was used and indicates poor decision-making in implementing these software systems.
(b) The intent of the software failure incident related to accidental decisions is evident in the case of North Tyneside council using TransUnion's system for benefit claims. The system erroneously identified low-risk claims as high risk, leading to welfare payments being wrongly delayed. The council report concluded that most cases deemed high risk by the software were actually lower risk, and there was no reason for the payment to be withheld, but claims were delayed due to the system's errors [Article 91514]. This incident reflects a failure caused by unintended decisions or mistakes within the software system. |
| Capability (Incompetence/Accidental) |
development_incompetence |
(a) The software failure incident related to development incompetence is evident in the case of North Tyneside council using TransUnion's system for benefit claims. The system erroneously identified low-risk claims as high risk, leading to delays in welfare payments to individuals. The council report highlighted that the software provided no reason for cases meeting a high-risk category, and the reasons for being classified as high risk could not be established, causing delays in benefit claims [91514].
(b) The software failure incident related to accidental factors is seen in the case of Hackney council dropping Xantura from a project to predict child abuse. The council mentioned that the system did not deliver the expected benefits, indicating that the failure was not intentional but rather a result of the system's inability to provide useful insights due to issues of variable data quality [91514]. |
| Duration |
temporary |
The articles do not provide specific information about the duration of the software failure incident in terms of being permanent or temporary. The incidents mentioned in the articles highlight issues with the reliability and effectiveness of the software systems being used by councils for various purposes, such as benefit claims and child abuse prediction. These incidents point towards failures caused by contributing factors introduced by certain circumstances, such as inaccuracies in predictive analytics leading to delays in benefit payments and the inability of the system to provide useful insights for child abuse prediction [91514]. |
| Behaviour |
crash, omission, value, other |
(a) crash: The software failure incident related to the crash behavior is exemplified by the case of North Tyneside council dropping TransUnion's system, which erroneously delayed welfare payments to individuals due to the system's "predictive analytics" wrongly identifying low-risk claims as high risk, leading to benefit claims being wrongly delayed [91514].
(b) omission: The software failure incident related to the omission behavior is demonstrated by Hackney council dropping Xantura from a project to predict child abuse, stating that the system did not deliver the expected benefits, indicating an omission in providing useful insights [91514].
(c) timing: The software failure incident related to the timing behavior is not explicitly mentioned in the provided article.
(d) value: The software failure incident related to the value behavior is illustrated by the case of North Tyneside council's system by TransUnion incorrectly classifying cases as high risk when they were actually lower risk, resulting in benefit claims being wrongly delayed, indicating a failure in performing the intended functions correctly [91514].
(e) byzantine: The software failure incident related to the byzantine behavior is not explicitly mentioned in the provided article.
(f) other: The software failure incident related to other behavior includes concerns raised about privacy and data security, the difficulty for citizens in challenging automated decisions, and the lack of understanding by council officials on how some of the systems work, raising questions about how citizens' data is used [91514]. |