Published Date: 2013-08-16
| Postmortem Analysis | |
|---|---|
| Timeline | 1. The software failure incident related to the Horizon system happened between 2000 and 2014 as mentioned in [Article 122321, Article 123006, Article 128018]. 2. The incident was also mentioned to have occurred between 2000 and 2014 in [Article 124605]. 3. The incident was further specified to have occurred between 2000 and 2015 in [Article 131957]. |
| System | 1. Horizon system - The Horizon system developed by Fujitsu failed due to defects, bugs, and flaws in the software [20678, 113495, 117646, 120774, 124053, 124605, 125084]. |
| Responsible Organization | 1. The Post Office [20678, 120774, 122321, 124049, 124053, 124605, 125084, 126890] 2. Fujitsu [120774, 124049, 124053, 124605, 134019, 136177] |
| Impacted Organization | 1. Sub-postmasters running small post offices [20678, 93049, 113496, 116601, 117646, 120774, 124049, 124053, 124605, 125084, 126890] 2. Post Office operators [124605] 3. Individuals prosecuted by the Post Office based on Horizon software [126890] 4. Mizuho bank [134019] 5. Tokyo Stock Exchange (TSE) [134019] 6. Securities firm involved in the trading loss incident [134019] 7. Criminal Cases Review Commission in England and Wales [126890] 8. Scottish legal system [126890] |
| Software Causes | 1. Bugs and defects in the Horizon software [20678, 117646, 120774, 122321, 123006, 124049] 2. Errors in the Horizon software [122321, 123006, 124049] 3. Faulty software with accounting discrepancies, lost transactions, system freezes, and printer failures [120774] 4. Poor coding leading to software unreliability and lack of integrity [120774] 5. Software flaws causing financial ruin, false accounting, and theft convictions [122321, 123006] 6. Withheld evidence of software bugs and defects [124049] 7. Flaws in the Horizon software used in court to convict sub-postmasters [124053] 8. Failings of the Horizon software leading to one of the biggest miscarriages of justice in British legal history [124605] |
| Non-software Causes | 1. Training failures and inadequate support for sub-postmasters [20678] 2. Lack of oversight on the damage to morale of sub-postmasters [93049] 3. Withholding of key evidence and shredding of documents [124049] 4. Lack of accountability and justice for victims [125084] 5. Delay in holding accountable those responsible within government, the Post Office, and Fujitsu [126890] |
| Impacts | 1. Dozens of sub-postmasters were wrongly prosecuted, jailed, and financially ruined due to losses caused by the faulty Horizon computer system [20678, 113496, 116601, 117646, 122321, 123006]. 2. Some sub-postmasters lost their homes, failed to get insurance, and even died as a result of the convictions stemming from the software errors [113496, 117646]. 3. The faulty software led to significant financial impacts on individuals, with some losing their life savings and facing financial ruin [20678, 122321, 123006]. 4. The software failure resulted in a miscarriage of justice affecting more people than any other in the UK [116601, 124605]. 5. The victims faced emotional distress, depression, and feelings of worthlessness due to the software failure [125084]. 6. Over 2,000 individuals faced financial losses due to the faulty Horizon software, with only about half receiving compensation and many still waiting for resolution [126890]. |
| Preventions | 1. Proper testing and validation of the software before rollout to identify and address defects and bugs [120774]. 2. Adequate training and support for users to ensure they can effectively use the system [20678]. 3. Transparent communication and reporting of software issues and bugs to relevant stakeholders [124049]. 4. Regular monitoring and maintenance of the software to address errors and prevent system failures [120774]. 5. Implementing a system for accountability and oversight to ensure that software developers and vendors are held responsible for any faults in the system [126890]. 6. Avoiding rushing the rollout of software updates or new systems without addressing known issues and concerns [120774]. 7. Ensuring that there are mechanisms in place to address technical issues promptly and effectively [136177]. |
| Fixes | 1. Conducting a thorough investigation into the faults and flaws in the Horizon software system, particularly focusing on the bugs, defects, and errors that have been identified [20678, 120774, 124049]. 2. Implementing comprehensive measures to ensure that IT systems, like Horizon, are reliable and do not lead to financial losses or legal issues for sub-postmasters [93049, 120774]. 3. Holding accountable the software firm Fujitsu, which developed the faulty Horizon software, and ensuring that senior employees are called to a statutory inquiry to provide answers [124049, 124053]. 4. Providing just compensation to those affected by the software failure incident, including those who lost their homes, businesses, and faced financial hardships [113496, 126890]. 5. Addressing the training failures and inadequate support for sub-postmasters in using the Horizon system [20678]. 6. Ensuring transparency and accountability in the development and maintenance of software systems to prevent similar incidents in the future [120774, 134019]. 7. Considering the option of cancelling contracts with software developers if concerns about product reliability persist [136177]. | References | 1. Second Sight - private firm that conducted an independent review of the Horizon system [20678] 2. Post Office - organization where the software failure incident occurred [20678, 93049, 113496, 120774, 124049, 124053, 124605, 125084, 126890] 3. Fujitsu - the global IT company that developed the faulty software [120774, 124049, 124053, 134019, 136177] |
| Category | Option | Rationale |
|---|---|---|
| Recurring | one_organization, multiple_organization | (a) The software failure incident has happened again at one_organization: - Fujitsu, the software firm that developed the Horizon software, was involved in a scandal in 2002 for glitches at ATMs of a major bank and in 2005 for a trading loss at the Tokyo Stock Exchange due to software issues [Article 134019]. (b) The software failure incident has happened again at multiple_organization: - The Post Office Horizon software failure incident led to wrongful prosecutions and convictions of more than 700 post office operators between 2000 and 2014 [Article 124605]. - Fujitsu, the company behind the Horizon software, faced criticism for software issues at ATMs of a major bank in 2002 and a trading loss at the Tokyo Stock Exchange in 2005 [Article 134019]. |
| Phase (Design/Operation) | design, operation | (a) The software failure incident related to the development phases: - The Horizon accounting system developed by Fujitsu was found to contain "bugs, errors, and defects" that led to accounting discrepancies and losses in branch accounts [Article 113496]. - An internal analysis before the system went live identified six "high severity" hitches causing accounting discrepancies, lost transactions, system freezes, and other issues [Article 120774]. - The Post Office prosecuted post office operators based on information from the Horizon system, which was found to have bugs, errors, and defects that caused problems leading to false accounting and theft convictions [Article 122321]. - The software developed by Fujitsu for the Post Office was described as unreliable, with poor coding leading to crashes, incorrect information transmission, and accounting discrepancies [Article 120774]. (b) The software failure incident related to the operation phases: - Post office operators were prosecuted and convicted based on information from the faulty Horizon system, leading to financial ruin, jail time, and other severe consequences [Article 122321]. - The faulty software caused postmasters to be wrongly blamed for losses identified by the computer system, leading to prosecutions, jail time, and financial ruin for many [Article 20678]. - The Horizon system had bugs and defects that left a black hole in accounts, contributing to the convictions and financial ruin of postmasters [Article 117646]. - The Horizon software was found to have flaws that led to convictions for false accounting and theft, causing financial ruin for many post office operators [Article 123006]. |
| Boundary (Internal/External) | within_system, outside_system | (a) within_system: - The software failure incident was primarily within the system as it was caused by defects, bugs, errors, and faults in the Horizon software developed by Fujitsu [20678, 113496, 120774, 124053, 124605, 125084]. - The articles highlight how the Horizon system had internal issues such as bugs, errors, defects, and flaws that led to accounting discrepancies, lost transactions, system freezes, and other malfunctions [120774]. - Fujitsu, the company that developed the software, made numerous changes to the Horizon software code over the years, indicating ongoing internal issues within the system [120774]. - The Post Office and Fujitsu were involved in a long-running dispute with sub-postmasters and postmistresses affected by the software issues, leading to a settlement agreement [113496]. - The software failure incident led to prosecutions of post office operators based on information from the faulty Horizon IT system, indicating internal issues within the system that affected individuals [124605]. (b) outside_system: - The software failure incident also involved external factors such as inadequate training and support provided by the Post Office to sub-postmasters, contributing to the overall failure of the system [20678]. - The articles mention the human impact of the software failings, indicating that external factors such as the treatment of individuals affected by the software issues played a role in the incident [124605, 125084]. - The inquiry into the Horizon IT system is expected to investigate whether staff at Fujitsu knew about the system's flaws when data was used in court to convict sub-postmasters, suggesting external factors influencing the software failure incident [124053]. |
| Nature (Human/Non-human) | non-human_actions, human_actions | (a) The software failure incident occurring due to non-human actions: - The software system, Horizon, was found to have 'defects' or 'bugs' that affected 76 branches, leading to sub-postmasters being wrongly blamed for losses identified by the faulty computer system [20678]. - An internal analysis before the project went live listed six 'high severity' hitches causing accounting discrepancies, lost transactions, system freezes, and other issues, indicating problems with the software itself [120774]. - One specific bug in the software, known as the Reversal Bug, involved an incorrect sign in the software code that caused transactions to be doubled instead of canceled, highlighting a non-human error in the software code [120774]. (b) The software failure incident occurring due to human actions: - The Post Office board refused to sign off the project due to serious doubts about the reliability of the software, but later gave the go-ahead to roll out the system despite knowing about the issues, indicating human decision-making contributing to the failure [120774]. - Fujitsu, the company that developed the Horizon software, had internal cultural beliefs that the software was faultless and not capable of being the source of accounting errors, leading to a lack of accountability and responsibility for the software issues [120774]. - Fujitsu engineers were working full-time on catching, spotting, and correcting errors in the software, indicating ongoing human efforts to address the software issues [120774]. - The software failure incident led to more than 700 post office operators being prosecuted based on information from the faulty Horizon IT system, highlighting the human impact of the software failure [124605]. |
| Dimension (Hardware/Software) | software | (a) The software failure incident occurring due to hardware: - The articles do not specifically mention the software failure incident occurring due to contributing factors originating in hardware. Therefore, this information is unknown. (b) The software failure incident occurring due to software: - The software failure incident was primarily attributed to defects, bugs, and errors in the Horizon software developed by Fujitsu, which affected Post Office operations [Article 20678]. - An internal analysis highlighted high severity hitches causing accounting discrepancies, lost transactions, system freezes, and other issues due to gaps in data and software reliability concerns [Article 120774]. - The Horizon software had flaws that impacted sub-postmasters and postmistresses, leading to convictions based on flawed data from the software [Article 124053]. - The Horizon software was found to be seriously flawed, leading to tragic consequences, with deep sympathy expressed for those affected [Article 136177]. |
| Objective (Malicious/Non-malicious) | malicious, non-malicious | (a) In the software failure incident related to the Post Office Horizon IT system, there are indications of both malicious and non-malicious factors contributing to the failure: Malicious: - The articles mention that there were concerns about potential bugs, errors, and defects in the software, and key evidence was withheld while documents were shredded [Article 124049]. - It is highlighted that alterations in the system could be made without those at the front end being aware of it, and there were financial penalties for breakdowns, creating a situation where it was easier to blame Subpostmasters rather than address the software issues [Article 120774]. - The inquiry into the Horizon IT system is expected to look at whether staff at Fujitsu knew about flaws in the system when data was used in court to convict Subpostmasters, suggesting a potential malicious intent [Article 124053]. Non-malicious: - The articles also discuss how the software had serious doubts about its reliability even before it went live, with high severity issues causing discrepancies, lost transactions, and system freezes [Article 120774]. - There are mentions of coding errors, poor coding practices, and the need for constant upgrades and patches to fix errors, indicating non-malicious factors contributing to the software failure [Article 120774]. - The inquiry into the Horizon IT system is also investigating the human impact of the software failings, suggesting that there were unintended consequences of the software issues [Article 124605]. - Victims of the software failure express a desire for justice and accountability, indicating that they believe the failures were not intentional but had severe consequences [Article 125084, Article 126890]. Overall, the software failure incident appears to involve a combination of malicious factors such as potential cover-ups and non-malicious factors such as technical issues, poor coding practices, and unintended consequences. |
| Intent (Poor/Accidental Decisions) | poor_decisions, accidental_decisions | (a) The software failure incident related to poor decisions: - The Post Office board refused to sign off the project due to serious doubts about the reliability of the software but later gave the go-ahead to roll out the system despite the identified issues [Article 120774]. - The Post Office had known about issues in the accounting software but failed to inform the Crown Prosecution Service, leading to employees being accused of crimes they never committed [Article 121245]. - The software firm Fujitsu, which developed the Horizon software, may have known about flaws in the system when data from the software was used in court to convict sub-postmasters [Article 124053]. (b) The software failure incident related to accidental decisions: - The software had bugs and defects that left a black hole in accounts, indicating unintended consequences of the software issues [Article 117646]. - The scandal involving wrongful accusations of theft, fraud, and false accounting was described as one of Britain's biggest miscarriages of justice, suggesting unintended outcomes of the software failure [Article 121245]. - The Horizon product was described as seriously flawed, leading to tragic and completely unacceptable consequences, indicating unintended negative impacts of the software failure incident [Article 136177]. |
| Capability (Incompetence/Accidental) | development_incompetence, accidental | (a) The software failure incident occurring due to development incompetence: - The Horizon software developed by Fujitsu had serious flaws and defects that led to tragic consequences for post office operators [Article 136177]. - The software had multiple faults and defects, including high severity hitches causing accounting discrepancies, lost transactions, system freezes, printer failures, and general losses of accounting integrity [Article 120774]. - The software developers at Fujitsu were criticized for poor coding practices, with only a couple of developers knowing how to code properly [Article 120774]. (b) The software failure incident occurring accidentally: - The software had bugs and defects that left a black hole in accounts, indicating accidental introduction of issues [Article 117646]. - The system had gaps in data and other issues that were not fully understood by the developers, leading to unintended consequences [Article 120774]. - The software failures resulted in wrongful accusations of theft, fraud, and false accounting against post office operators due to computer errors [Article 124605]. |
| Duration | permanent, temporary | (a) The software failure incident in the articles appears to be more of a permanent nature. The failure was attributed to various contributing factors introduced by all circumstances, such as bugs, errors, defects, poor coding, unreliable software, withheld evidence, and serious doubts about the reliability of the software [120774, 124049, 124053, 125084, 126890]. (b) However, there were also certain circumstances that contributed to the temporary nature of the failure, such as the ongoing investigation, delays in progress, and the need for a statutory inquiry to uncover the facts and provide answers to those affected by the faulty software [124049, 126890, 136177]. |
| Behaviour | crash, omission, value, byzantine, other | (a) crash: The software failure incident involved crashes where the system lost state and did not perform its intended functions. For example, the system had "bugs and defects that left a black hole in accounts" leading to prosecutions and convictions [117646]. Additionally, the software had "faults in the system" causing accounting discrepancies, lost transactions, system freezes, and other issues [120774]. (b) omission: The software failure incident also involved omissions where the system omitted to perform its intended functions at instances. For instance, there were cases where transactions were not canceled as intended but instead doubled due to a software bug [120774]. (c) timing: The timing of the software failure incident was related to the system performing its intended functions incorrectly, either too late or too early. This was evident in cases where the system processed transactions incorrectly, leading to financial discrepancies and legal issues [124049]. (d) value: The software failure incident included failures where the system performed its intended functions incorrectly, affecting the value of transactions and accounts. For example, the system had coding errors that caused transactions to be processed incorrectly, leading to financial discrepancies and legal consequences [120774]. (e) byzantine: The software failure incident exhibited byzantine behavior where the system behaved erroneously with inconsistent responses and interactions. This was seen in the case of the Reversal Bug, where a coding error caused transactions to be doubled instead of canceled as intended [120774]. (f) other: In addition to the above behaviors, the software failure incident also involved issues with the system's reliability, poor coding leading to system crashes, inadequate support for users, and a lack of understanding of root causes of problems by the IT company [20678, 120774]. |
| Layer | Option | Rationale |
|---|---|---|
| Perception | None | None |
| Communication | None | None |
| Application | None | None |
| Category | Option | Rationale |
|---|---|---|
| Consequence | death, property | (a) death: People lost their lives due to the software failure - Some postmasters who were convicted due to the faulty Horizon software have since died [Article 113496]. - Some of the convicted workers were sent to prison, others lost their livelihoods and their homes. Many went bankrupt, and some died before their names were cleared [Article 124605]. (b) harm: People were physically harmed due to the software failure - No information in the articles suggests that people were physically harmed due to the software failure. (c) basic: People's access to food or shelter was impacted because of the software failure - No information in the articles suggests that people's access to food or shelter was impacted due to the software failure. (d) property: People's material goods, money, or data was impacted due to the software failure - Postmasters were financially ruined, lost their homes, and failed to get insurance due to their convictions based on the faulty software [Article 113496]. - The software caused financial ruin for many postmasters, leading to loss of homes and financial struggles [Article 117646]. - The faulty software caused financial losses for postmasters, leading to ruined lives and imprisonment [Article 120774]. - The software caused financial problems for post office operators, leading to convictions for false accounting and theft [Article 122321]. - Many individuals lost out financially due to the faulty Horizon software, with some receiving compensation offers [Article 126890]. (e) delay: People had to postpone an activity due to the software failure - No information in the articles suggests that people had to postpone activities due to the software failure. (f) non-human: Non-human entities were impacted due to the software failure - No information in the articles suggests that non-human entities were impacted due to the software failure. (g) no_consequence: There were no real observed consequences of the software failure - The articles clearly outline the significant consequences of the software failure on individuals involved. (h) theoretical_consequence: There were potential consequences discussed of the software failure that did not occur - No theoretical consequences were discussed in the articles. (i) other: Was there consequence(s) of the software failure not described in the (a to h) options? What is the other consequence(s)? - No other consequences beyond those described in options (a) to (h) were mentioned in the articles. |
| Domain | finance, government | (a) The failed system was intended to support the finance industry, specifically the Post Office operations. The Horizon IT system introduced in 2000 was meant to handle financial transactions and accounts for the Post Office branches [20678, 93049, 117646, 120774, 124049, 124605, 125084]. (l) The failed system was also related to the government industry as it was implemented by the Post Office, which is a government-owned entity. The software failure incident affected the operations of the Post Office and led to legal issues for sub-postmasters [20678, 93049, 117646, 120774, 124049, 124605, 125084]. (m) Additionally, the failed system was related to the utilities industry indirectly as it impacted the services provided by the Post Office, which is a utility service for the public [120774]. |
Article ID: 126890
Article ID: 136177
Article ID: 121245
Article ID: 114619
Article ID: 131957
Article ID: 113496
Article ID: 121897
Article ID: 93049
Article ID: 128018
Article ID: 117646
Article ID: 122321
Article ID: 113495
Article ID: 124605
Article ID: 123006
Article ID: 134019
Article ID: 124049
Article ID: 124053
Article ID: 120774
Article ID: 105204
Article ID: 20678
Article ID: 129317
Article ID: 125084
Article ID: 113852
Article ID: 116601