Incident: Crossrail Software Failure Causes Major Project Delays and Issues

Published Date: 2018-08-31

Postmortem Analysis
Timeline 1. The software failure incident happened in November 2017. - Step 1: The article mentions an electrical explosion in east London last November. - Step 2: The article was published on 2018-08-31. - Step 3: By deducting around 9 months from the publication date, the incident occurred in November 2017. [74861]
System 1. Railway systems software [74861] 2. Signalling systems reconciliation [74861]
Responsible Organization 1. Crossrail admitted that more time was needed to develop the railway systems software, contributing to the software failure incident [74861].
Impacted Organization 1. Crossrail [74861] 2. Transport for London (TfL) [74861]
Software Causes 1. Crossrail admitted running out of time to adequately test the Elizabeth line trains due to the need for more time to develop the railway systems software [74861]. 2. Major issues were reported in reconciling the three different signalling systems, leading to delays in testing [74861].
Non-software Causes 1. Missed deadlines in fitting out tracks and infrastructure in central tunnels [74861]. 2. Major issues reconciling three different signalling systems [74861]. 3. Departure of the scheme's leaders, the chairman, Terry Morgan, and the chief executive, Andrew Wolstenholme, before delivery [74861]. 4. Issues and delays in station upgrades and other work on outer sections of the line [74861].
Impacts 1. The software failure incident led to delays in the Crossrail project as the railway systems software needed more time to be developed, impacting the overall completion timeline [74861]. 2. The delay in the project resulted in financial implications for Transport for London (TfL) as Crossrail revenues were a significant part of a business plan to address TfL's deficit, potentially leading to lost revenue and potential fare increases [74861]. 3. The delay in the Crossrail project could have serious implications for retailers in the West End who were expecting a rise in footfall due to the completion of the project, causing concerns about the economic impact of the delay [74861].
Preventions 1. Adequate time for testing: One preventive measure could have been allocating sufficient time for testing the Elizabeth line trains across central London. The incident mentioned that Crossrail ran out of time to adequately test the trains, which could have contributed to the software failure [74861]. 2. Proper development and testing of railway systems software: Crossrail mentioned that more time was needed to develop the railway systems software. Ensuring thorough development and testing of the software could have potentially prevented the software failure incident [74861]. 3. Addressing issues with reconciling different signalling systems: Insiders mentioned major issues reconciling the three different signalling systems. Resolving these issues before implementation could have prevented the software failure incident [74861].
Fixes 1. Adequate testing and development of the railway systems software to ensure proper functionality [74861]. 2. Addressing major issues with reconciling the three different signalling systems to improve system integration and compatibility [74861]. 3. Implementing better oversight and project management to prevent further delays and issues with the software and overall project delivery [74861].
References 1. Crossrail officials 2. Unnamed insiders 3. Unions 4. Critics such as Andrew Adonis

Software Taxonomy of Faults

Category Option Rationale
Recurring unknown The articles do not mention any specific software failure incident happening again at either the same organization (one_organization) or at multiple organizations (multiple_organization). Therefore, the information related to these options is unknown.
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase is evident in the article as Crossrail admitted that more time was needed to develop the railway systems software. Insiders mentioned major issues reconciling the three different signalling systems, which contributed to delays in testing the Elizabeth line trains across central London [74861]. (b) The software failure incident related to the operation phase is implied in the article when workers were told to put things in and take them out, fitting out parts of stations, only to be later told they had been wrongly designed. This indicates issues arising from the operation or execution of the system, leading to delays and inefficiencies in the project [74861].
Boundary (Internal/External) within_system (a) The software failure incident related to the Crossrail project was primarily within the system. Crossrail admitted that they ran out of time to adequately test the Elizabeth line trains across central London and needed more time to develop the railway systems software [74861]. Additionally, there were major issues reconciling the three different signalling systems, which was an internal challenge faced by the project [74861]. The delays and issues with fitting out tracks, infrastructure, and stations were all internal factors contributing to the software failure incident within the system [74861].
Nature (Human/Non-human) non-human_actions, human_actions (a) The software failure incident occurring due to non-human actions: Crossrail admitted that they ran out of time to adequately test the Elizabeth line trains across central London, and more time was needed to develop the railway systems software [74861]. (b) The software failure incident occurring due to human actions: Insiders mentioned major issues reconciling the three different signalling systems, and there were delays in fitting out tracks and other infrastructure in the central tunnels. Workers also referred to the scheme as the "hokey cokey" line, indicating confusion and errors in fitting out parts of stations [74861].
Dimension (Hardware/Software) hardware, software (a) The software failure incident occurring due to hardware: - The article mentions that contractors have missed deadlines to fit out the tracks and other infrastructure in the central tunnels, and more time is needed to develop the railway systems software [74861]. - An electrical explosion in east London last November took months to resolve and postponed testing, indicating hardware issues [74861]. (b) The software failure incident occurring due to software: - Crossrail admitted it ran out of time to adequately test the Elizabeth line trains across central London, indicating software-related issues [74861]. - Insiders spoke of major issues reconciling the three different signalling systems, pointing towards software challenges [74861].
Objective (Malicious/Non-malicious) non-malicious (a) The software failure incident related to the Crossrail project does not appear to be malicious. The delays and issues with the railway systems software were attributed to missed deadlines by contractors, difficulties reconciling different signalling systems, and the need for more time to develop the software [74861]. There is no indication in the articles that the software failure was due to any malicious intent to harm the system.
Intent (Poor/Accidental Decisions) poor_decisions (a) The software failure incident related to the Crossrail project seems to be more aligned with poor_decisions. The delays and issues with the Elizabeth line trains and railway systems software were a result of missed deadlines, inadequate testing, and major issues reconciling different signalling systems [74861]. Additionally, there were problems with fitting out parts of stations and design errors, indicating a lack of proper planning and decision-making in the project [74861]. The departure of key leaders before the project's delivery also raised questions about oversight and decision-making processes [74861].
Capability (Incompetence/Accidental) development_incompetence, accidental (a) The software failure incident related to development incompetence is evident in the delays faced by Crossrail due to inadequate testing of the Elizabeth line trains and the need for more time to develop the railway systems software [74861]. (b) The software failure incident related to accidental factors is highlighted in the major issues faced in reconciling the three different signalling systems, as well as incidents like the electrical explosion in east London that took months to resolve and postponed testing [74861].
Duration temporary The software failure incident related to the Crossrail project appears to be temporary rather than permanent. The delay in the project was primarily attributed to the inadequate time for testing the Elizabeth line trains and the need for more time to develop the railway systems software [74861]. Additionally, there were issues with reconciling different signalling systems and delays in fitting out tracks and infrastructure, which contributed to the temporary nature of the software failure incident.
Behaviour crash, omission, timing, value, other (a) crash: The software failure incident related to the Crossrail project can be attributed to a crash. The article mentions that Crossrail ran out of time to adequately test the Elizabeth line trains across central London, and there were major issues reconciling the three different signalling systems, leading to delays and postponement of testing [74861]. (b) omission: The software failure incident can also be linked to omission. Workers were told to put things in and take them out, fitting out parts of stations, and then being told they had been wrongly designed. This indicates a failure of the system to perform its intended functions correctly at certain instances [74861]. (c) timing: The timing of the software failure incident is also a factor. The delays in fitting out tracks and infrastructure, as well as the need for more time to develop the railway systems software, point to a failure of the system to perform its intended functions correctly but too late [74861]. (d) value: The software failure incident can be associated with a failure in value as well. The delays and issues with Crossrail have led to a year's delay, potentially resulting in hundreds of millions of pounds in lost revenue and other fares rising, impacting the value proposition of the project [74861]. (e) byzantine: The software failure incident does not directly align with a byzantine behavior as described in the articles. (f) other: The software failure incident could also be categorized under the "other" behavior as it involved issues with fitting out parts of stations incorrectly designed, which may not fit directly into the crash, omission, timing, or value categories [74861].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence delay, theoretical_consequence (a) unknown (b) unknown (c) unknown (d) unknown (e) The software failure incident led to a delay in the completion of the Crossrail project. The specific deadline of December 2018 was not met, and the latest schedule target is autumn 2019 [74861]. (f) unknown (g) unknown (h) The software failure incident led to potential financial consequences, such as hundreds of millions of pounds in lost revenue for TfL due to the delay in Crossrail project completion [74861]. (i) unknown
Domain transportation (a) The software failure incident reported in the articles is related to the transportation industry. The failure occurred in the context of the Crossrail project, specifically impacting the testing and development of railway systems software for the Elizabeth line trains across central London [Article 74861]. This incident has led to delays in the completion of the project, affecting commuters and stakeholders in the transportation sector.

Sources

Back to List