Published Date: 2022-03-08
| Postmortem Analysis | |
|---|---|
| Timeline | 1. The software failure incident involving Pony.ai's autonomous driving system happened on October 28, as mentioned in both [Article 126091] and [Article 126222]. Therefore, the incident occurred in October 2021. |
| System | 1. Autonomous driving system software [126091, 126222] 2. Planning system diagnostic check [126091, 126222] |
| Responsible Organization | 1. Pony.ai's autonomous driving system software [126091, 126222] |
| Impacted Organization | 1. National Highway Traffic Safety Administration (NHTSA) [126091, 126222] 2. California Department of Motor Vehicles [126091, 126222] |
| Software Causes | 1. The software failure incident was caused by a planning system diagnostic check that could generate a 'false positive' indication of a geolocation mismatch, leading to the crash [126091, 126222]. 2. The automated driving system software had a safety defect that prompted the National Highway Traffic Safety Administration (NHTSA) to request a recall, indicating a flaw in the software [126222]. |
| Non-software Causes | 1. The Pony.ai vehicle hit a street sign on a median in Fremont, California, after turning right, which led to the crash incident [126091, 126222]. 2. The incident prompted California in December to suspend Pony.ai's driverless testing permit [126091, 126222]. 3. The crash occurred less than 2.5 seconds after the automated driving system shut down [126091, 126222]. 4. In very rare circumstances, a planning system diagnostic check "could generate a 'false positive' indication of a geolocation mismatch" [126091, 126222]. |
| Impacts | 1. The software failure incident involving Pony.ai's autonomous driving system led to a crash in California, where a vehicle hit a street sign on a median after turning right, resulting in the suspension of the company's driverless testing permit by California authorities [126091, 126222]. 2. The National Highway Traffic Safety Administration (NHTSA) initiated a review to determine whether Pony.ai complied with government reporting requirements for driverless crashes, indicating potential regulatory scrutiny and oversight as a consequence of the incident [126091, 126222]. 3. The incident prompted Pony.ai to issue a recall for some versions of its autonomous driving system software, marking the "first recall of an automated driving system" as stated by the NHTSA, highlighting the impact on the company's product safety and reputation [126091, 126222]. 4. Pony.ai reported that the crash occurred due to a software issue where a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch, emphasizing the technical implications of the software failure incident [126091, 126222]. 5. The company updated the software code and repaired the three affected vehicles, indicating corrective actions taken to address the software failure and ensure the safety of their autonomous driving system [126091, 126222]. |
| Preventions | 1. Implementing more robust testing procedures to catch potential software defects before deployment could have prevented the software failure incident [126091, 126222]. 2. Conducting thorough reviews and audits of the software code to identify and address any potential safety defects or anomalies could have helped prevent the incident [126091, 126222]. 3. Enhancing the monitoring and diagnostic capabilities of the autonomous driving system software to quickly detect and respond to any geolocation mismatches or false positives could have mitigated the risk of the incident [126091, 126222]. |
| Fixes | 1. Updating the software code to address the safety defect identified by the NHTSA [126091, 126222] 2. Conducting a thorough review of the autonomous driving system software to prevent similar incidents in the future [126222] |
| References | 1. National Highway Traffic Safety Administration (NHTSA) [Article 126091, Article 126222] 2. Pony.ai [Article 126091, Article 126222] 3. California authorities [Article 126091, Article 126222] |
| Category | Option | Rationale |
|---|---|---|
| Recurring | one_organization | (a) The software failure incident has happened again at one_organization: - Pony.ai experienced a software failure incident in October involving its autonomous driving system software, leading to a crash in California [126091, 126222]. - The incident prompted California to suspend Pony.ai's driverless testing permit [126091, 126222]. - Pony.ai agreed to issue a recall for some versions of its software after the crash [126091, 126222]. - The National Highway Traffic Safety Administration (NHTSA) reviewed Pony.ai's compliance with reporting requirements related to the incident [126091, 126222]. (b) The software failure incident has happened again at multiple_organization: - There is no specific mention in the articles about the software failure incident happening at other organizations or with their products and services. |
| Phase (Design/Operation) | design, operation | (a) The software failure incident related to the design phase can be seen in the articles. Pony.ai's autonomous driving system software had a flaw where in very rare circumstances, a planning system diagnostic check "could generate a 'false positive' indication of a geolocation mismatch" [126091, 126222]. This flaw in the software design contributed to the incident where a Pony.ai vehicle hit a street sign in California while operating in autonomous mode. (b) The software failure incident related to the operation phase is evident in the articles as well. The crash in California occurred less than 2.5 seconds after the automated driving system shut down, indicating an operational issue with the system [126091, 126222]. Additionally, the National Highway Traffic Safety Administration (NHTSA) emphasized the need for vehicle manufacturers and developers to prioritize safety during the operation of automated driving systems. |
| Boundary (Internal/External) | within_system | (a) within_system: The software failure incident involving Pony.ai's autonomous driving system was primarily due to factors originating from within the system. Pony.ai reported that the crash occurred less than 2.5 seconds after the automated driving system shut down, indicating an issue within the system itself. The company mentioned that in very rare circumstances, a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch, which was an internal system issue [126091, 126222]. |
| Nature (Human/Non-human) | non-human_actions, human_actions | (a) The software failure incident in the articles seems to be related to non-human actions. The incident occurred when a Pony.ai vehicle operating in autonomous mode hit a street sign on a median in Fremont, California, after turning right. The company mentioned that the crash occurred less than 2.5 seconds after the automated driving system shut down, and in very rare circumstances, a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch. The National Highway Traffic Safety Administration (NHTSA) requested a recall of the software, indicating a safety defect that needed to be addressed [126091, 126222]. (b) Human actions also played a role in the incident as Pony.ai, the startup technology firm, had to comply with government reporting requirements for driverless crashes. The NHTSA will review whether Pony.ai complied with reporting requirements regarding the timeliness and accuracy of its reports. Additionally, Pony.ai mentioned that it reported the incident to the NHTSA in a good faith effort to comply with the relevant requirements and has been fully cooperating with the NHTSA throughout the process. The company also updated the software code and repaired the three affected vehicles [126091, 126222]. |
| Dimension (Hardware/Software) | hardware, software | (a) The software failure incident related to hardware: - The incident involving Pony.ai's autonomous driving system hitting a street sign in California was attributed to a software issue where a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch, leading to the crash [126091, 126222]. (b) The software failure incident related to software: - The software failure incident involving Pony.ai's autonomous driving system was primarily due to a software issue where the automated driving system shut down less than 2.5 seconds before the crash occurred [126091, 126222]. |
| Objective (Malicious/Non-malicious) | non-malicious | (a) The software failure incident related to Pony.ai's autonomous driving system was non-malicious. The incident occurred when a Pony.ai vehicle hit a street sign on a median in Fremont, California, after turning right in autonomous mode. The company stated that the crash occurred less than 2.5 seconds after the automated driving system shut down, and in very rare circumstances, a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch. The National Highway Traffic Safety Administration (NHTSA) requested a recall of the software, believing it had a safety defect, and Pony.ai cooperated by updating the software code and repairing the affected vehicles. The company also suspended its testing program to conduct a thorough review [126091, 126222]. |
| Intent (Poor/Accidental Decisions) | poor_decisions | (a) The software failure incident related to Pony.ai's autonomous driving system can be attributed to poor decisions. The incident occurred when a Pony.ai vehicle operating in autonomous mode hit a street sign on a median in Fremont, California, after turning right. This prompted California to suspend the company's driverless testing permit. The National Highway Traffic Safety Administration (NHTSA) reviewed whether Pony.ai complied with government reporting requirements for driverless crashes, indicating that there were concerns about the company's decisions and actions regarding the incident [126091, 126222]. |
| Capability (Incompetence/Accidental) | development_incompetence, accidental | (a) The software failure incident related to development incompetence is evident in the articles. Pony.ai's autonomous driving system software had a safety defect that led to a crash in California. The National Highway Traffic Safety Administration (NHTSA) believed that the software had a safety defect and requested Pony.ai to conduct a recall [126091, 126222]. This indicates that the failure was due to contributing factors introduced as a result of development incompetence. (b) The software failure incident related to accidental factors is also present in the articles. Pony.ai mentioned that in very rare circumstances, a planning system diagnostic check "could generate a 'false positive' indication of a geolocation mismatch," which could have contributed to the crash [126091, 126222]. This suggests that the incident was also influenced by accidental factors. |
| Duration | temporary | The software failure incident involving Pony.ai's autonomous driving system can be categorized as a temporary failure. The incident occurred when a Pony.ai vehicle hit a street sign in California after turning right in autonomous mode, prompting the suspension of the company's driverless testing permit. The crash occurred less than 2.5 seconds after the automated driving system shut down, indicating a specific circumstance led to the failure [126091, 126222]. Additionally, Pony.ai mentioned that in very rare circumstances, a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch, further suggesting a specific condition or trigger for the failure [126091]. |
| Behaviour | crash, timing, other | (a) crash: The software failure incident in the articles is related to a crash. The Pony.ai vehicle operating in autonomous mode hit a street sign on a median in Fremont, California, after turning right, resulting in the crash [126091, 126222]. (b) omission: There is no specific mention of the software failure incident being related to omission in the articles. (c) timing: The incident involved a timing issue as the crash occurred less than 2.5 seconds after the automated driving system shut down [126091, 126222]. (d) value: The software failure incident did not involve the system performing its intended functions incorrectly. (e) byzantine: There is no indication in the articles that the software failure incident exhibited byzantine behavior. (f) other: The software failure incident could be categorized as a crash due to the system losing its state and failing to perform its intended functions as expected [126091, 126222]. |
| Layer | Option | Rationale |
|---|---|---|
| Perception | processing_unit, embedded_software | (a) sensor: The software failure incident related to Pony.ai's autonomous driving system did not specifically mention a sensor error as the contributing factor to the crash. The incident was described as occurring when the automated driving system shut down, leading to a geolocation mismatch indication in very rare circumstances [Article 126091]. (b) actuator: The articles did not mention an actuator error as a contributing factor to the software failure incident involving Pony.ai's autonomous driving system [Article 126091, Article 126222]. (c) processing_unit: The failure in the software incident was related to a processing error in the planning system diagnostic check that could generate a 'false positive' indication of a geolocation mismatch in very rare circumstances [Article 126091, Article 126222]. (d) network_communication: The articles did not indicate network communication error as a contributing factor to the software failure incident involving Pony.ai's autonomous driving system [Article 126091, Article 126222]. (e) embedded_software: The software failure incident was related to an error in the embedded software of Pony.ai's autonomous driving system, where a planning system diagnostic check could generate a 'false positive' indication of a geolocation mismatch in very rare circumstances [Article 126091, Article 126222]. |
| Communication | unknown | Unknown |
| Application | FALSE | The software failure incident involving Pony.ai's autonomous driving system was not explicitly attributed to the application layer of the cyber physical system. The incident was primarily described as a crash involving the autonomous driving system software, with mentions of a safety defect in the software and a geolocation mismatch issue. However, specific details regarding bugs, operating system errors, unhandled exceptions, or incorrect usage contributing to the failure were not provided in the articles [126091, 126222]. Therefore, it is unknown whether the failure was related to the application layer of the cyber physical system based on the information available in the articles. |
| Category | Option | Rationale |
|---|---|---|
| Consequence | other | (a) death: People lost their lives due to the software failure - There is no mention of any deaths resulting from the software failure incident reported in the articles [126091, 126222]. (b) harm: People were physically harmed due to the software failure - The articles do not mention any physical harm caused to individuals due to the software failure incident [126091, 126222]. (c) basic: People's access to food or shelter was impacted because of the software failure - The articles do not indicate any impact on people's access to food or shelter as a consequence of the software failure incident [126091, 126222]. (d) property: People's material goods, money, or data was impacted due to the software failure - The software failure incident did not result in any direct impact on people's material goods, money, or data as per the articles [126091, 126222]. (e) delay: People had to postpone an activity due to the software failure - There is no mention of any activities being postponed by people due to the software failure incident in the articles [126091, 126222]. (f) non-human: Non-human entities were impacted due to the software failure - The software failure incident primarily affected the autonomous driving system software of Pony.ai vehicles, and there is no mention of non-human entities being impacted in the articles [126091, 126222]. (g) no_consequence: There were no real observed consequences of the software failure - The software failure incident did have consequences, such as the recall of affected vehicles and the suspension of driverless testing permits, as reported in the articles [126091, 126222]. (h) theoretical_consequence: There were potential consequences discussed of the software failure that did not occur - The articles do not discuss any potential consequences of the software failure incident that did not actually occur [126091, 126222]. (i) other: Was there consequence(s) of the software failure not described in the (a to h) options? What is the other consequence(s)? - The primary consequence of the software failure incident was the need for Pony.ai to issue a recall for some versions of its autonomous driving system software and the suspension of its driverless testing permit by California authorities. The incident also led to a review by the NHTSA regarding compliance with reporting requirements and software safety defects [126091, 126222]. |
| Domain | transportation | (a) The failed system was related to the transportation industry as it involved an autonomous driving system developed by Pony.ai [126091, 126222]. The system was designed for driverless testing and autonomous driving of vehicles, indicating its application in the transportation sector. |
Article ID: 126091
Article ID: 126222