Incident: Self-Driving Car Prototype Runs Red Light Due to Sensor Interference

Published Date: 2018-05-23

Postmortem Analysis
Timeline 1. The software failure incident of the self-driving car prototype running a red light in Jerusalem happened last week as reported in Article 71269. 2. Published on 2018-05-23. 3. The incident likely occurred in May 2018.
System The system that failed in the software failure incident involving the self-driving car prototype in Jerusalem was: 1. Mobileye's self-driving car system, specifically affected by electromagnetic interference from a GoPro camera attached to the car [71269].
Responsible Organization 1. The GoPro camera attached to the car interfered with the car's sensors, causing the software failure incident [Article 71269].
Impacted Organization 1. The self-driving car prototype built by Intel-owned Mobileye [Article 71269] 2. The TV news crew on board the self-driving car [Article 71269] 3. Three women waiting at a crosswalk as the car passed by [Article 71269]
Software Causes 1. The software failure incident in the self-driving car prototype in Jerusalem was caused by the interference of a GoPro camera attached to the car with the car's sensors, leading it to run a red light [71269].
Non-software Causes 1. The failure incident was caused by a GoPro camera attached to the self-driving car interfering with the car's sensors, leading it to run a red light [71269].
Impacts 1. The self-driving car prototype built by Mobileye failed to stop at a red light in Jerusalem, putting pedestrians at risk and causing a safety concern [Article 71269].
Preventions 1. Implementing rigorous testing procedures to account for unexpected scenarios like interference from external devices, such as the GoPro camera in this case [71269]. 2. Enhancing the software's ability to detect and react to sudden changes in traffic signals, like the red light that was missed in the incident [71269]. 3. Improving the communication and coordination between the software system and the safety driver to ensure prompt corrective actions in case of failures [71269].
Fixes 1. Implementing better shielding for the equipment to prevent interference from external devices like the GoPro camera [71269].
References 1. News 10 [Article 71269] 2. Bloomberg News [Article 71269]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) In the provided article [71269], it is mentioned that Mobileye, the Intel-owned company responsible for the self-driving car prototype that ran a red light in Jerusalem, faced a unique situation where a GoPro attached to the car interfered with the car's sensors. This incident highlights an internal software failure within the organization that led to the car running the red light. (b) The article [71269] also mentions Mobileye's past collaboration with Tesla and how it ended in 2016 due to disagreements over the safe use of technology in Tesla's Autopilot feature. This indicates that software failure incidents or disagreements related to technology have occurred between Mobileye and other organizations like Tesla in the past.
Phase (Design/Operation) design (a) The software failure incident in the article can be attributed to a design-related issue. The incident occurred because a GoPro camera attached to the self-driving car interfered with the car's sensors, leading to the car running a red light. Mobileye's CEO mentioned that they had never anticipated such a situation and that they are taking steps to better shield the equipment from electromagnetic radiation, indicating a design flaw introduced during the development phase [71269]. (b) There is no specific information in the article pointing to the software failure incident being caused by operation-related factors such as misuse of the system.
Boundary (Internal/External) within_system (a) within_system: The software failure incident involving the self-driving car prototype running a red light in Jerusalem was attributed to a unique situation where a GoPro attached to the car interfered with the car's sensors. This internal factor led to the car not detecting the red light and proceeding through the intersection [71269]. The company, Mobileye, acknowledged this issue and mentioned taking steps to better shield the equipment from electromagnetic radiation, indicating an internal system adjustment to prevent similar failures in the future.
Nature (Human/Non-human) non-human_actions (a) The software failure incident in the self-driving car prototype running a red light in Jerusalem was attributed to a non-human action. Specifically, it was reported that a GoPro camera attached to the car interfered with the car's sensors, leading to the car cruising through the red light without the safety driver taking corrective action [71269].
Dimension (Hardware/Software) hardware (a) The software failure incident in the article was related to hardware. The incident occurred when a self-driving car prototype built by Mobileye, owned by Intel, ran a red light in Jerusalem. It was reported that a GoPro camera attached to the car interfered with the car's sensors, leading to the car cruising through the red light. Mobileye's CEO mentioned that they had not anticipated such a situation and are taking steps to better shield the equipment from electromagnetic radiation, indicating that the hardware (GoPro camera) interference caused the failure [71269].
Objective (Malicious/Non-malicious) non-malicious (a) The software failure incident described in the article is non-malicious. The incident where the self-driving car ran a red light in Jerusalem was attributed to a GoPro camera attached to the car interfering with the car's sensors, causing it to fail to detect the red light and stop at the intersection. Mobileye's CEO mentioned that they had never anticipated such a situation, indicating that the failure was not due to malicious intent but rather an unforeseen technical issue [71269].
Intent (Poor/Accidental Decisions) poor_decisions (a) The software failure incident involving the self-driving car prototype in Jerusalem can be attributed to poor decisions. The incident occurred when the car ran a red light with a TV news crew on board. The CEO of Mobileye mentioned that the failure was due to a GoPro camera attached to the car interfering with the car's sensors, leading to the unexpected behavior of running the red light. This indicates that the decision to attach the GoPro camera in a way that interfered with the car's sensors was a poor decision, contributing to the failure [71269].
Capability (Incompetence/Accidental) development_incompetence, accidental (a) The software failure incident in the article can be attributed to development incompetence. The incident where the self-driving car ran a red light in Jerusalem was caused by a GoPro camera attached to the car interfering with the car's sensors. Mobileye's CEO mentioned that they had never anticipated something like this happening, indicating a lack of foresight in considering such scenarios during the development of the software [71269]. (b) Additionally, the incident can also be categorized as accidental. The interference caused by the GoPro camera was an accidental factor that led to the failure of the software in detecting the red light. This accidental interference was not intentional but had a significant impact on the car's behavior, resulting in the software failure incident [71269].
Duration temporary (a) The software failure incident in the article appears to be temporary. The incident where the self-driving car ran a red light in Jerusalem was attributed to a specific circumstance - the interference caused by a GoPro attached to the car's sensors. Mobileye's CEO mentioned that they had never anticipated something like this happening, indicating that it was a unique situation [71269].
Behaviour crash, omission, timing, value, other (a) crash: The self-driving car prototype experienced a failure when it ran a red light in Jerusalem with a TV news crew on board. The safety driver behind the wheel did not take any action to correct the mistake, indicating a crash in the system's intended function [71269]. (b) omission: The failure occurred when the car passed through the red light without stopping, omitting the intended function of obeying traffic signals [71269]. (c) timing: The incident involved the car cruising through the intersection when the light had turned red, indicating a timing issue where the system performed its function (moving through the intersection) at the wrong time (when the light was red) [71269]. (d) value: The failure was due to the system performing its intended function of driving, but incorrectly as it ran a red light, indicating a value-related failure [71269]. (e) byzantine: There is no indication in the article that the failure involved inconsistent responses or interactions, so a byzantine behavior is not evident in this incident [71269]. (f) other: The failure was unique and unexpected, with the cause being attributed to electromagnetic interference from a GoPro attached to the car, leading to a behavior not fitting into the defined categories [71269].

IoT System Layer

Layer Option Rationale
Perception sensor (a) The failure was related to the sensor: The incident with the self-driving car in Jerusalem was attributed to a GoPro camera attached to the car interfering with the car's sensors, causing it to run a red light. Mobileye's CEO mentioned that they had never anticipated such a situation and are taking steps to better shield the equipment from electromagnetic radiation, indicating that the sensor error played a significant role in the failure [71269].
Communication unknown The software failure incident involving the self-driving car prototype in Jerusalem was not directly related to the communication layer of the cyber physical system. The failure was attributed to a GoPro camera attached to the car interfering with the car's sensors, causing it to run a red light. This issue was not specifically linked to the communication layer of the system but rather to sensor interference [71269].
Application FALSE The software failure incident involving the self-driving car prototype in Jerusalem, as reported in Article 71269, was not directly related to the application layer of the cyber physical system. The failure was attributed to the interference caused by a GoPro camera attached to the car, which affected the car's sensors and led to the car running a red light. This issue was not caused by bugs, operating system errors, unhandled exceptions, or incorrect usage typically associated with application layer failures [71269].

Other Details

Category Option Rationale
Consequence death, harm, non-human (a) death: People lost their lives due to the software failure - The article mentions a pedestrian that was struck and killed by a self-driving Uber prototype vehicle in Tempe, Ariz., in March. Mobileye's CEO stated that Mobileye's software would have spotted the pedestrian [71269].
Domain transportation (a) The failed system was intended to support the transportation industry as it was a self-driving car prototype built by Mobileye, a company that supplies cameras to major automakers for use in semi-autonomous systems [71269]. (m) The failed system was not related to any other industry mentioned in the options.

Sources

Back to List