Incident: Accidental Human Error Leads to Bing Search Failure on "Tank Man"

Published Date: 2021-06-04

Postmortem Analysis
Timeline 1. The software failure incident with Microsoft's Bing search engine not showing image results for the query "tank man" happened on June 4, as reported in Article 116046.
System 1. Bing search engine 2. Image-recognition software used by Bing 3. DuckDuckGo search engine (which licenses results from Microsoft) [116046]
Responsible Organization 1. Microsoft Corp was responsible for causing the software failure incident with its Bing search engine not showing image results for the query "tank man" due to accidental human error [116046].
Impacted Organization 1. Users in the United States, United Kingdom, Germany, and Singapore were impacted by the software failure incident related to the Bing search engine not showing image results for the query "tank man" [116046].
Software Causes 1. Accidental human error in the Bing search engine led to the issue of not showing image results for the query "tank man" [116046].
Non-software Causes 1. Accidental human error in content moderation by Microsoft (Article 116046) 2. Possible purposeful suppression at the request of a powerful state (Article 116046)
Impacts 1. Users in various countries, including the United States, United Kingdom, Germany, and Singapore, were unable to see image results for the query "tank man" on Microsoft's Bing search engine, leading to concerns about possible censorship around the Tiananmen Square crackdown anniversary [116046]. 2. The search results for "tank man" on Bing initially returned the message "There are no results for tank man," causing confusion and frustration among users [116046]. 3. The incident raised questions about the accuracy and reliability of content moderation on search engines, with concerns that such errors could either be accidental or purposeful suppression at the request of a powerful state [116046]. 4. Smaller search engines like DuckDuckGo, which license results from Microsoft, also faced similar issues with "tank man" searches, indicating a broader impact beyond just Bing [116046]. 5. The incident highlighted the potential influence of China, where a significant percentage of Microsoft employees working on Bing are based, on search engine results and censorship practices, even outside of China's jurisdiction [116046].
Preventions 1. Implementing robust quality assurance processes to catch and prevent accidental human errors in content moderation, such as implementing thorough testing protocols before deploying changes to the search engine [116046]. 2. Conducting regular audits and reviews of search engine algorithms and results to ensure transparency and accuracy, potentially involving external oversight or audits to maintain integrity [116046]. 3. Enhancing employee training and awareness on sensitive topics like censorship and ensuring a clear understanding of ethical guidelines and principles to avoid unintentional biases or errors [116046].
Fixes 1. Implementing stricter quality control measures in content moderation processes to prevent accidental errors [116046]. 2. Conducting thorough training and education for employees involved in image recognition software to enhance accuracy and prevent similar incidents in the future [116046]. 3. Reviewing and updating search engine algorithms to ensure that censorship requests from powerful states do not influence search results inappropriately [116046].
References 1. David Greene, civil liberties director at the nonprofit Electronic Frontier Foundation [116046] 2. Microsoft Corp [116046] 3. DuckDuckGo [116046] 4. Google [116046] 5. Former employee of Microsoft [116046]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) The software failure incident related to the Bing search engine not showing image results for the query "tank man" due to accidental human error happened at Microsoft Corp. This incident was acknowledged by Microsoft, and they mentioned actively working to resolve the issue [116046]. (b) The incident also affected smaller search engines such as DuckDuckGo that license results from Microsoft. These search engines faced similar issues around "tank man" searches, indicating that the software failure was not limited to Microsoft but also impacted other organizations using Microsoft's services [116046].
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase is evident in the article. Microsoft's Bing search engine failed to show image results for the query "tank man" due to "accidental human error" in the system. This failure was attributed to a mistake introduced during the development or maintenance of the system, leading to the censorship of search results related to the Tiananmen Square crackdown anniversary [116046]. (b) The software failure incident related to the operation phase is also highlighted in the article. Users reported that when they performed the search query "tank man" on Bing, they received a message stating, "There are no results for tank man." This failure was a result of the operation or use of the system, where users were unable to access the desired search results due to the system error introduced during operation [116046].
Boundary (Internal/External) within_system (a) The software failure incident related to the Bing search engine not showing image results for the query "tank man" was within the system. Microsoft attributed the issue to "accidental human error" [Article 116046]. The incident was acknowledged by Microsoft, and they mentioned actively working to resolve the issue, indicating that the failure originated from within the system itself.
Nature (Human/Non-human) non-human_actions, human_actions (a) The software failure incident related to the Bing search engine not showing image results for the query "tank man" was attributed to "accidental human error" by Microsoft [116046]. This indicates that the failure was due to a contributing factor introduced without human participation, as it was unintentional. (b) However, there were concerns raised about possible censorship around the Tiananmen Square crackdown anniversary, suggesting that there could have been human actions influencing the incident. David Greene from the Electronic Frontier Foundation mentioned that "egregious mistakes are made all the time" in content moderation, hinting at the potential for human involvement in the failure [116046].
Dimension (Hardware/Software) software (a) The software failure incident in the article was not attributed to hardware issues. It was explicitly mentioned that Microsoft blamed "accidental human error" for the Bing search engine not showing image results for the query "tank man" [116046]. (b) The software failure incident in the article was attributed to accidental human error. Microsoft stated that the issue with Bing search results for "tank man" was "due to an accidental human error" [116046].
Objective (Malicious/Non-malicious) non-malicious (a) The software failure incident related to the Bing search engine not showing image results for the query "tank man" was initially suspected to be a case of purposeful suppression at the request of a powerful state, indicating a potential malicious intent behind the failure [116046]. However, Microsoft later attributed the issue to "accidental human error," suggesting a non-malicious cause for the failure [116046].
Intent (Poor/Accidental Decisions) accidental_decisions (a) The software failure incident related to the Bing search engine not showing image results for the query "tank man" was attributed to "accidental human error" by Microsoft. Users raised concerns about possible censorship around the Tiananmen Square crackdown anniversary. The incident was described as an accidental error by Microsoft, leading to the absence of search results for "tank man" initially. This suggests that the failure was not due to poor decisions but rather an accidental mistake [116046].
Capability (Incompetence/Accidental) accidental (a) The software failure incident in this case was attributed to "accidental human error" by Microsoft. Users reported that the Bing search engine was not showing image results for the query "tank man" in various countries, including the United States, United Kingdom, Germany, and Singapore. Microsoft acknowledged the issue and stated that it was "due to an accidental human error" and that they were actively working to resolve it [116046].
Duration temporary From the provided article [116046], the software failure incident related to Bing search engine not showing image results for the query "tank man" was temporary. Microsoft attributed the issue to "accidental human error" and mentioned actively working to resolve it. The incident was acknowledged and addressed by Microsoft, with the search results returning to normal after a certain period of time. Additionally, smaller search engines like DuckDuckGo, which license results from Microsoft, also faced similar issues but expected a fix soon.
Behaviour omission, value, other (a) crash: The software failure incident in the article is not described as a crash where the system loses state and does not perform any of its intended functions [116046]. (b) omission: The software failure incident in the article is related to the system omitting to perform its intended functions at an instance(s). Users reported that when they searched for "tank man" on Bing, the search engine returned the message "There are no results for tank man," indicating an omission of search results for that query [116046]. (c) timing: The software failure incident in the article is not related to the system performing its intended functions correctly but too late or too early [116046]. (d) value: The software failure incident in the article is related to the system performing its intended functions incorrectly. Users were not getting the expected image results for the query "tank man" on Bing, which led to concerns about possible censorship around the Tiananmen Square crackdown anniversary [116046]. (e) byzantine: The software failure incident in the article is not described as a byzantine failure where the system behaves erroneously with inconsistent responses and interactions [116046]. (f) other: The software failure incident in the article could be categorized as an "other" behavior. The incident involved the system not showing image results for a specific query, which could be seen as a form of suppression or censorship. This behavior could be considered unique and not fitting into the defined categories of failure behaviors [116046].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence theoretical_consequence The consequence of the software failure incident related to the Bing search engine not showing image results for the query "tank man" was primarily categorized as (h) theoretical_consequence. The incident raised concerns about possible censorship around the Tiananmen Square crackdown anniversary, with users speculating on whether it was purposeful suppression at the request of a powerful state. There were discussions about the implications of content moderation and the challenges of achieving perfection in such processes, but there were no direct observed consequences such as death, harm, basic needs impact, property loss, or delays reported in the articles [116046].
Domain information [a116046] The software failure incident reported in the article is related to the information industry. Specifically, it involves Microsoft's Bing search engine not showing image results for the query "tank man" in various countries, including the United States, the United Kingdom, Germany, and Singapore. This incident impacted the production and distribution of information, highlighting the importance of accurate search results in the information industry.

Sources

Back to List