Incident: WhatsApp's End-to-End Encryption Failure and Data Sharing Scandal

Published Date: 2021-09-07

Postmortem Analysis
Timeline 1. The software failure incident mentioned in the article happened in 2014 when Facebook purchased WhatsApp for $19 billion [118550].
System The system that failed in the software failure incident reported in Article 118550 is: 1. WhatsApp's promise of private messages with end-to-end encryption [118550]
Responsible Organization 1. Facebook - The software failure incident was caused by Facebook's actions of hiring contractors to sift through private messages on WhatsApp, contrary to the promise of end-to-end encryption and privacy made to users [118550].
Impacted Organization 1. WhatsApp users [118550]
Software Causes 1. Lack of true end-to-end encryption in WhatsApp despite claims made by Facebook and WhatsApp [118550] 2. Use of special Facebook software by contractors to sift through private messages, images, and videos reported by WhatsApp users, indicating potential flaws in the software system [118550] 3. Failure of WhatsApp's AI systems to accurately identify and handle reported content, leading to human intervention and potential errors in judgment [118550]
Non-software Causes 1. Lack of privacy protection as promised by WhatsApp and Facebook when the messaging app was acquired [118550]. 2. Hiring of contractors by Facebook to sift through and analyze millions of messages on WhatsApp, contradicting the assurance of data privacy [118550]. 3. Sharing of WhatsApp user data with law enforcement and the U.S. Department of Justice for criminal investigations, despite claims of end-to-end encryption [118550]. 4. Failure to maintain the expected level of privacy and security for users' messages on WhatsApp [118550].
Impacts 1. The software failure incident led to a breach of user trust as WhatsApp's promise of private messages with end-to-end encryption was revealed to be false [Article 118550]. 2. Facebook, the parent company of WhatsApp, was found to have hired contractors to sift through millions of private messages on WhatsApp, leading to concerns about privacy and data security [Article 118550]. 3. The incident resulted in WhatsApp sharing some of the messages with law enforcement and the U.S. Department of Justice, potentially compromising user privacy and confidentiality [Article 118550]. 4. Users expressed concerns and lack of surprise over the incident, indicating a potential erosion of trust in tech companies' ability to protect user data and privacy [Article 118550].
Preventions 1. Implementing stricter internal policies and controls to ensure that user data privacy promises are upheld, thus preventing misleading claims and potential breaches [118550]. 2. Conducting regular audits and oversight to verify that end-to-end encryption is maintained and that user data is not accessed or shared inappropriately [118550]. 3. Enhancing transparency with users about data handling practices, including any instances where data may be shared with law enforcement, to build trust and manage expectations [118550].
Fixes 1. Implement stricter privacy policies and practices to ensure end-to-end encryption is truly maintained as promised by WhatsApp and Facebook [118550]. 2. Enhance transparency regarding data sharing practices with law enforcement and other entities to build trust with users [118550]. 3. Conduct a thorough review of the content moderation processes to ensure that user privacy is respected while addressing illegal activities [118550].
References 1. ProPublica [Article 118550] 2. WhatsApp Head Will Cathcart [Article 118550] 3. WhatsApp Director of Communications, Carl Woog [Article 118550] 4. Facebook CEO Mark Zuckerberg [Article 118550]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) The software failure incident of privacy breach and data sharing by WhatsApp has happened again within the same organization, Facebook. The incident involved Facebook hiring contractors to sift through private messages on WhatsApp, despite assurances of end-to-end encryption and data privacy [118550]. (b) The software failure incident of privacy breach and data sharing similar to the WhatsApp incident has also occurred at other organizations or with their products and services. The article mentions that Facebook has a 95% rate of handing over 'at least some data' from its users when requested by law enforcement, indicating a broader issue of data privacy and sharing across tech companies [118550].
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase can be seen in the case of WhatsApp's promise of private messages with end-to-end encryption being revealed as false. Despite assurances from both WhatsApp and Facebook that user data could not be accessed, it was discovered that Facebook had hired contractors to sift through millions of messages on WhatsApp and shared some of those messages with law enforcement and the U.S. Department of Justice [Article 118550]. (b) The software failure incident related to the operation phase is evident in the fact that Facebook had hired contractors in different locations to look at millions of pieces of users' content on WhatsApp. These contractors, using special Facebook software, sifted through private messages, images, and videos reported by users as improper, passing judgment on various types of content in less than a minute [Article 118550].
Boundary (Internal/External) within_system, outside_system (a) within_system: The software failure incident in this case is related to the failure of WhatsApp to uphold its promise of private messages with end-to-end encryption. The failure originated from within the system as WhatsApp, despite claiming that user data could not be accessed by Facebook, was found to have hired contractors to sift through millions of messages on the platform and share some of those messages with law enforcement and the U.S. Department of Justice [Article 118550]. This failure was a result of decisions made internally within WhatsApp and Facebook, indicating a within_system contributing factor to the software failure incident. (b) outside_system: The software failure incident can also be attributed to contributing factors that originate from outside the system. For example, the incident involved law enforcement agencies and the U.S. Department of Justice requesting and receiving data from WhatsApp for use in criminal cases [Article 118550]. This external involvement and pressure from law enforcement agencies could be considered as outside_system contributing factors to the software failure incident.
Nature (Human/Non-human) non-human_actions, human_actions (a) The software failure incident occurring due to non-human actions: The failure in this case can be attributed to the fact that Facebook's special software and artificial intelligence systems were used to sift through private messages on WhatsApp, leading to the sharing of some of those messages with law enforcement and the U.S. Department of Justice [118550]. (b) The software failure incident occurring due to human actions: Human actions also played a significant role in this incident as Facebook hired contractors in different locations to manually look at millions of pieces of users' content on WhatsApp. These contractors were responsible for passing judgment on the reported content, including claims of fraud, spam, child porn, and potential terrorist plotting, typically in less than a minute [118550].
Dimension (Hardware/Software) software (a) The articles do not provide information about a software failure incident occurring due to hardware issues. (b) The software failure incident reported in the articles is related to software itself. The incident involves WhatsApp's failure to uphold its promise of private messages with end-to-end encryption. The investigation revealed that Facebook, the parent company of WhatsApp, had hired contractors to sift through millions of messages on WhatsApp, which goes against the assurance of data privacy given to users [Article 118550].
Objective (Malicious/Non-malicious) malicious, non-malicious (a) The software failure incident in the article can be categorized as malicious. The incident involved Facebook hiring contractors to sift through private messages on WhatsApp, which were supposed to be end-to-end encrypted and inaccessible to both companies. These contractors were tasked with reviewing millions of pieces of users' content, including private messages, images, and videos, to identify inappropriate content such as fraud, spam, child porn, and potential terrorist plotting. The data from WhatsApp was shared with law enforcement and the U.S. Department of Justice to aid in criminal investigations and prosecutions [118550]. This action goes against the initial promise of privacy and end-to-end encryption made to WhatsApp users, indicating a malicious intent to access and utilize private user data for purposes beyond what was disclosed. (b) The software failure incident can also be considered non-malicious to some extent. WhatsApp officials, including Head Will Cathcart and Director of Communications Carl Woog, defended the actions of reviewing and sharing data with law enforcement as necessary for maintaining security and safety on the platform. They argued that the contractors were hired to identify and remove abusive content and bad actors from the platform, rather than for content moderation purposes. WhatsApp emphasized that its decisions are focused on user privacy, reliability, and abuse prevention [118550]. This perspective suggests that the incident may have been driven by a perceived need to address security concerns and maintain the integrity of the platform, rather than a deliberate attempt to harm the system.
Intent (Poor/Accidental Decisions) poor_decisions (a) The intent of the software failure incident was poor_decisions. The incident involved Facebook's decision to hire contractors to sift through private messages on WhatsApp, despite assuring users that their data would remain private and inaccessible to the company [118550]. This decision led to a breach of trust with users and raised concerns about privacy and data security.
Capability (Incompetence/Accidental) development_incompetence, unknown (a) The software failure incident related to development incompetence is evident in the case of WhatsApp's failure to uphold its promise of end-to-end encryption and privacy for user messages. Despite assurances from WhatsApp and Facebook that user data would remain private, it was revealed that Facebook had hired contractors to sift through millions of messages on WhatsApp and share some of those messages with law enforcement and the U.S. Department of Justice [118550]. (b) The software failure incident related to accidental factors is not explicitly mentioned in the provided article.
Duration unknown The software failure incident reported in the articles does not align with the typical definitions of a permanent or temporary software failure incident. The incident described in the articles is more related to privacy concerns and the alleged false claims made by WhatsApp regarding end-to-end encryption and data privacy. Therefore, the concept of a software failure incident being permanent or temporary does not directly apply to the information provided in the articles.
Behaviour crash, omission, value, other (a) crash: The software failure incident related to WhatsApp can be categorized as a crash. This is evident from the fact that the investigation revealed that Facebook had hired contractors to sift through millions of messages on WhatsApp, which has two billion users around the world, and shared some of those messages with law enforcement and the U.S. Department of Justice to help put people in prison [Article 118550]. (b) omission: The incident can also be categorized as an omission failure. This is because the software failed to maintain the privacy and security promises made to users, as it was revealed that WhatsApp messages were not as private as claimed, and data was being shared with law enforcement and other entities [Article 118550]. (c) timing: There is no specific information in the article to categorize the incident as a timing failure. (d) value: The incident can be categorized as a value failure. This is evident from the fact that the software was not performing its intended function of keeping user data private and away from the hands of Facebook, as promised to users [Article 118550]. (e) byzantine: There is no specific information in the article to categorize the incident as a byzantine failure. (f) other: The incident can also be categorized as an "other" failure. This is because the software was behaving in a way that was not aligned with the expectations and promises made to users, leading to a breach of trust and privacy concerns [Article 118550].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence property, theoretical_consequence (a) unknown (b) unknown (c) unknown (d) Property: The software failure incident led to the impact on people's material goods, money, or data. The incident involved Facebook hiring contractors to sift through millions of pieces of users' content on WhatsApp, sharing some of those messages with law enforcement and the U.S. Department of Justice, and using the data to help put people in prison [118550]. (e) unknown (f) unknown (g) unknown (h) Theoretical_consequence: There were potential consequences discussed of the software failure that did not occur. The article mentions that an unnamed whistleblower had filed a complaint with the U.S. Securities and Exchange Commission, alleging that WhatsApp's claims of protecting users' privacy and data were false. However, the SEC has not taken action against the issue [118550]. (i) unknown
Domain information (a) The failed system in this incident is related to the information industry, specifically the messaging service provided by WhatsApp. The incident involved the revelation that WhatsApp's promise of private messages with end-to-end encryption was false, as Facebook, the parent company of WhatsApp, was found to have access to and shared users' messages with law enforcement and the U.S. Department of Justice [Article 118550]. This failure incident highlights a breach of trust in the information industry where users expected their messages to be private and secure.

Sources

Back to List