Incident: WatchGuard Firewall Vulnerability Exploited by Russian Hackers, Delayed Disclosure

Published Date: 2022-04-08

Postmortem Analysis
Timeline 1. The software failure incident involving WatchGuard's critical vulnerability in its firewall devices happened in May 2021 [126498]. 2. The incident was further highlighted in November, when FBI agents informed WatchGuard about infections by Cyclops Blink [126498]. 3. The incident was publicly disclosed on February 24, following the court document being unsealed [126498].
System 1. WatchGuard Firebox and XTM appliances [126498]
Responsible Organization 1. WatchGuard [126498]
Impacted Organization 1. WatchGuard customers were impacted by the software failure incident [126498].
Software Causes 1. The software cause of the failure incident was a critical vulnerability in WatchGuard firewalls that allowed unauthorized remote access to the management panels of the devices [126498].
Non-software Causes 1. Lack of explicit disclosure by WatchGuard regarding the critical vulnerability in their firewall devices [126498]. 2. Delay in assigning a CVE by WatchGuard even after being informed by the FBI about the infections [126498]. 3. Failure of WatchGuard to respond to requests for clarification or comment on the incident [126498].
Impacts 1. The software failure incident led to WatchGuard firewalls being infected by malware developed by a Russian hacking group, resulting in the firewalls becoming part of a vast botnet [126498]. 2. The vulnerability in the WatchGuard firewalls allowed unauthorized remote access to the management panels of the devices, putting customers at risk of exploitation [126498]. 3. WatchGuard's failure to explicitly disclose the critical vulnerability in a timely manner resulted in security professionals criticizing the company for putting their customers at unnecessary risk [126498].
Preventions 1. Timely and transparent disclosure of the critical vulnerability by WatchGuard when it was first fixed in the May 2021 software updates could have prevented the incident [126498]. 2. Assigning a CVE (Common Vulnerabilities and Exposures) when the vulnerability was fixed in the software updates would have helped raise awareness and prompt customers to take necessary actions to secure their devices [126498]. 3. Responding promptly to notifications from law enforcement agencies, such as the FBI, about infections and vulnerabilities in their products could have mitigated the risk to customers [126498].
Fixes 1. WatchGuard fixed the critical vulnerability in its firewall devices by releasing software updates starting in May 2021 that addressed the CVE-2022-23176 vulnerability [126498].
References 1. FBI agent (unsealed court documents) [126498] 2. Security professionals, including Will Dormann (vulnerability analyst at CERT) [126498]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization (a) The software failure incident related to WatchGuard's firewall devices being vulnerable to a critical exploit has happened again within the same organization. The incident involved a critical vulnerability (CVE-2022-23176) that allowed unauthorized remote access to the management panels of WatchGuard firewalls. This vulnerability was exploited by hackers from Russia's military apparatus to assemble a giant botnet. WatchGuard released a software tool and instructions for identifying and locking down infected devices after law enforcement agencies warned them about the exploitation [126498]. (b) There is no specific information in the provided article about the software failure incident happening again at other organizations or with their products and services.
Phase (Design/Operation) design, operation (a) The software failure incident in the article can be attributed to the design phase. WatchGuard had a critical vulnerability in its firewall devices that allowed unauthorized remote access to the management panels of those devices. This vulnerability, identified as CVE-2022-23176, was present in the Fireware OS versions before 12.7.2_U1, 12.x before 12.1.3_U3, and 12.2.x through 12.5.x before 12.5.7_U3. The vulnerability was fully addressed by security fixes that started rolling out in software updates in May 2021. However, WatchGuard did not explicitly disclose this vulnerability until much later, putting their customers at unnecessary risk [126498]. (b) The software failure incident can also be linked to the operation phase. The FBI informed WatchGuard in November that about 1 percent of the firewalls it had sold had been infected by a new strain of malware called Cyclops Blink, developed by the Russian hacking group Sandworm. Despite being aware of these infections, WatchGuard only released a detection tool and remediation plan for infected devices three months later. This delay in addressing the operational impact of the malware infection also contributed to the overall failure incident [126498].
Boundary (Internal/External) within_system (a) The software failure incident reported in the articles is primarily within_system. The critical vulnerability in WatchGuard's firewall devices, which allowed unauthorized remote access to the management panels, was a result of an exploit within the system itself. The vulnerability, identified as CVE-2022-23176, was present in the Fireware OS versions before 12.7.2_U1, 12.x before 12.1.3_U3, and 12.2.x through 12.5.x before 12.5.7_U3 [126498]. The failure to explicitly disclose this vulnerability by WatchGuard despite knowing about it and releasing software updates contributed to the incident originating from within the system.
Nature (Human/Non-human) non-human_actions, human_actions (a) The software failure incident in this case was primarily due to non-human actions, specifically a critical vulnerability in WatchGuard firewalls that allowed unauthorized remote access to the management panels of the devices. This vulnerability was exploited by hackers from Russia's military apparatus to assemble a giant botnet [126498]. (b) Human actions also played a role in this software failure incident as WatchGuard did not explicitly disclose the critical vulnerability until after law enforcement agencies warned them about the exploitation by hackers. The company released a detection tool and instructions for identifying and locking down infected devices only after being informed by the FBI, which led to criticism from security professionals for not assigning a CVE earlier and for withholding technical details to prevent exploitation [126498].
Dimension (Hardware/Software) hardware, software (a) The software failure incident reported in the article is primarily related to a critical vulnerability in a line of WatchGuard firewall devices. The vulnerability allowed unauthorized remote access to the management panels of the devices, indicating a hardware-related issue [126498]. (b) The software failure incident is also related to contributing factors originating in software. The vulnerability, identified as CVE-2022-23176, was fully addressed by security fixes that started rolling out in software updates in May 2021. The failure to explicitly disclose the critical vulnerability in the software updates led to criticism from security professionals, as it put customers at unnecessary risk [126498].
Objective (Malicious/Non-malicious) malicious (a) The software failure incident in this case was malicious. The failure was due to a critical vulnerability in WatchGuard firewalls that was exploited by hackers from Russia's military apparatus to assemble a giant botnet. The hackers infected the firewalls with malware, making them part of a vast botnet controlled by a Russian hacking group. The vulnerability allowed unauthorized remote access to the management panels of the devices, putting customers at unnecessary risk [126498].
Intent (Poor/Accidental Decisions) poor_decisions (a) The software failure incident reported in Article 126498 can be attributed to poor_decisions. WatchGuard quietly fixed a critical vulnerability in its firewall devices and did not explicitly disclose the flaw until after law enforcement agencies warned that a Russian hacking group had infected some of its firewalls [126498]. The company released a detection tool for customers and did not disclose the vulnerability even after FBI agents informed them about infections by a new strain of malware developed by Sandworm [126498]. Security professionals criticized WatchGuard for not explicitly disclosing the critical vulnerability, which put their customers at unnecessary risk [126498].
Capability (Incompetence/Accidental) development_incompetence (a) The software failure incident in Article 126498 can be attributed to development incompetence. WatchGuard quietly fixed a critical vulnerability in its firewall devices and did not explicitly disclose the flaw until after hackers exploited it en masse to assemble a giant botnet. The company released a detection tool for customers only after law enforcement agencies warned them about the Russian hacking group infecting their firewalls with malware. Security professionals criticized WatchGuard for failing to explicitly disclose the vulnerability, which put their customers at unnecessary risk [126498]. (b) The software failure incident in Article 126498 does not seem to be accidental. The failure was a result of a critical vulnerability in WatchGuard's firewall devices that was exploited by hackers, leading to the creation of a botnet. The company's delayed response and lack of explicit disclosure of the flaw indicate a failure in addressing the issue promptly rather than an accidental introduction of contributing factors [126498].
Duration temporary (a) The software failure incident in this case can be considered as temporary. The vulnerability in the WatchGuard firewalls, identified as CVE-2022-23176, was fully addressed by security fixes that started rolling out in software updates in May 2021. However, the company did not explicitly disclose this critical vulnerability until much later, even after being informed by the FBI about infections in November. This delayed disclosure put customers at unnecessary risk, as threat actors were able to exploit the vulnerability during this period [126498].
Behaviour crash, omission, value, other (a) crash: The software failure incident in the article can be categorized as a crash. The vulnerability in WatchGuard firewalls allowed unauthorized remote access to the management panels of the devices, leading to a loss of control over the system's state and potentially compromising its intended functions [126498]. (b) omission: The incident can also be categorized as an omission. WatchGuard did not explicitly disclose the critical vulnerability in its firewalls until after law enforcement agencies warned about the exploitation by hackers. This omission to disclose the flaw promptly led to customers being unaware of the risks associated with their devices [126498]. (c) timing: The timing of the software failure incident can be considered a factor in this case. WatchGuard released software updates in May 2021 that addressed the vulnerability, but the explicit disclosure of the flaw and the CVE designation did not occur until much later, after law enforcement intervention and customer infections. The delayed timing in informing customers and assigning a CVE impacted the effectiveness of the response to the vulnerability [126498]. (d) value: The incident can also be related to a failure in value. The software vulnerability allowed remote attackers to access the system with privileged management sessions, indicating that the system was performing its intended functions incorrectly by granting unauthorized access [126498]. (e) byzantine: The behavior of the software failure incident does not align with a byzantine failure, which involves inconsistent responses and interactions. The incident in the article primarily revolves around a critical vulnerability in the WatchGuard firewalls that allowed unauthorized access, rather than erratic or inconsistent behavior [126498]. (f) other: The other behavior observed in this software failure incident is the failure to proactively disclose critical vulnerabilities to customers. WatchGuard's decision not to explicitly disclose the flaw in their firewalls, despite being aware of the exploitation by hackers, can be considered a significant oversight that put customers at unnecessary risk [126498].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence property, non-human, theoretical_consequence (a) unknown (b) unknown (c) unknown (d) Property: The software failure incident impacted people's material goods, money, or data. WatchGuard firewalls were hacked by a Russian hacking group, leading to the devices being infected with malware and becoming part of a botnet. This incident put customers at unnecessary risk and potentially exposed more customers to exploitation [126498]. (e) unknown (f) Non-human: The software failure incident impacted non-human entities. WatchGuard firewalls were vulnerable to an exploit that allowed unauthorized remote access to the management panels of the devices. This vulnerability impacted Fireware OS versions before specific updates were released [126498]. (g) unknown (h) Theoretical_consequence: There were potential consequences discussed of the software failure that did not occur. Security professionals criticized WatchGuard for not explicitly disclosing the critical vulnerability that had been fixed in the May 2021 software updates. They mentioned that threat actors found and exploited the issues, and WatchGuard's behavior put their customers at unnecessary risk [126498]. (i) unknown
Domain information [a] The failed system in the incident was related to the **information** industry, specifically in the realm of cybersecurity. The software vulnerability in WatchGuard's firewall devices allowed unauthorized remote access to the management panels, leading to the exploitation by hackers from Russia's military apparatus to assemble a botnet [126498].

Sources

Back to List