Incident: Title: NYSE Trading Halted Due to Internal Technical Issue

Published Date: 2015-07-08

Postmortem Analysis
Timeline 1. The software failure incident at the New York Stock Exchange (NYSE) happened on July 8, 2015, as reported in Article 37887.
System 1. NYSE trading system 2. Nasdaq trading system 3. London Stock Exchange trading system 4. NZX trading system 5. Circuit breakers system approved by the Securities and Exchange Commission [37887]
Responsible Organization 1. Internal technical issue at the New York Stock Exchange (Article 37887)
Impacted Organization 1. New York Stock Exchange (NYSE) - Article [37887] 2. Nasdaq - Article [37887] 3. London Stock Exchange - Article [37887] 4. New Zealand Stock Exchange (NZX) - Article [37887]
Software Causes 1. In 1994, Nasdaq shut down for more than half an hour due to a squirrel chewing through a power line, with other problems resulting from new software and a faulty disk drive [37887]. 2. In 2013, Nasdaq experienced a technical failure that paralyzed trading for three hours, with Nasdaq suggesting that garbled data from NYSE triggered the outage [37887].
Non-software Causes 1. A squirrel chewing through a power line causing a shutdown at Nasdaq in 1994 [37887]. 2. Garbled data from NYSE triggering an outage at Nasdaq in 2013 [37887]. 3. Trading tactics leading to price distortions in the "flash crash" of May 6, 2010 [37887].
Impacts 1. The software failure incident at the New York Stock Exchange led to the halt of trading, causing the market to be closed for about an hour [37887]. 2. The incident raised concerns about the reliability of the exchanges in guaranteeing a reliable and open market for American investors [37887]. 3. Nasdaq also faced a serious outage in 2013 due to a technical failure, which paralyzed trading for three hours [37887]. 4. The software failure incidents at various stock exchanges, including the NYSE and Nasdaq, have sparked controversies and discussions about the ability of these exchanges to handle the complexities of the market [37887].
Preventions 1. Implementing robust software testing procedures to identify and address any potential bugs or faults before they impact trading operations [37887]. 2. Conducting regular maintenance and updates on critical systems to ensure they are functioning properly and are up to date with the latest security patches [37887]. 3. Enhancing cybersecurity measures to protect against potential cyber breaches that could lead to system disruptions [37887].
Fixes 1. Implementing robust backup systems and redundancy measures to prevent future technical issues like the one experienced by the NYSE [37887]. 2. Conducting thorough testing and quality assurance of software updates to avoid introducing new software-related problems that could lead to outages [37887]. 3. Enhancing cybersecurity measures to protect against potential cyber breaches that could disrupt trading operations [37887].
References 1. NYSE official statement - NYSE Twitter account [37887] 2. Historical incidents at NYSE and Nasdaq - NYSE's website [37887] 3. Previous Nasdaq outage in 2013 - Nasdaq [37887] 4. Previous Nasdaq outage due to garbled data from NYSE - Nasdaq [37887] 5. Software failure at London Stock Exchange in 2008 - London Stock Exchange [37887] 6. Software failure at NZX in the previous summer - NZX [37887] 7. Flash crash incident in May 6, 2010 - Authorities and experts [37887]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) The software failure incident having happened again at one_organization: - The article mentions that in 1994, Nasdaq shut down for more than half an hour after a squirrel chewed through a power line. This outage was the third in just a few weeks, with the other problems resulting from new software and a faulty disk drive. It was also noted that it was the second time a squirrel had caused problems for the exchange, with the first incident occurring in 1987 [37887]. (b) The software failure incident having happened again at multiple_organization: - The article discusses how the London Stock Exchange experienced a software failure in 2008, which stalled trading in major companies' stocks for several hours. Additionally, New Zealand's market, the NZX, blamed a three-hour interruption on technical problems [37887].
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase can be seen in the article where it mentions that Nasdaq experienced an outage in 1994 when a squirrel chewed through a power line, causing an outage due to new software and a faulty disk drive [37887]. This incident highlights how factors introduced during system development, such as new software, contributed to the failure. (b) The software failure incident related to the operation phase is evident in the article discussing Nasdaq's outage in 2013, where a technical failure paralyzed trading for three hours. Nasdaq suggested that garbled data from NYSE triggered the outage, indicating issues related to the operation or misuse of the system [37887].
Boundary (Internal/External) within_system (a) The software failure incident at the New York Stock Exchange (NYSE) in 2015 was within_system. The NYSE stated that the cause of the trading halt was an "internal technical issue" and clarified that it was not the result of a cyber breach [37887]. This indicates that the failure originated from within the NYSE's system itself. Additionally, the incident was compared to a previous shutdown in 2005, which was also attributed to a systems communications problem within the exchange [37887]. (b) There is no specific information in the provided articles indicating that the software failure incident at the NYSE in 2015 was due to contributing factors originating from outside the system. The focus of the reports is on internal technical issues and past incidents related to system failures within the exchange [37887].
Nature (Human/Non-human) non-human_actions (a) The software failure incident occurring due to non-human actions: - In 1994, Nasdaq shut down for more than half an hour after a squirrel chewed through a power line, causing an outage. This incident was particularly memorable as it was the third outage in just a few weeks, with other problems resulting from new software and a faulty disk drive [37887]. - Nasdaq's most recent serious outage in 2013 was attributed to a technical failure that paralyzed trading for three hours. Nasdaq suggested that garbled data from NYSE, its rival, had triggered the outage [37887]. (b) The software failure incident occurring due to human actions: - The articles do not provide specific instances of software failure incidents caused by human actions.
Dimension (Hardware/Software) hardware, software (a) The software failure incident occurring due to hardware: - In 1994, Nasdaq shut down for more than half an hour after a squirrel chewed through a power line, causing an outage. This incident was particularly memorable as it was the third outage in just a few weeks, with the other problems resulting from new software and a faulty disk drive [37887]. - The article mentions a previous incident in 1987 where a squirrel caused problems for the exchange, indicating hardware-related issues [37887]. (b) The software failure incident occurring due to software: - In 2013, Nasdaq experienced a serious outage when a technical failure paralyzed trading for three hours. Nasdaq suggested that garbled data from NYSE, its rival, had triggered the outage, pointing to a software-related issue [37887].
Objective (Malicious/Non-malicious) non-malicious (a) The software failure incident at the New York Stock Exchange in 2015 was classified as non-malicious. The NYSE stated that the cause of the trading halt was an "internal technical issue" and explicitly mentioned that it was not the result of a cyber breach [37887]. This indicates that the failure was not caused by malicious intent but rather by technical issues within the system.
Intent (Poor/Accidental Decisions) accidental_decisions (a) The software failure incident at the New York Stock Exchange in 2015 was not due to a cyber breach but was attributed to an "internal technical issue" [37887]. This indicates that the incident was likely not caused by deliberate actions but rather by poor decisions or contributing factors introduced by internal technical issues. (b) The article mentions past incidents where software failures occurred due to various reasons such as a squirrel chewing through a power line causing a shutdown, new software issues, faulty disk drives, and garbled data triggering outages [37887]. These incidents suggest that software failures in the financial markets can also be attributed to accidental decisions or mistakes rather than deliberate actions.
Capability (Incompetence/Accidental) accidental (a) The software failure incident related to development incompetence is not explicitly mentioned in the provided article. Therefore, it is unknown whether the NYSE shutdown or other market interruptions were caused by factors introduced due to lack of professional competence by humans or development organizations. (b) The software failure incident related to accidental factors is evident in the article. For example, the Nasdaq shutdown in 1994 was caused by a squirrel chewing through a power line, leading to a more than half-hour outage. This accidental incident was particularly memorable as it was the third outage in a few weeks, with other problems resulting from new software and a faulty disk drive [37887].
Duration permanent, temporary The articles provide information about both permanent and temporary software failure incidents: (a) Permanent software failure incident: - The article mentions a software failure incident in 2013 where a technical failure paralyzed trading on Nasdaq for three hours [37887]. - The article also discusses a software failure at the London Stock Exchange in 2008 that stalled trading in major companies' stocks for several hours [37887]. (b) Temporary software failure incident: - The article mentions a software failure incident at the NYSE in 2015, where trading was halted due to an internal technical issue but not due to a cyber breach. The market reopened the following day, indicating a temporary interruption [37887]. - Additionally, the article talks about Nasdaq experiencing an outage in 1994 when a squirrel chewed through a power line, causing a shutdown for more than half an hour. This incident, along with others, suggests temporary interruptions due to specific circumstances [37887].
Behaviour crash, omission, value, other (a) crash: The New York Stock Exchange (NYSE) halted trading due to an "internal technical issue" which caused a system shutdown, leading to the market being closed for about an hour [37887]. (b) omission: The Nasdaq exchange experienced a shutdown in 1994 when a squirrel chewed through a power line, causing an outage. This incident was one of a series of problems, including other outages resulting from new software and a faulty disk drive [37887]. (c) timing: The NYSE had a previous incident in 2005 when trading was halted due to a systems communications problem just four minutes before closing. The market reopened the following day [37887]. (d) value: In 2013, Nasdaq faced a serious outage for three hours due to a technical failure, which was later suggested to have been triggered by garbled data from NYSE, Nasdaq's rival [37887]. (e) byzantine: The articles do not provide information on a byzantine behavior of the software failure incident. (f) other: The articles mention controversial trading tactics that can distort prices, such as the "flash crash" in 2010, where prices were wildly distorted. Authorities later arrested a trader in connection with the crash, although some experts believed his actions couldn't fully explain the disruption [37887].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence theoretical_consequence, unknown (a) unknown (b) unknown (c) unknown (d) [37887] The software failure incident at the New York Stock Exchange did not result in any direct harm or loss of life. However, there were potential consequences discussed regarding the reliability and trustworthiness of the stock market exchanges in ensuring a stable and open market for investors.
Domain finance (a) The software failure incident reported in the articles is related to the finance industry. The New York Stock Exchange (NYSE) halted trading due to an internal technical issue, which impacted the trading of stocks and securities [37887]. The incident highlighted the challenges faced by stock exchanges in ensuring reliable and open markets for investors in the finance sector. Additionally, the article mentions past incidents in the finance industry, such as Nasdaq's outage in 2013 and the "flash crash" in 2010, which further emphasize the connection to the finance industry.

Sources

Back to List