Incident: RBS IT Glitch Delays 600,000 Payments Causing Customer Disruption

Published Date: 2015-06-17

Postmortem Analysis
Timeline 1. The software failure incident at Royal Bank of Scotland (RBS) happened on June 17, 2015, as reported in Article 36886.
System 1. RBS's IT system [36886]
Responsible Organization 1. Royal Bank of Scotland (RBS) - The software failure incident was caused by RBS, as reported in the article [36886].
Impacted Organization 1. Customers of Royal Bank of Scotland, including NatWest and Ulster Bank [36886]
Software Causes 1. The software cause of the failure incident at the Royal Bank of Scotland was a computer problem that caused delays in payments entering customer accounts [36886].
Non-software Causes 1. Insufficient communication and customer support: The delay in payments and lack of immediate resolution led to customer frustration and inconvenience [36886]. 2. Operational issues: The failure to process payments on time resulted in customers missing out on essential funds, such as wages, tax credits, and disability allowances [36886]. 3. Human error: The delay in payments impacted individuals like Lynne Kenny, who did not receive her widow's pension and working tax credit due to the bank's failure [36886].
Impacts 1. Customers experienced missing payments, including wages, tax credits, and disability allowances, causing financial distress and inconvenience [36886]. 2. Some customers were unable to make important payments, such as monthly deliveries and government benefits, leading to disruptions in their daily lives [36886]. 3. One individual missed a job interview for a once-in-a-lifetime overseas job opportunity due to the delayed payment, resulting in personal disappointment and potential career setbacks [36886]. 4. The reputation of the Royal Bank of Scotland was further tarnished, with customers expressing frustration and concern over recurring IT failures, impacting trust in the bank [36886].
Preventions 1. Implementing rigorous testing procedures before deploying any software updates or changes could have helped prevent the software failure incident [36886]. 2. Investing in modernizing and upgrading the bank's IT infrastructure to prevent system failures and glitches could have been a preventive measure [36886]. 3. Enhancing cybersecurity measures to protect against potential hacks or unauthorized access that could lead to IT fiascos [36886].
Fixes 1. Implementing more robust testing procedures to catch potential issues before they impact customers [36886]. 2. Investing in upgrading and modernizing the bank's IT systems to prevent similar incidents from occurring in the future [36886]. 3. Ensuring prompt and effective communication with affected customers to provide updates and support during such incidents [36886].
References 1. RBS Help account on Twitter [36886] 2. Andrew Tyrie, chairman of the Treasury select committee [36886] 3. Lynne Kenny, an affected RBS customer [36886] 4. Jack Admans, an affected customer with thinkmoney account [36886] 5. RBS spokeswoman [36886] 6. Financial Conduct Authority (FCA) [36886]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) The software failure incident having happened again at one_organization: The Royal Bank of Scotland (RBS) experienced another IT fiasco in 2015, where 600,000 payments failed to enter accounts overnight, causing significant inconvenience to customers [36886]. This incident occurred seven months after RBS was fined for an IT meltdown in 2012 that left millions of customers locked out of their accounts for days. The previous incident in 2012 led to the bank being fined and the then CEO forfeiting his annual bonus [36886]. (b) The software failure incident having happened again at multiple_organization: The article mentions that Lloyds and Nationwide, among other lenders, have also faced problems with their systems, with Nationwide's customers experiencing online payment delays in the same week as the RBS incident [36886]. This indicates that software failure incidents are not isolated to just one organization but can affect multiple organizations in the banking sector.
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase can be attributed to contributing factors introduced by system development or updates. In the case of the Royal Bank of Scotland (RBS) IT fiasco, the incident was caused by a computer problem that led to delays in payments entering customer accounts [36886]. The bank mentioned that the computer problem causing the delays had been fixed, indicating a design or development issue that needed resolution to prevent similar incidents in the future. (b) The software failure incident related to the operation phase can be linked to contributing factors introduced by the operation or misuse of the system. In this case, customers were affected by missing or delayed payments, which impacted their financial situations and daily activities [36886]. The bank advised affected customers to visit branches or contact call centers for assistance, highlighting the operational aspect of dealing with the aftermath of the glitch. Additionally, customers expressed concerns and frustrations on social media platforms like Twitter, indicating the operational challenges faced by both the bank and its clients during the incident.
Boundary (Internal/External) within_system, outside_system (a) The software failure incident at the Royal Bank of Scotland (RBS) was primarily within the system. The incident was caused by a computer problem within RBS's systems that resulted in delayed payments to and from customers' accounts [36886]. The bank acknowledged that the payments were not missing but were delayed due to the glitch, which had been fixed [36886]. RBS assured customers that they would not lose money as a result of the issue [36886]. (b) Additionally, external factors such as customer reliance on these payments and the impact on their financial well-being were also evident. Customers expressed concerns and frustrations on social media platforms like Twitter about missing payments affecting their ability to pay for essential expenses like food and transportation [36886]. The incident had real-world consequences for individuals like Jack Admans, who missed a job interview due to not receiving his monthly wage on time [36886].
Nature (Human/Non-human) non-human_actions, human_actions (a) The software failure incident in the RBS case was primarily due to non-human actions. The incident was caused by a computer problem that resulted in delayed payments to and from customers' accounts, affecting hundreds of thousands of transactions [36886]. The bank mentioned that the payments were not missing but were delayed, indicating that the issue was related to a technical glitch rather than human error [36886]. (b) Human actions also played a role in exacerbating the impact of the software failure incident. For example, the bank faced criticism for the delay in resolving the issue, with customers expressing concerns about missing payments for essential expenses like food and transportation [36886]. Additionally, the bank's response to affected customers, such as advising them to visit branches or call centers for assistance, could be seen as a human action that influenced the customer experience during the incident [36886].
Dimension (Hardware/Software) software (a) The software failure incident occurring due to hardware: - The article does not mention any specific hardware-related issues contributing to the software failure incident at the Royal Bank of Scotland [36886]. (b) The software failure incident occurring due to software: - The software failure incident at the Royal Bank of Scotland was primarily attributed to a computer problem that caused delays in payments, which was later fixed [36886].
Objective (Malicious/Non-malicious) non-malicious (a) The software failure incident reported in the article is non-malicious. The incident at the Royal Bank of Scotland was due to a computer problem that caused delays in payments, affecting customers' accounts. The bank mentioned that the payments were not missing but were delayed, and they were working to update the payments for customers by a certain deadline [36886].
Intent (Poor/Accidental Decisions) poor_decisions (a) The software failure incident at the Royal Bank of Scotland (RBS) was primarily due to poor decisions. The incident was a result of an IT glitch that caused delays in processing payments for customers, affecting around 600,000 payments. This failure was not accidental but rather a consequence of underlying issues within the bank's systems. The bank had previously faced a similar IT meltdown in 2012, for which it was fined £56m, indicating a pattern of poor decision-making in managing its IT infrastructure [36886].
Capability (Incompetence/Accidental) development_incompetence (a) The software failure incident at the Royal Bank of Scotland (RBS) can be attributed to development incompetence. This is evident from the fact that RBS had previously experienced an IT meltdown in 2012, for which they were fined £56m. Despite this prior incident, another IT fiasco occurred in 2015, causing delays in processing payments for customers. The chairman of the Treasury select committee, Andrew Tyrie, criticized RBS for not acting quickly enough to restore the delayed payments, emphasizing that the restoration of payments should be a top priority, especially for those in financial need [36886]. (b) The software failure incident at RBS can also be considered accidental. The bank's spokeswoman mentioned that the payments were not missing but were delayed due to a computer problem that had been fixed. She assured customers that they would not lose money as a result of the glitch. Additionally, Thinkmoney, which uses RBS systems, expected the delayed payments to come through by Friday at the latest, indicating that the delay was unintentional and efforts were being made to rectify the situation [36886].
Duration temporary (a) The software failure incident reported in the articles is temporary. The incident involved delayed payments to and from customers' accounts, affecting hundreds of thousands of transactions. The bank acknowledged the issue and assured customers that the problem was fixed, with payments expected to be updated for customers by a certain deadline. The bank also stated that the computer problem causing the delays had been resolved and that customers would not lose money as a result of the glitch. Additionally, the bank advised affected customers to visit branches or contact call centers for assistance [36886].
Behaviour crash, omission, timing, value, other (a) crash: The software failure incident in the article can be categorized as a crash as it resulted in the system losing state and not performing its intended functions. Customers reported missing payments, including wages, tax credits, and disability allowances, due to the IT glitch at RBS [36886]. (b) omission: The incident also involved omission as the system failed to perform its intended functions at an instance(s). Customers experienced missing payments, such as monthly wages and benefits, which were not deposited into their accounts as expected [36886]. (c) timing: The timing aspect of the software failure incident is evident as the system performed its intended functions incorrectly by delaying payments. Customers were informed that the delayed payments would be processed no later than Saturday, causing inconvenience and financial distress [36886]. (d) value: The software failure incident can be linked to the value behavior as the system performed its intended functions incorrectly. Customers did not receive their expected payments, leading to financial difficulties and concerns about important payments not being made [36886]. (e) byzantine: There is no explicit mention of the software failure incident exhibiting a byzantine behavior with inconsistent responses and interactions in the provided article. (f) other: The other behavior observed in the software failure incident is the impact on customers' daily lives and critical events. For example, a customer missed a job interview due to not receiving their monthly wage on time, causing significant personal and professional consequences [36886].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence basic, property, delay (a) death: People lost their lives due to the software failure (b) harm: People were physically harmed due to the software failure (c) basic: People's access to food or shelter was impacted because of the software failure (d) property: People's material goods, money, or data was impacted due to the software failure (e) delay: People had to postpone an activity due to the software failure (f) non-human: Non-human entities were impacted due to the software failure (g) no_consequence: There were no real observed consequences of the software failure (h) theoretical_consequence: There were potential consequences discussed of the software failure that did not occur (i) other: Was there consequence(s) of the software failure not described in the (a to h) options? What is the other consequence(s)? The software failure incident at Royal Bank of Scotland resulted in delays for customers in receiving payments, causing inconvenience and financial stress. Some customers reported missing payments for wages, tax credits, disability allowances, and other essential funds [36886]. Customers expressed concerns about not being able to make important payments, such as for food or bills, due to the delays [36886]. Additionally, individuals like Jack Admans missed job interviews and faced financial difficulties as a result of the delayed payments [36886]. The incident impacted people's access to essential funds and caused delays in financial transactions.
Domain finance The software failure incident reported in the news article [36886] was related to the finance industry. The incident involved the Royal Bank of Scotland (RBS) experiencing an IT glitch that caused delays in processing payments for customers, including wages, tax credits, and disability allowances. This failure impacted various accounts under RBS, NatWest, Ulster Bank, and Coutts bank, which are all financial institutions owned by RBS. Customers reported missing payments, such as pensions, tax credits, and working allowances, due to the system failure. The glitch affected individuals' financial stability, causing inconvenience and distress to customers who were relying on timely payments [36886].

Sources

Back to List