Incident: AT&T 3G Network Shutdown Impacting Connected Car Features

Published Date: 2022-02-23

Postmortem Analysis
Timeline 1. The software failure incident of AT&T shutting down its 3G network, impacting connected features in millions of cars, occurred on February 22, 2022 [Article 124451].
System 1. 3G network system by AT&T [Article 124451] 2. Smart features in millions of cars, mostly released before 2017, relying on the 3G network [Article 124451] 3. Legacy monitoring devices used by the San Francisco Municipal Transportation Agency (SFTMA) [Article 124451] 4. 3G modems in vehicles from various manufacturers including Acura, Audi, BMW, GM, Honda, Hyundai, Jaguar, Land Rover, Nissan, Infiniti, Porsche, Subaru, Toyota, Lexus, Tesla, Volkswagen, and Volvo [Article 124451]
Responsible Organization 1. AT&T - AT&T is responsible for causing the software failure incident by shutting down its 3G network, leading to the loss of connected features in millions of cars [124451, 124451]. 2. Automotive manufacturers - Some automotive manufacturers were responsible for shipping vehicle models with 3G modems even in 2019, 2020, 2021, and 2022, despite warnings of the impending 3G shutdown, contributing to the software failure incident [124451]. 3. Sierra Wireless - Sierra Wireless, the firm responsible for tracking, explained the problems faced during the switch from 2G to 3G, highlighting past issues with network transitions [124451]. 4. Honda - Honda confirmed it would fix the issue with an over-the-air update and had been working on the problem since October, indicating some responsibility in addressing the software failure incident [124451].
Impacted Organization 1. Millions of cars, including models from manufacturers such as Acura, Audi, Ford, GM, Honda, Subaru, Tesla, Porsche, Volvo, Bentley, BMW, and Volkswagen, were impacted by the shutdown of AT&T's 3G network, leading to the loss of connected features like traffic navigation, emergency calls, and voice assist [124451, 124451]. 2. Other entities affected by the software failure incident include ATM machines, security systems, and some EV chargers that also relied on the older 3G network for connectivity [124451]. 3. The San Francisco Municipal Transportation Agency (SFTMA) faced issues during the switch from 2G to 3G in 2017, with about 70% of buses and trains not appearing on real-time tracking maps due to the network change [124451].
Software Causes 1. The software cause of the failure incident was the shutdown of AT&T's 3G network, which led to the loss of connectivity and functionality in millions of cars that relied on the 3G network for features like traffic navigation, emergency calls, and voice assist [124451, 124451]. 2. Some vehicle models released in 2019, 2020, 2021, and even 2022 still shipped with a 3G modem despite warnings about the impending shutdown, indicating a lack of software readiness and planning by manufacturers [124451]. 3. Manufacturers like Honda confirmed they would fix the issue with an over-the-air update and had been working on the problem since October, suggesting that software updates were necessary to address the connectivity issues caused by the 3G shutdown [124451].
Non-software Causes 1. The shutdown of AT&T's 3G network, leading to the discontinuation of connected features in millions of cars, was a result of the telecom giant freeing up frequencies for its faster and more reliable 5G infrastructure [124451]. 2. The decision to sunset the 3G network was planned by AT&T and other providers to retire the two-decade-old cellular standard in order to make space for a wider range of 5G bands [124451]. 3. Some vehicle models released in 2019, 2020, 2021, and even 2022 still shipped with a 3G modem despite warnings of the shutdown [124451]. 4. The global supply chain shortages caused delays in upgrading to 4G or LTE modems, leaving some drivers waiting until the end of the year to access smart features in their cars again [124451]. 5. The shutdown of the 3G network was part of a broader industry trend as other cellular providers like Sprint, T-Mobile, and Verizon also planned to shut down their 3G networks by the end of the year [124451].
Impacts 1. Millions of cars, particularly older models released before 2017, lost connected features such as traffic navigation, emergency calls, voice assist, in-car wifi, remote lock/unlock functions, and location data due to the shutdown of AT&T's 3G network [124451, 124451]. 2. Some vehicles, despite warnings, still shipped with 3G modems in 2019, 2020, 2021, and even 2022, causing a disruption in smart features [124451]. 3. The shutdown impacted various manufacturers' vehicles, including Acura, Audi, BMW, GM, Honda, Hyundai, Jaguar, Land Rover, Nissan, Infiniti, Porsche, Subaru, Toyota, Lexus, Tesla, Volkswagen, and Volvo [124451]. 4. Some manufacturers offered solutions such as over-the-air updates or hardware upgrades to transition from 3G to 4G modems, but delays were experienced due to global supply chain shortages [124451]. 5. The shutdown posed potential security risks as some cars couldn't receive updates without internet access, and the loss of safety features was a concern for vehicle owners [124451]. 6. The incident highlighted the need for automakers to plan for network transitions better, with some companies like GM and Honda having functionality built into modems for network switching [124451]. 7. The shutdown of 3G networks by AT&T was part of a broader industry trend as other cellular providers like Sprint, T-Mobile, and Verizon also planned to phase out their 3G networks, pushing for the adoption of 4G and 5G technologies [124451].
Preventions 1. Ensuring that all vehicle manufacturers transitioned their connected cars to newer technology before the 3G services ended on February 22, 2022, through effective communication and collaboration [Article 124451]. 2. Implementing proactive measures to upgrade vehicles with 3G modems to 4G or LTE modems before the shutdown of the 3G network, to avoid disruption of smart features in cars [Article 124451]. 3. Conducting thorough testing and quality assurance to ensure that all smart features in vehicles are compatible with the latest network technologies, thereby preventing future connectivity issues [Article 124451]. 4. Providing timely software updates and over-the-air upgrades to vehicles with outdated technology to maintain functionality and security features [Article 124451]. 5. Collaborating with cellular providers to ensure a smooth transition from older network technologies to newer ones, minimizing the impact on customers and their connected vehicles [Article 124451].
Fixes 1. Upgrading affected vehicles to 4G or LTE modems, either through physical updates, new vehicle purchases, or over-the-air updates, can fix the software failure incident [124451, 124451]. 2. Over-the-air updates provided by manufacturers like Honda to address the issue with the 3G shutdown can also resolve the software failure incident [124451]. 3. Some manufacturers, such as Subaru, offer free hardware upgrades from 3G to 4G modems for drivers with active subscriptions, while others may require drivers to pay for the upgrade [124451]. 4. Implementing solutions like Mojio's SyncUP DRIVE offering from T-Mobile, which provides connected services via a wifi hotspot for a monthly fee, can address the software failure incident for some vehicles [124451].
References 1. Consumer Reports [Article 124451] 2. AT&T spokesperson [Article 124451] 3. Sierra Wireless [Article 124451] 4. William Wallace, manager of safety policy for Consumer Reports [Article 124451] 5. Brian Moody, Autotrader executive editor [Article 124451] 6. Alex Knizek, expert from Consumer Reports [Article 124451] 7. Robert Boyanovsky, vice president of the Internet of Things and Mobility operation at AT&T Business [Article 124451] 8. Kyle MacDonald, VP of marketing and business development at Mojio [Article 124451]

Software Taxonomy of Faults

Category Option Rationale
Recurring one_organization, multiple_organization (a) The software failure incident related to the shutdown of the 3G network by AT&T for connected cars is not a new occurrence within the organization. AT&T had previously shut down its 2G infrastructure in 2017, which also caused issues with real-time tracking for transportation services in San Francisco [124451]. (b) The software failure incident of shutting down the 3G network for connected cars is not isolated to AT&T. Other cellular providers like Sprint, T-Mobile, and Verizon have also indicated plans to shut down their 3G networks by the end of the year, following a similar trajectory as AT&T in transitioning to newer technologies like 5G [124451].
Phase (Design/Operation) design, operation (a) The software failure incident related to the design phase can be seen in the shutdown of AT&T's 3G network, leading to the loss of connected features in millions of cars. This failure was due to the decision to sunset the 3G network to make way for 5G infrastructure, impacting various smart features in older car models that relied on the 3G network [124451]. (b) The software failure incident related to the operation phase can be observed in the impact on various services like in-traffic navigation, location data, in-car wifi, emergency callouts, and remote lock/unlock functions as a result of the 3G network shutdown. This failure was due to the operation and use of the affected systems that were dependent on the 3G network, which ceased to function [124451].
Boundary (Internal/External) within_system, outside_system (a) The software failure incident related to the shutdown of AT&T's 3G network and its impact on connected features in millions of cars falls under the category of within_system failure. This is because the failure is directly caused by the decision to shut down the 3G network, which is an internal decision made by AT&T to transition to newer technologies like 5G. The shutdown of the 3G network led to the loss of functionality in various smart features in cars that relied on the 3G network for connectivity [124451]. (b) The software failure incident can also be considered as an outside_system failure to some extent. This is because the decision to shut down the 3G network was influenced by external factors such as the need to free up frequencies for faster and more reliable 5G infrastructure. Additionally, the impact of the shutdown extended beyond just cars to other devices like ATM machines, security systems, and EV chargers that also relied on the 3G network [124451].
Nature (Human/Non-human) non-human_actions, human_actions (a) The software failure incident occurring due to non-human actions: - The software failure incident in this case is primarily due to the shutdown of AT&T's 3G network, which was a planned action by the telecom giant to free up frequencies for its faster and more reliable 5G infrastructure [124451]. - The shutdown of the 3G network led to the loss of connectivity and functionality in millions of cars, affecting features like traffic navigation, emergency calls, voice assist, in-car wifi, and remote lock and unlock functions [124451]. - The transition from 3G to newer technologies like 4G or LTE modems was necessary to address the issue, but delays in upgrades were expected due to global supply chain shortages [124451]. (b) The software failure incident occurring due to human actions: - Some vehicle models released as late as 2019, 2020, 2021, and even 2022 still shipped with a 3G modem despite warnings about the shutdown of 3G services [124451]. - Some manufacturers have been planning for the transition better than others, with GM and Honda having functionality built into the modems to switch to other networks with an update [124451]. - Ford informed customers that they could upgrade the modem to 4G at their own expense if they wished to continue using smart features, indicating a potential lack of proactive measures by the manufacturer to address the issue [124451].
Dimension (Hardware/Software) hardware, software (a) The software failure incident occurring due to hardware: - The shutdown of AT&T's 3G network is causing connected features in millions of cars to stop functioning, as these features rely on the 3G network which is being phased out [124451]. - Some manufacturers have taken steps to rectify the issue by offering hardware upgrades to replace the 3G modems with more modern connections [124451]. - Subaru drivers with an active Starlink Safety and Security subscription will receive a free hardware upgrade from 3G to 4G modems [124451]. (b) The software failure incident occurring due to software: - Honda confirmed it would fix the issue with an over the air update and had been working on the problem since October [124451]. - Ford mentioned that its vehicles will be relatively unimpacted by the sunsetting of 3G, other than an older version of an app no longer being available [124451]. - Some models will have varying impacts depending on how many smart features relied on an internet connection, indicating a software dependency for these features [124451].
Objective (Malicious/Non-malicious) non-malicious (a) The software failure incident related to the shutdown of AT&T's 3G network for connected cars is categorized as non-malicious. This shutdown was a planned action by AT&T to transition from the older 3G network to the faster and more reliable 5G infrastructure. The shutdown was part of a broader industry trend to phase out older cellular technologies like 3G to make room for newer technologies like 5G [124451]. (b) The shutdown of the 3G network for connected cars was not a result of any malicious intent but rather a strategic decision by AT&T to upgrade its infrastructure. The transition from 3G to 5G was planned well in advance, and efforts were made to communicate with customers and provide solutions such as upgrading to 4G modems. While the shutdown did lead to the loss of certain smart features in older vehicles, it was not a malicious act but rather a necessary step in the evolution of cellular networks [124451].
Intent (Poor/Accidental Decisions) poor_decisions (a) The software failure incident related to the shutdown of AT&T's 3G network for connected cars can be attributed to poor decisions made by manufacturers and service providers. Despite warnings and announcements about the impending shutdown of the 3G network, some vehicle models released as late as 2022 still shipped with a 3G modem, causing these models to be affected by the shutdown [124451]. This indicates a lack of proactive decision-making by manufacturers to transition to newer technology before the 3G services ended. Furthermore, some manufacturers have failed to offer a solution to owners of 3G-connected vehicles, leaving them unable to take advantage of safety features and other beneficial connectivity functions. This lack of foresight and preparation on the part of manufacturers can be seen as poor decision-making contributing to the software failure incident [124451].
Capability (Incompetence/Accidental) development_incompetence, accidental (a) The software failure incident occurring due to development incompetence: The shutdown of the 3G network by AT&T, leading to the loss of connected features in millions of cars, can be attributed to a form of development incompetence. Despite warnings and announcements about the 3G network shutdown for years, some vehicle models released as late as 2022 still shipped with a 3G modem, causing these models to be affected by the network shutdown [124451]. (b) The software failure incident occurring due to accidental factors: The shutdown of the 3G network by AT&T and the subsequent impact on connected features in cars can also be seen as a failure due to accidental factors. While some manufacturers have taken steps to rectify the issue, delays in upgrading to 4G or LTE modems due to global supply chain shortages have left some drivers waiting until the end of the year to regain access to smart features in their cars [124451].
Duration permanent The software failure incident related to the shutdown of AT&T's 3G network for connected cars is considered a permanent failure. This is because the shutdown of the 3G network is a deliberate action taken by AT&T to transition to newer technologies like 5G, and it affects millions of cars that rely on the 3G network for various smart features. The shutdown is not expected to be reversed, and affected vehicles will need to upgrade to 4G or LTE modems to regain connectivity [124451]. Additionally, the impact of the 3G network shutdown on connected cars is significant, as it affects various functionalities such as in-traffic navigation, location data, in-car wifi, emergency callouts, and remote lock and unlock functions. Some manufacturers have already taken steps to rectify the issue, but there are delays in upgrading to 4G modems due to global supply chain shortages. This indicates a long-term impact on the affected vehicles [124451].
Behaviour crash, omission, timing, value, other (a) crash: The software failure incident in this case can be categorized as a crash. The shutdown of AT&T's 3G network led to the loss of connected features in millions of cars, including traffic navigation, emergency calls, and voice assist. These features ceased to work as the 3G network was switched off, indicating a failure of the system to perform its intended functions [124451]. (b) omission: The software failure incident can also be categorized as an omission. The shutdown of the 3G network resulted in the omission of essential functions in millions of cars, such as in-traffic navigation, location data, in-car wifi, emergency callouts, and remote lock and unlock functions. These functions were omitted as they relied on the 3G network, which was no longer available [124451]. (c) timing: The software failure incident can be categorized as a timing issue. Despite warnings and announcements about the shutdown of the 3G network, some vehicle models released in 2019, 2020, 2021, and even 2022 still shipped with a 3G modem. This indicates a timing failure in terms of transitioning to newer technology before the 3G services ended on February 22, 2022 [124451]. (d) value: The software failure incident can be categorized as a value issue. Some vehicle models released in recent years still had 3G modems, which led to the loss of valuable connected features for customers. Manufacturers offered solutions to upgrade to 4G or LTE modems, but delays in the process meant that some drivers could be waiting until the end of the year to regain access to smart features in their cars [124451]. (e) byzantine: The software failure incident does not exhibit characteristics of a byzantine failure. There is no indication in the articles of inconsistent responses or interactions from the system during the shutdown of the 3G network. (f) other: The software failure incident can be categorized as a failure due to the system behaving in a way not described in the options (a to e). This incident can be seen as a consequence of the planned shutdown of an older technology (3G) to make way for newer and more advanced infrastructure (5G). The failure resulted in the loss of essential connected features in millions of cars, impacting users who relied on these functions for navigation, communication, and safety [124451].

IoT System Layer

Layer Option Rationale
Perception None None
Communication None None
Application None None

Other Details

Category Option Rationale
Consequence property, delay, non-human, theoretical_consequence, other (a) death: People lost their lives due to the software failure - There is no mention of any deaths resulting from the software failure incident related to the shutdown of AT&T's 3G network affecting connected features in cars [124451]. (b) harm: People were physically harmed due to the software failure - There is no mention of physical harm to individuals due to the software failure incident [124451]. (c) basic: People's access to food or shelter was impacted because of the software failure - There is no mention of people's access to food or shelter being impacted by the software failure incident [124451]. (d) property: People's material goods, money, or data was impacted due to the software failure - The software failure incident resulted in the loss of connected features in millions of cars, affecting services like in-traffic navigation, location data, in-car wifi, emergency callouts, and remote lock and unlock functions [124451]. (e) delay: People had to postpone an activity due to the software failure - Some drivers may have to wait until the end of the year to regain access to smart features in their cars due to delays in upgrading to 4G modems caused by global supply chain shortages [124451]. (f) non-human: Non-human entities were impacted due to the software failure - Non-human entities such as ATM machines, security systems, and some EV chargers also rely on the older generation cellular signal and will be affected by the shutdown of the 3G network [124451]. (g) no_consequence: There were no real observed consequences of the software failure - The software failure incident had real observed consequences, particularly in terms of the loss of connected features in cars and other devices that relied on the 3G network [124451]. (h) theoretical_consequence: There were potential consequences discussed of the software failure that did not occur - The potential consequences discussed included a security risk due to the inability to install updates in cars without access to Wifi or a physical USB port, as well as the loss of safety features that could help prevent injuries or fatalities in crashes [124451]. (i) other: Was there consequence(s) of the software failure not described in the (a to h) options? What is the other consequence(s)? - The software failure incident led to the discontinuation of 'smart features' in millions of cars, impacting user experience and convenience for drivers who relied on services like in-traffic navigation and emergency callouts [124451].
Domain transportation, utilities (a) The software failure incident reported in the articles is related to the transportation industry. AT&T's shutdown of its 3G network impacted millions of cars, affecting connected features such as traffic navigation, emergency calls, and voice assist in vehicles from various manufacturers [124451]. (g) The software failure incident also had implications for utilities as some ATM machines, security systems, and EV chargers relied on the older 3G network for connectivity, which would be affected by the shutdown [124451]. (m) The software failure incident is also relevant to the automotive industry, as it highlighted the impact of technological transitions on connected car features and the need for manufacturers to address the issue through updates or upgrades [124451].

Sources

Back to List