If you’re wondering whether the recent global IT outage that brought down millions of Windows PCs due to an improper software update from cybersecurity company CrowdStrike was an isolated case, we can assure you that it is not.
In fact, something similar happened earlier this year. A government investigation into February’s nationwide AT&T outage revealed that the cause was improper network updates, Ars Technica reported on Tuesday.
According to an FCC report, the outage “affected customers in all 50 states” and “all voice and 5G data services are unavailable to AT&T Wireless customers, affecting more than 125 million devices and blocking more than 92 million voice calls and blocked more than 25,000 more calls to the 911 call center. The FCC also noted that it “took AT&T at least 12 hours to fully restore service.”
Mix and match speed of light
We have reported the outage and noted that it is causing disruption to users on other unaffected networks as they are unable to call AT&T customers. AT&T offered its customers a $5 credit as an apology.
How did a CrowdStrike bug bring the world to a halt? We asked three experts.
The incident began “after AT&T implemented network changes due to equipment misconfiguration,” the report said. But it wasn’t just this isolated issue that made this outage so severe.
The FCC’s Bureau of Public Safety and Homeland Security analyzed the incident and found that the outage “was the result of multiple factors, all of which were attributed to AT&T Mobility, including configuration errors, non-compliance with AT&T Mobility’s internal procedures, and lack of peer support.” The review cited failure to conduct adequate post-installation testing, insufficient laboratory testing, inadequate safeguards and controls to ensure approval of changes affecting the core network, a lack of controls to mitigate post-outage impacts, and various long-standing system issues once When the configuration error is corrected, an outage occurs” (per Ars Technica).
Four days later, Delta Air Lines is still reeling from the CrowdStrike outage. Here’s why.
AT&T’s story may not be over yet, and the company could face hefty fines. But it’s yet another reminder that the global IT networks we rely on are often more vulnerable than we think, and that in many cases security procedures for critical systems need to be carefully studied.
