Facebook has revealed the cause of one of the worst service outages ever to hit the social networking site.
The downtime occurred on Monday, October 4, and affected billions of users globally for around six hours. It also knocked out other Facebook-owned services such as Messenger, Instagram, and WhatsApp.
During the outage, the company took to Twitter to acknowledge the situation and apologize for the inconvenience. But it took until Monday evening — by which time the service had been mostly restored — for the company to offer its global community a reason for the mishap.
Writing in a blog post, Facebook apparently ruled out a hack or some such nefarious act, claiming instead that the outage was caused by configuration changes to its routers that prevented its computer systems from communicating with each other.
“Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication,” Facebook said in its blog post, adding: “This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt.”
Declining to go into further detail, it reiterated that “at this time we believe the root cause of this outage was a faulty configuration change,” adding that it has “no evidence that user data was compromised as a result of this downtime.”
Facebook confirmed that its systems are now up and running again, but said the problem took longer than expected to fix because the underlying cause “also impacted many of the internal tools and systems we use in our day-to-day operations, complicating our attempts to quickly diagnose and resolve the problem.”
It added: “We apologize to all those affected, and we’re working to understand more about what happened today so we can continue to make our infrastructure more resilient.”
If you’re keen for a deeper dive into what went wrong, check out Cloudflare’s super-detailed explanation.
The calamity wiped nearly $50 billion off the company’s market value and is also thought to have cost Facebook around $79 million in ad revenue.
During the downtime, many folks hit Twitter to make fun of the situation. Even Twitter itself joined in, tweeting, “Hello literally everyone,” as more and more people flocked to the microblogging service to find out why they couldn’t access Facebook, Instagram, and WhatsApp.
