Blue Screens, BlackBerry AtHoc, and Business Continuity: A Global Outage Story
In the wake of the recent CrowdStrike triggered global IT outage, there were two distinct types of organizational responses. Some organizations were caught scrambling to communicate after typical channels went down. Uncertainty isolated outposts, branches, employees and customers.
However, other organizations were prepared for outage contingencies and mounted a rapid out-of-band and coordinated response to the crisis by utilizing the BlackBerry® AtHoc® critical event management (CEM) platform.
Our customers worldwide made this unfortunate situation a success story for business continuity.
The Blue Screen of Death Spreads
Most organizations use out-of-band communication tools to address natural disasters. However, cyberattacks increasingly block traditional communications channels, and as we've recently seen, so do unplanned IT outages.
In this case, the infamous "blue screen of death" spread rapidly during July 2024, forcing companies worldwide to cancel what they do best: flights, surgeries, meal deliveries, and even merchandise sales in their big-box stores. At the same time, BlackBerry AtHoc use surged.
We monitored the performance of the system as our customers worldwide collectively sent millions of messages to their teams. It was a large-scale test of our BlackBerry AtHoc systems, which we built to handle the kind of traffic surge a September 11–level event could generate. In this case, we can report that our systems remained reliable, fast, and secure.
"Please Reply Now with Your Status"
What kind of messages did BlackBerry AtHoc customers send? Some of the messages alerted employees to what was going on. For example:
“A global CrowdStrike software update has caused a widespread Windows system outage, affecting users worldwide. If you encounter a blue screen error on your device, please try restarting your computer. If the issue persists after the reboot, contact the Service Desk at [Service Desk PlaceHolder] for further assistance.”
Think about the power of a message like this. It's along the lines of "Hey, here's what we're experiencing. Here's what we're doing about it. Only call the help desk after you've tried this suggested fix."
This means our customers were executing their communication plans, which is excellent, and it's a sign of good organizations that invest in business continuity and put plans into place.
Other messages our customers sent took advantage of the simple two-way communications BlackBerry AtHoc enables. Here's an example:
Please reply
I am online and fully operational.
I am online but with limited functionality.
I am offline due to an outage.
I am on vacation/out of the office.
One organization received nearly 15,000 employee responses within ten minutes. This information provides strong decision support for the business continuity team, and leaders, within impacted organizations. BlackBerry AtHoc makes it incredibly easy to communicate with today's distributed workforce. Some customers used custom-created message formats, while most utilized our large number of templates for ease and expediency.
Watch the brief video below where I explain more about how our customers utilized BlackBerry AtHoc during the global tech outage.
Truly Actionable Situational Intelligence
While IT organizations can run an audit to see who's offline, a question lingers: Is someone offline because they're on vacation or out sick, or are they offline because they have a blue screen error and are suffering from this problem?
Not being online in this decentralized, remote work-from-anywhere era doesn't necessarily correlate to the problem at hand, and that's why IT and business continuity leaders used this system to survey their folks.
Also worth mentioning is that we saw several customers use geo-targeting to reach employees in specific regions or work locations with the unique information they needed. Others sent varying types of messages based on employee job profiles or seniority.
How AtHoc Delivers Messages During an Outage
How were these messages sent when millions of laptops and PCs were down? Our BlackBerry AtHoc customers sent notices and surveys through multiple channels at one time, including mobile app, work phone, work mobile, work email, text message, and desktop. Other options include the ability to send messages through social media channels, monitors and electronic signs, and even loudspeakers.
Many highly regulated organizations rely on BlackBerry AtHoc because we maintain stringent global certifications. Our continuous monitoring, failover testing, and compliance ensure communications are secure.
Hats Off to Business Continuity Leaders
The impact of this global IT outage was massive. However, if it hadn't been for the prominence of business continuity leaders planning for the full spectrum of threats like this type of event, then the pain would have been much more profound.
We often talk about emergencies being catastrophic, like severe weather events, a gunman on-site, and civil unrest. Those are impactful, but an event like this can be equally so.
Planning for operational outages and impacts kept things from worsening, as did the communication plans built on the BlackBerry AtHoc platform. We are proud of our customers for helping their organizations during this global event.
Please reach out for a demonstration of BlackBerry AtHoc if you'd like to be better prepared for the next unplanned emergency, regardless of its root cause.
