Detecting Cyber Anomalies: Two Ways the BlackBerry Threat Research Team Uncovers Attacks
Instead of mining the earth for diamonds, BlackBerry Senior Threat Researcher Jacob Faires mines data to detect cyberattacks. Both types of mining share similarities: They require specialized tools, persistence, and an eye for an often “small” detail that stands out.
In Faires’ case, it can mean hunting through a billion rows of data to find a cyberattack he must then analyze.
“We go find the threat actors that are being ‘quiet,’ that are silently infiltrating client machines,” Faires explains. “We find those incidents, and then build detections around them and make those types of attacks public, so organizations will know what to look for. We can do this because we’re a leading-edge company in finding and detecting those things.”
Like finding a diamond in the rough, if it was easy, anybody could do it. So how, exactly, does BlackBerry’s world-class threat research team accomplish this difficult task?
Detecting Cyber Threats and Attacks
Faires and his teammates employ highly specialized tools and methods to detect cyberattacks, update the BlackBerry® technology to defend against them, and finally, create and disseminate contextualized cyber threat intelligence (CTI) reports — so the world can learn about the latest tactics, tools, and procedures (TTPs) that threat actors are using.
Accomplishing these things starts with analyzing a seemingly impossible amount of data.
“With a single client over a three-month period, we'll have close to a billion rows in the database. And whenever you're looking across a billion rows, you're going to have to use some large clusters to deal with all that data, right? So even if you're just trying to filter it down to very, very simple things, you must be able to load all that data, process it, and figure out unique things on those command lines that are being run.”
He analyzes this kind of unusually rich data — and can track the lifecycle of processes — by threat hunting within CylanceOPTICS®, the BlackBerry cloud-enabled endpoint detection and response (EDR) tool that protects customers. “You can filter down to just specific devices, over a timeframe, and then be able to contextualize what the attack looked like. And then you can build out that entire attack chain. Compare this to normal detection where you're only going to get a file and you say, ‘Well, this one was detected and we know that there was something we need to be looking for on their system, so now let's go talk to them.’ With CylanceOPTICS, you can say, ‘Here is the entire timeline of the attack. And here is how we can prevent it in the future.’”
(Note: CylanceOPTICS is included in CylanceENDPOINT™ Pro and CylanceENDPOINT™ Advanced).
Managing Massive Amounts of Data for Threat Detection
If you have the rich data you need for threat hunting, then how can you manage it and mine it to spot what really matters? One of the strengths of the BlackBerry Threat Research and Intelligence team is its unusually strong internal collaboration process. Team members closely partner with the BlackBerry data architecture team to maximize results.
To initially implement the approach, the team required a unified data platform that could seamlessly integrate with BlackBerry’s existing data lake (the centralized repository where data is stored) and provide high-performance and low-latency read/write capabilities. This is important because every second matters when detecting a cyberattack, enriching data, and responding to the attack.
The team also required comprehensive tooling to enable collaboration among our data analysts, scientists, and threat researchers, for both batch and streaming data. Streaming data enables real-time decision-making and the ability to respond to changing conditions, while batch data allows us to run complex queries on vast amounts of historical data.
Data warehousing increases the speed and flexibility to meet these demands, so our teams evaluated multiple cloud data warehouse vendors. BlackBerry chose the Databricks Lakehouse Platform by Databricks for its ability to meet the requirements and simplify data architecture at the same time.
Previously, our threat researchers had to navigate a dispersed system where you needed to hit different application programming interfaces (APIs) and different databases and then join all that information together somewhere unique. You'd have to create a whole new system just to put everything together.
“That system required a fully dedicated team to support and maintain,” says BlackBerry Distinguished Data Architect Justin Lai. “Now with Databricks, it’s all centralized. Our threat researchers can easily query and make use of that data.”
Also, when you are dealing with massive amounts of customer data for cybersecurity threat detection, it is imperative only the right people have access to the data and that they have an end-to-end view of where data is coming from and how it is being consumed, for regulatory compliance. This makes it easy to govern our data lake.
“It allowed us to create a unified view of our data estate, simplifying collaboration across teams. We now have a standard approach to manage access permissions and audit files or tables in our lake, with the ability to define fine-grained access controls on rows and columns,” says Lai. “Automated data lineage helped us see where the data is coming from to pinpoint the source of a potential threat and to understand which research projects or teams are leveraging the data for threat detection.”
Using this approach, the teams also broke down data silos and iteratively improved their EDR pipeline, which is the cloud infrastructure involved in the process of extracting, loading, and transforming collected data. The team was able to ingest data faster and reduce querying latency by more than 20% — speed that is passed on to BlackBerry customers in the form of rapid threat detection.
This approach to managing data is just one of the ways the BlackBerry data architecture and threat research teams are leveraging Cylance® data telemetry from our award-winning artificial intelligence (AI)-driven products.
In the end, BlackBerry protects customers, contextualizes cyber threat intelligence, and feeds back into public reports, such as the quarterly Global Threat Intelligence Report.
YARA Rules to Identify Related Cyberattack Campaigns
A second approach the BlackBerry research team uses to mine Cylance and open-source intelligence (OSINT) data for cyberattack signals, involves YARA rules, which threat researchers apply across large datasets. These rules set up malware detection patterns for identifying malware “families” that share common code, but are not necessarily identical.
The general approach is to have at least two different sets of YARA. An example might be that one dataset is for hunting. This means a researcher may write "dirty YARA rules" where they accept a certain level of false positives, because the purpose is to find files, and those objects on disk, which are suspicious to them. The researcher assumes they're malicious or that they are potentially malicious based on pre-determined criteria including things like timestamp language, hash, obfuscation, and more.
At the same time, BlackBerry threat researchers utilize a set of “curated YARA rules.” These may not necessarily be for hunting but are instead intended for regular scans and detections of malicious campaigns by known threat actors. The goal in this case is to keep the system detecting samples by the same threat actor. Our researchers test these results which allows them to confirm that they have zero false positives. The results of this process are often good enough to help the threat researcher detect additional malicious samples. The BlackBerry Threat Research and Intelligence team explains this is one example of what they accomplish through YARA.
For a deeper dive into YARA rules, watch our live broadcast on the topic.
“Whenever we find one campaign, we can then easily build rules and search for other instances of those same types of attacks inside our datasets,” says Faires.
“While hunting, we also create our own rules whenever we find things that are unique, often within our CylanceOPTICS platform. And that is because we have much more information that we can analyze — more than file data and a file path, for example.”
You can find YARA rule examples in many of our threat research blogs, including our recent blog on NOBELIUM Using Poland Ambassador’s U.S. Visit to Target EU Governments Assisting Ukraine.
Conclusion
This is a small glimpse behind the scenes focusing on just two examples of how the BlackBerry Threat Research and Intelligence team leverages its people, processes, and technology to help protect the world’s endpoints. BlackBerry clients include everything from small-to-medium sized businesses (SMBs) to 17 of the globe’s G20 governments.
The results of this work are clear. “We're already releasing notifications of leading-edge research about new cyber threat actor campaigns that they are taking on. Then we're introducing those detections back into our platform.”
And for Senior Threat Researcher Jacob Faires, that’s a positive feedback loop that motivates him to continually mine data and discover new cyberattacks.
