New Independent Tests of Endpoint Protection Reveal Marked Differences in Performance, Efficacy
Lab Tests of Endpoint Protection Platforms
BlackBerry commissioned The Tolly Group — a leading independent test lab and provider of third-party validation services for the tech industry — to evaluate several endpoint protection platforms. Specifically, Tolly Group tested CylanceENDPOINT™ EPP and several competing solutions.
The testing looked at the following:
- Detection rates for malware, based on two collections of 1,000 recent VirusTotal samples
- CPU (central processing unit) utilization during the scanning of the samples
- Total time taken to scan and convict the malicious samples
The analysis occurred in a fully updated Windows 10 environment. The testing was conducted both online, where the endpoints were cloud-connected, and in an offline scenario where the internet connections of all the endpoints were disabled. This was done to force the endpoints to rely only on local information when examining the malware.
The results of these tests are extremely eye-opening.
Independent EPP Testing Results
Tolly tested CylanceENDPOINT as well as Microsoft Defender for Business, Sophos Intercept X Advanced with XDR, and Trellix Endpoint Protection Platform.
We are pleased to announce that the Tolly report shows CylanceENDPOINT achieving a market-leading 98.9% detection rate for both online and offline tests. Competitors’ solutions performed significantly worse, allowing 9x more malware at their best, and a whopping 52x more malware at their worst.
At the same time, competing solutions required full (100%) CPU utilization to complete the necessary steps, while CylanceENDPOINT utilized just 5% of the CPU. In addition, the Microsoft, Sophos, and Trellix solutions took significantly longer to complete the scanning and blocking processes: One of the solutions took 6 hours to do what CylanceENDPOINT completed in only 27 minutes.
“Cloud-Enabled” Becoming “Cloud-Required” Invites Risk
Another key finding in the research reveals an Achilles’ heel exists for many endpoint protection platforms — if their cloud connection is disrupted, they become much less effective.
Over the past five years, the market has shifted towards an “always-on” approach, where cloud computing is seen as a requirement for effective cybersecurity. However, this over-reliance on cloud computing and cloud storage exposes organizations to unnecessary risk.
The Tolly analysis reveals that disabling the network connection — or limiting cloud access for the endpoint — significantly reduces the efficacy of some “cloud-native” (read “cloud-required”) EPPs. In effect, this inbuilt cloud reliance provides a convenient “kill switch” for attackers: Shut off the connection, and the device becomes substantially more vulnerable to exploitation.
CylanceENDPOINT was the lone exception to this cloud-dependent limitation during testing. It detected 98.9% of malware when connected to the cloud — and the same percentage when offline.
At the same time, the research found Microsoft Defender for Business detected just 81% of the samples in offline testing. Trellix Endpoint Protection Platform detected only 64% of the samples while offline. Finally, Sophos Intercept X Advanced with XDR detected a meager 42.4% of the samples when disconnected from the cloud.
Unlike its cloud-required competitors, CylanceENDPOINT has no built-in kill switch, making it consistently effective against modern malware even in isolated or non-connected environments.
The fact that CylanceENDPOINT performs at the top of the field regardless of network connection also proves it is uniquely positioned to protect OT (operational technology), including air-gapped and highly secure environments.
Lightweight Solutions Yield Better Total Cost of Ownership
Now let’s add context to the test results that show CylanceENDPOINT utilizing 5% of CPU in the tests, and competing solutions utilizing 100% of CPU. Imagine the impact on resources, especially if you are protecting a significant number of endpoints.
The shift toward cloud computing for endpoint security tools has paradoxically resulted in significantly higher CPU utilization for many security solutions. Despite having the benefits of cloud connectivity, theoretically offloading computing resources from the local device, these cloud-dependent solutions remain burdensome for devices. This increased usage also drives up the total cost of ownership (TCO) over time for IT and security teams. Additionally, high CPU utilization can negatively impact end users, leading to slower load times and sluggish performance.
In contrast, CylanceENDPOINT provides a better end-user experience by consuming minimal system resources while still delivering best-in-class protection. Its lightweight design means it is imperceptible to end users, regardless of connection status, while its efficacy in protecting endpoints remains uncompromised. Furthermore, using less CPU can yield a longer lifetime for devices by limiting the workload for the system.
Learn More About EPP Testing and Analysis
The Tolly report on endpoint protection platform efficacy and resource demands is a crucial tool for organizations considering deploying an EPP for the first time, or those looking for a more lightweight and effective solution than they have presently.