Every year, organizations around the world spend big bucks on IT products, especially in the cybersecurity space. And generally, customers get what they want. They do some due diligence, find a cybersecurity solution that meets their needs, and are pleased with the results of their purchase.
But then comes the financial burden of legacy antivirus (AV) – the annual renewal of a product they may no longer like, wish they didn’t need, and resent because of the budget dollars they’re spending to maintain it. And all while watching it miss malware on an all-too-regular basis.
If there’s one product that customers love to hate, it’s antivirus, and for good reason.
A Case Study in Failure
There are many reasons why legacy antivirus products are failing us. One predominant reason is their dependency on external (off the endpoint) reliance in detecting malware.
Here’s a recent example. A client of mine discovered a number of machines infected by the virulent Locky ransomware. They isolated the machines off the network and restored them from backups, with only a moderate amount of data loss. But when they analyzed the malware, they saw that it had been detected by their AV product on other machines.
So what was going on? As they looked deeper, they found that the infected machines did not have up-to-date DAT files, despite being on the corporate network. The machines had full network connectivity and seemingly no problems connecting to the antivirus management server.
But they just didn’t have the latest updates and so were unable to detect the Locky malware that was newer than their week-old DAT files. As my client looked elsewhere, they found a number of other desktops that were also out of date.
The client couldn’t diagnose exactly why, but a reboot or two fixed all of the machines with malware. On the positive side, they saw that their up-to-date machines did indeed correctly block the malware, so it wasn’t a ‘detection quality’ issue as we normally see it.
The Painful Reality
This sounds a bit odd, but that is generally the way of the world when it comes to desktop computing. Operating systems and software occasionally crash and do funny things. Now, consider how connectivity issues would exacerbate this issue further, with mobile users coming in and out of connectivity to the AV update server. Patches fail to deploy and assumed security measures disappear in an instant.
This is the real world. This is the world we live in.
DAT file signature updates are one thing, but what about those products that rely on real-time scanning in the cloud? What happens if you take away connectivity there? The reality is they fall back to their local protection, which for some vendors is basic and others is simply non-existent. Malware authors have known this for a long time and regularly exploit this dependency.
It’s not uncommon for malware to change DNS and routing settings on the infected machines to ensure that all AV product connectivity is broken, leaving the malware command and control (C&C) traffic intact.
The power of the cloud is very alluring until someone takes it away.
Dependency on the Mothership is the Problem
The dependency of many AV products on external resources providing continuous updates is deeply flawed and is responsible for most of the malware infections we see today.
We would all like to live in a hyper-connected world of seamless synchronicity and everything working as per design, but that’s just not how it goes. Ask any organization’s Helpdesk Manager and they’ll give you a statistic of around 10% of enterprise machines being about a week out of date on their security updates. That’s like saying that one in every ten windows on your locked house is being left open, and has the same effect on your organization’s security.
Again, this is the real world, and we must test for it.
Clearly, this flies in the face of what most AV vendors want us to do. They want to see ‘ideal conditions’ that show their product in the best light. As a member of the Anti-Malware Testing Standards Organization (AMTSO), I have deep insight into what these AV vendors and the big testing companies recommend.
I encourage you to do the complete opposite. I do a lot of AV testing for our clients who want to see real product performance in true real world scenarios. In the real world scenario that follows, solutions that are not dependent on constant signature updates will surely have the upper hand.
The Holiday Test
One of my favourite tests is The Holiday Test, which pits new malware against an out-of-date offline instance of the AV product. You need to plan this somewhat in advance and create snapshots of old machines frozen in time.
For example, on Day One, create a VMware image of your AV software, make sure it is fully up-to-date, and then snapshot it. Then, two weeks later, disconnect the network connectivity from the VMware image, restore the image, and you now have a two-week old instance ready to face some brand new malware.
A more refined practice would be to firewall off the AV product connectivity, rather than disconnecting the whole network, since this allows for malware C&C traffic to communicate normally. Whether you disconnect the entire network or just AV connectivity depends on what kind of scenario you wish to replicate.
If you want to see how a product performs in the full out-of-date offline scenario, stick with no network at all. Then just run some malware and see how the product performs.
Recently, a client asked me to test Cylance in this way. We had 100 samples of ransomware, all under a day old. We ran these against an Internet-disconnected installation of our endpoint protection product, CylancePROTECT® that had not connected to its cloud management since August 2016 (that’s a version that’s almost six months old).
Impressively, ALL of the malware was detected and blocked. It’s a simple test to perform and very compelling as a demonstration of the efficacy of CylancePROTECT’s underlying local engine.
At this point, depending on who you are, you’ll have one of two reactions. End-users may say, “That’s interesting, I’ll try that.” Legacy AV vendors, and a few testers, will say, “But that’s artificial! It’s not real world!” and advise against it. We’ll leave you to decide for yourself.
Final Thoughts
Personally, I’m fairly aggressive when it comes to AV testing. Push the products to their limits, don’t trust anyone implicitly, and find the weak spots that many AV vendors don’t want you to know about.
The concept of ‘the real world’ is an ambiguous one, so it’s up to you to decide what your real world is, and to make sure you test against it. The good news is that AV has moved on and newer solutions no longer rely on the flawed architecture of continuous updates.
Whatever anyone may say, no product is perfect, but if you test for your environment, you will be able to find a worthy AV product.
Check out some really impressive new testing methodologies in this report: AV-TEST Results.
Carl Gottlieb,
Consulting Director of Cognition, AMTSO member and Antivirus Consulting Provider.
