Cylance® recently engaged Carbonview Research to conduct a national survey of cybersecurity professionals across various industries and company sizes. The aim was to understand exactly what has led to past breaches, and whether these breaches have affected IT security professionals’ ability to prioritize, evaluate, and select new security solutions.
This blog details the results ascertained around industry testing and reports.
The Test
In total, 201 IT decision makers completed the survey. Those that completed the survey had at least 5 years’ experience at a company with 300+ employees, and had experience with endpoint anti-malware products. The qualifying question to take the survey was that they had to describe the difference between file-less malware and an exploit.
The survey results are located here.
In this survey, 48% of respondents read AV-TEST industry testing reports. This was a significantly higher result than found in other organizations. This was consistent across all target segments in the study.
Not only are the survey respondents familiar with AV-TEST, they also trust it most among all organizations in the survey. MRG Effitas was found to have the lowest trust level. Applicability to real world scenarios and sample/testing prevalence are shown to be the main drivers for trusting a testing organization.
There were very few differences revealed among organizations surveyed. Those that distrusted a testing organization noted that this distrust was due to the testing methods, which respondents felt were not applicable to real world scenarios, and the fact that the malware samples used were not prevalent. However, the malware samples that are more well-known appear to have better applicability to the real world.
Improving the Test
Regarding the improvement of industry testing reports, across the competitive set, about a quarter of respondents felt that improvements were possible. Including more products and more test cases, and making the test more applicable to real-world scenarios are main areas cited for future improvement of the test.
When querying the respondents on replicating real-world test scenarios, most of the testing organizations were thought to do a good job replicating real world scenarios. Currently, AV-TEST tops the list, while MRG Effitas has the lowest level of replication.
A wish for improvement in real-world efficacy is cited by the majority of respondents (62%), who think that the reports/tests mirror the real world “a little” in terms of what they see for detection efficacy.
83% of respondents agree most with the statement, “I would rather have the vendor(s) provide a proof-of-concept within my environment.”
Why You Should Test For Yourself
We also asked if respondents test for themselves, as we have been advocating here at Cylance for many years now. Given that 62% think that the reports/tests mirror the real world “a little” in terms of what they personally see for detection efficacy, and that testing methods were generally not applicable to real world scenarios where the malware samples were not prevalent, we saw that almost two-thirds of respondents (62%) conduct their own endpoint anti-malware testing. This is encouraging!
For those who don’t test for themselves, the majority says they would like to do so in the future. Across all the respondents, however, “not enough resources” and “a lack of in-house expertise” are seen as the biggest barriers to conducting their own testing. The biggest cited pitfalls in conducting their own testing were “not having enough expertise to conduct the QA,” and “lacking breadth to cover all scenarios.”
These test results should be a powerful motivator to IT decision makers and leaders to engage more fully with freely available online resources that make it possible for anyone, regardless of skill or experience, to test for themselves.
Cylance’s goal is to empower IT professionals with the tools, knowledge and malware to test any vendor’s product in a safe and secure manner, in a real environment, for the future betterment of our security community as a whole.
To get started, visit our testing site at https://www.cylance.com/knowthetruth
INFOGRAPHIC: IT Decision Makers Report on Testing
