Imperva anti-virus study "flawed", claims IT security expert

Methodology used to test anti-virus products in study is not true to life , claims Trend Micro's Rik Ferguson.

Security

Imperva has been forced to defend the findings of one of its recent security reports, following accusations the methodology used to create it was flawed.

The security vendor released its "Assessing the Effectiveness of Anti-virus Solutions" report last week, which pitted 80 previously non-catalogued viruses against more than 40 anti-virus products.

In the report, the company claimed that less than five per cent of these products were able to detect new viruses and that some took up to a month or more to update their signatures.

In the run up to the report's release, Tal Be'ery, web research team leader at Imperva, told IT Pro the findings were not designed to put people off adopting anti-virus.

They were not exposing the products to threats in the way they would be in the wild.

"[The research shows] malware has windows of opportunities and it shows there are a few weeks before it gets detected by the anti-virus products," he said.

"[Even so] you should definitely have anti-virus to protect against most malwares and [because] after a time it recognises and can protect against them."

The company used the website, VirusTotal, to analyse the samples before each one was tested by the anti-virus products.

This resulted in the creation of a report, which revealed whether or not the sample was picked up by the anti-virus product.

However, the study's findings have been criticised by one security expert for not exposing the products to viruses in the same way they would be "in the wild".

Speaking to IT Pro Rik Ferguson, director of security research and communications at rival security vendor Trend Micro, described the study as flawed.

"Simply scanning a collection of files, no matter how large or how well sourced misses the point of security software entirely," he said.

"They were not exposing the products to threats in the way they would be in the wild."

For instance, where was the email with the malware attached, he asked, or - if the threat was URL-based where was the analysis of its content?

"To decide whether or not a threat would be blocked, it must be processed in a test in the same way it would be delivered to the victim," he added

In a follow-up statement to IT Pro, Be'ery defended the research, claiming the evolving nature of security threats mean Ferguson's recommendations may not work for every testing scenario.

"[The recommendations] address an old threat model in which the attacker would try to infect many possible targets with a single campaign," he said.

"When the old threat model is considered...and all the defences are tested, the same conclusion holds: while anti-virus is effective in fighting widespread malware, for new [threats], there is a good chance it will evade the anti-virus solutions."

Featured Resources

Navigating the new normal: A fast guide to remote working

A smooth transition will support operations for years to come

Download now

Leading the data race

The trends driving the future of data science

Download now

How to create 1:1 customer experiences at scale

Meet the technology capable of delivering the personalisation your customers crave

Download now

How to achieve daily SAP releases

Accelerate the pace of SAP change to support your digital strategy

Download now

Recommended

Your essential guide to internet security
Security

Your essential guide to internet security

23 Sep 2020
How to enable private browsing on any device
privacy

How to enable private browsing on any device

22 Sep 2020
Third-party apps are tracking your WhatsApp activity
social media

Third-party apps are tracking your WhatsApp activity

21 Sep 2020
Ransomwiz lets you test your security with simulated ransomware
ransomware

Ransomwiz lets you test your security with simulated ransomware

21 Sep 2020

Most Popular

16 ways to speed up your laptop
Laptops

16 ways to speed up your laptop

16 Sep 2020
16 ways to speed up your laptop
Laptops

16 ways to speed up your laptop

16 Sep 2020
Windows Server flaw sparks emergency US gov warning
vulnerability

Windows Server flaw sparks emergency US gov warning

21 Sep 2020