Imperva anti-virus study "flawed", claims IT security expert

Methodology used to test anti-virus products in study is not true to life , claims Trend Micro's Rik Ferguson.

Security

Imperva has been forced to defend the findings of one of its recent security reports, following accusations the methodology used to create it was flawed.

The security vendor released its "Assessing the Effectiveness of Anti-virus Solutions" report last week, which pitted 80 previously non-catalogued viruses against more than 40 anti-virus products.

In the report, the company claimed that less than five per cent of these products were able to detect new viruses and that some took up to a month or more to update their signatures.

In the run up to the report's release, Tal Be'ery, web research team leader at Imperva, told IT Pro the findings were not designed to put people off adopting anti-virus.

They were not exposing the products to threats in the way they would be in the wild.

"[The research shows] malware has windows of opportunities and it shows there are a few weeks before it gets detected by the anti-virus products," he said.

"[Even so] you should definitely have anti-virus to protect against most malwares and [because] after a time it recognises and can protect against them."

The company used the website, VirusTotal, to analyse the samples before each one was tested by the anti-virus products.

This resulted in the creation of a report, which revealed whether or not the sample was picked up by the anti-virus product.

However, the study's findings have been criticised by one security expert for not exposing the products to viruses in the same way they would be "in the wild".

Speaking to IT Pro Rik Ferguson, director of security research and communications at rival security vendor Trend Micro, described the study as flawed.

"Simply scanning a collection of files, no matter how large or how well sourced misses the point of security software entirely," he said.

"They were not exposing the products to threats in the way they would be in the wild."

For instance, where was the email with the malware attached, he asked, or - if the threat was URL-based where was the analysis of its content?

"To decide whether or not a threat would be blocked, it must be processed in a test in the same way it would be delivered to the victim," he added

In a follow-up statement to IT Pro, Be'ery defended the research, claiming the evolving nature of security threats mean Ferguson's recommendations may not work for every testing scenario.

"[The recommendations] address an old threat model in which the attacker would try to infect many possible targets with a single campaign," he said.

"When the old threat model is considered...and all the defences are tested, the same conclusion holds: while anti-virus is effective in fighting widespread malware, for new [threats], there is a good chance it will evade the anti-virus solutions."

Featured Resources

Next-generation time series: Forecasting for the real world, not the ideal world

Solve time series problems with AI

Free download

The future of productivity

Driving your business forward with Microsoft Office 365

Free download

How to plan for endpoint security against ever-evolving cyber threats

Safeguard your devices, data, and reputation

Free download

A quantitative comparison of UPS monitoring and servicing approaches across edge environments

Effective UPS fleet management

Free download

Recommended

What is a botnet?
botnets

What is a botnet?

14 Jul 2021
Trend Micro home network security flaws could let hackers take over PCs
Security

Trend Micro home network security flaws could let hackers take over PCs

26 May 2021
The secure cloud configuration imperative
Whitepaper

The secure cloud configuration imperative

26 May 2021
Trend Micro and Snyk team up to combat open source flaws
vulnerability

Trend Micro and Snyk team up to combat open source flaws

10 May 2021

Most Popular

UK spy agencies supercharge espionage efforts with AWS data deal
cloud computing

UK spy agencies supercharge espionage efforts with AWS data deal

26 Oct 2021
Best Linux distros 2021
operating systems

Best Linux distros 2021

11 Oct 2021
Cryptocurrency: Should you invest?
cryptocurrencies

Cryptocurrency: Should you invest?

27 Oct 2021