With the June Virus Bulletin test, ESET became the first antivirus company in the world to pass 50 tests for VB100 awards.
As consumers I think you should know what the VB100 award means. First of all, a VB100 award does not mean that a product detects 100% of all viruses or malware. The VB test only includes self-replicating code (viruses and worms and potentially some bots). Normal Trojans, spyware, and adware are not in the test. The samples used by Virus Bulletin and most certification bodies are the samples on the Wildlist. The Wildlist organization has reporters throughout the world. If two reporters submit samples of the same virus from different locations then it is placed on the Wildlist. We know for a fact that there are lots of viruses out there infecting people that are not on the Wildlist.
One of the purposes of the Wildlist is to create a test set that only includes samples that we know are infecting users and actually do work. One of the problems with many tests is that they use files that should not be detected because they are corrupted and do not run. In many cases we have seen harmless text files in test sets as well.
The Wildlist provides a scientifically sound test set. The Wildlist does not provide a statistically significant test set. That is to say that you cannot tell what anti-virus product is best based upon Wildlist testing alone. Even if we could collect every real threat in the world, I don’t think you could decide what product is best based upon any single test. That would be like picking a stock because it had the biggest gain one day.
Another thing to know about the VB100 is that it is essentially a test that all of the anti-virus companies have the answers to in advance. The AV companies have fundamentally the same set of samples before the test.
In order to obtain a VB100 award a company has to detect all of the samples on the Wildlist and do so with no false positives, both on-demand and on-access.
Most of the critics the VB100 award have obviously never successfully tried to make an antivirus product of even low relative quality. The fact that so many companies fail the tests for missed viruses indicates it actually is a bit more difficult than one might think, especially since the companies have the answers to the test in advance.
50 VB100 awards is a track record and an indication of the dedication of quality that ESET’s developers and researchers have!
There are more difficult tests out there, but you should expect your AV product to be able to deal with the easier tests on a consistent basis, especially since we all know that the samples in the test are really infecting users!
Randy Abrams
Director of Technical Education
ESET LLC
