December's Virus Bulletin includes a comparative test for a number of products on the Windows Vista x64 platform, giving us our 53rd VB100 award. To get a VB100, a product needs to detect all "In the Wild" viruses on-demand and on-access, with no false positives. Note that "In the Wild" here refers to replicative malware listed on the current WildList: obviously, it would be unrealistic to expect anyone to manage 100% of all malware (replicative and non-replicative) that is currently out somewhere in the wild and woolly internet, even if it were possible to utilize it as a test set. Unless, of course, for whitelisting, which apparently not only catches all past, present and future threats, but also leaps buildings at a single bound and is about to cure the economic recession, world hunger, and the common cold. ;-)
Let's be serious about this for a moment. The present VB100 test is by no means valueless (we like it a lot, but I suppose we would, having scored more VB100s than anyone else...), but it's limited in scope by using WildCore, the sample set based on the WildList, to viruses, which are a pretty small part of the current malware population. What's more, it tends to lag behind the curve by a month or more, Why are so many detection certifications still based on it? Well, this is a complex issue, but I think John Hawes put it pretty well in the test report: "The purpose of the scheme is to provide certification of products proven to be legitimate, and to provide a basic level of protection." While a larger, more current, more diverse sample set would map more closely to the whole malware population, it would also introduce a far wider margin for error. In some ways, and with all its faults, ItW (In the Wild) testing is the nearest thing to a level playing field. I think I feel a paper coming on... Incidentally, we recently added a VB reprint to our white papers pag, covering our VB100s from June to October 2008.
Meanwhile, I'm feeling a bit chuffed that Virus Bulletin's December newsletter included a link to their book resources page, since it turns out to include one book I edited, one to which I contributed, and three of my book reviews (which are also available on our white papers page. :-)
David Harley
Director of Malware Intelligence
