Nine Ball: Juggling with VirusTotal

There’s been some media interest in an alert from WebSense about something they call Nine Ball (he, said, trying to keep his sense of humour in check). It has some pretty interesting characteristics. I’d like to pick up, though, one point that the reports I’ve seen have rather overstated.

WebSense mentioned that vendor detection is low on a Trojan Loader and a malicious PDF. This is true, or was at one point in time, in the sense that a PDF sample submitted to VirusTotal resulted in a report indicating that only three vendors identified it as malicious. Well, actually, even that isn’t quite accurate: two of those hits seem to be a generic packer/javascript detections rather than identification of the file as malicious in its own right. (Similarly, most of the Trojan Downloader detections are generic, and one simply says "suspicious".)

This industry is divided on whether detection purely on packer signature is a good idea. Some vendors flag almost all packed malware as malicious, packed or suspicious: this is because malware distributors use packers, obfuscators and protectors to make it more difficult for security software to recognize code that would otherwise be identified as known malicious code.

The problem is that a fair number of developers use the same tools (in some cases tools specifically developed for malicious purposes) to protect legitimate applications from disassembly and so on, as a Digital Rights Management (DRM) strategy. Well, that’s what they tell us. For this reason, some vendors don’t automatically detect packed apps as malicious, for the benefit of those to whom avoiding false positives may be as important as high detection rates. (Strangely enough, some organizations find FPs a very signific;ant problem. Of course, all companies do if an innocent and widely used file such as a Windows system file is misidentified as malicious.)

Note, however, that high detection rates and low false positive rates aren’t mutually incompatible. Products that don’t generally detect packers as automatically malicious (we don’t, with some exceptions) may well detect packed code anyway, using custom unpackers and other techniques such as emulation. So where’s the problem with VirusTotal? Well actually, the problem isn’t with VirusTotal (or rather with Hispasec, who provide the free VirusTotal service), but with the way that it’s used.

As we’ve mentioned here before, VirusTotal is not intended to provide some kind of measure of vendor performance, though it’s often used (or misused) that way. One particularly annoying example is provided by SRI International, who do fine work in other areas, but persist in ranking vendor performance based on VirusTotal reports. I'll come back to that another time. Slightly less irritating is the habit some security researchers have of measuring the effectiveness of vendor response to a new threat using VT reports, as long as they don't mislead themselves and others by reading too much into the reports.

If a number of vendors report something as malicious (or at least suspicious), that's a good indicator that it's a problem. But if a vendor doesn't record a hit on a specific file, that doesn't mean it can't detect the malware. It may mean that the malware is, because of obfuscation and other factors, only detected using some form of behavioural analysis that may not be deployed (or deployable) in on-demand scanning mode. As long as it detects on-access, though, this isn't automatically a bad thing, and it's a definite plus if it does so more accurately than a generic detection such as a "suspicious/packer" detection.

Of course, knowing that a product doesn't detect something on-demand is sometimes useful. I'm simply pointing out here that it isn't the same as a binary "detects/doesn't detect" indicator. In this particular instance, it's actually quite easy to detect a malicious PDF generically, but harder to detect it accurately (in other words, without false positives).

What does this mean in practice? Essentially, that you shouldn't leap to conclusions about a product's capabilities based on a VirusTotal report. Of course, you shouldn't, in any case, rely on anti-malware detection alone. In this case, the malicious code is only part of the problem, since it relies on one of a number of exploits to get a foothold on the system it attacks.