Consumer Reports recently came under heavy fire from some in the anti-virus industry for creating some 5,500 new virus variants to see how well a dozen leading products fared in detecting the new nasties. More than 100 security experts and executives from companies like Microsoft and HP as well as anti-virus vendors F-Secure, Kaspersky, McAfee, Sophos, Symantec and Trend Micro signed their names to a declaration denouncing Consumer Reports' methods, stating that it is "not necessary and ... not useful to write computer viruses to learn how to protect against them."
Some of the signatories noted -- via various media reports about the scandal -- that with so many viruses already in circulation today (estimates vary from 100,000 to 180,000) it was hardly necessary for Consumer Reports to gin up new ones that could, in theory, be leaked into the wild.
Today, however, I read a rather thoughtful article written by Juergen Schmidt, an editor with the German technology magazine Heise Security. Schmidt picks apart what he sees as the source of the industry's angst on this. He argues that testing anti-virus products against known viruses is a non-starter because the real battle against malicious worms and viruses these days is against previously unknown threats, of which he says about 250 emerge each day.
From the article: "The commandment 'Thou shalt not create new viruses' is a sensible self-imposed commitment by the manufacturers of anti-virus software, which prevents them from creating an atmosphere of threat to promote their products. In contrast, meaningful comparative testing of anti-virus software requires that testers work with self-generated virus variants. Anyone condemning such tests in general is certainly not doing so in the interests of the user."
Schmidt says that in light of the poor job most anti-virus programs do at spotting new threats (without the benefit of code snippets), it is clearly necessary to test anti-virus software using previously unseen malware.
"Known viruses no longer represent any great danger for users with anti-virus software -- pretty much every product will recognize them reliably. The real danger lies with the estimated 250 new malware programs that are released every day. And recognizing these as a threat is where many anti-virus products still fail miserably."
As I have noted here before, many malware authors are increasingly outpacing the security vendors by automagically updating the genetic makeup of their creations before anti-virus companies have time to ship updates. As a result, we have an industry whose business is predicated on 10 percent to 20 percent of its customers being successfully attacked before it can even begin to respond, according to some estimates.
If you'd care to see a slick, Web-based method some criminals use to evade fresh anti-virus signatures, check out this story I wrote from a few months back about a Russian hacking ring.
