The Internet Security Suites Blog

Monday, August 21, 2006

Consumer Reports Slammed for Creating 5.000 "test" viruses

As noted on this blog a few days ago, Consumer Reports has published the results of its own antivirus software testing in its last issue. It has some people in the industry a bit confused.

To put 12 tested antiviruses through the paces, the magazine hired Independent Security Evaluators (ISE), an external consultancy located in Baltimore, to create 5,500 new variants of known viruses, using them to test the products for their ability to detect unexpected threats.

The magazine describes the methodology used:

"To pit the software against novel threats not identified on signature lists, we created 5,500 new virus variants derived from six categories of known viruses, the kind you’d most likely encounter in real life."

Now Igor Muttik at McAfee has published an open letter challenging Consumer Reports' methodology. Now there are more than 100+ antivirus experts who placed their signature to say that creating new viruses is not an acceptable practice in testing antivirus performance.

Graham Cluley, of Sophos (UK), echoed these concerns:

"When I read about what ConsumerReports has done I want to bash my head against a brick wall. With over 185,000 viruses in existence was it really necessary for this magazine to create 5,000 more? It's a bit like Fire Monthly Magazine testing fire stations by lighting umpteen fires around the country and seeing who is the fastest at putting them out.

Consumer Reports' September issue is on sale now, a detailed description of its testing methodology is available at this link.

Anti Spam Review
Internet Security Settings
ID Theft Protection


Post a Comment

Links to this post:

Create a Link

<< Home