Award trophy

Tech Support Alert - Where to Find the Best Tech Support Resources on the Web

Gizmo's Top Picks

Best Free Utilities

Best Free Support Sites

Best Paid Support Sites

Best Shareware and Freeware Sites

Best Troubleshooting Sites

Best Sites for Tips, Tricks and Tweaking

Best Computer Security Sites

Best PC Support Resources

Best Sites for Scripts, Drivers and Fonts

Computer Glossaries, Guides and Tutorials

Contact info

Home


NEW Reviews:

 Inkjet Printer Cartridges

If you are in the market for compatible inkjet printer cartridges you should check out our Editor's two new series of  reviews of the best and cheapest inkjet printer  cartridge sites.

If you are looking for the very best inkjet cartridge sites in terms of customer support then go straight to Inkjet printer cartridge sites reviews. We've bought inkjet cartridges from all the sites listed and can attest to the quality of service offered.

If the price of your inkjet cartridges is this most important issue for you then check out cheapest inkjet cartridge sites. Here we carry out a weekly price check on scores of inkjet cartridge sites and list the cheapest ten.


Sick of Spam?

Gizmo reviews the best spam blockers for average users 
Best Spam Blockers

 


 The Best Windows Backup Software

Our editor reviews 18 of the best selling backup software products but can only recommend five. But only one, get his "Editor's Choice"


 

 

 

 

16 Security Scanners Unplugged

An analysis of how well 16 popular security products cope with the latest generation of security threats.

The Problem

In a previous article "The Home Computer Security Mess" I talked of the home computer security conundrum. On the one hand we have the rapidly increasing sophistication of malware. On the other hand, computer security products have become so numerous, so complex and so overlapping in function that's it's become almost impossible for home users to make rational assessments of their effectiveness.

Towards a Solution

As a first step towards clarification I decided earlier this year to carry out a series of tests on home computer security programs. The first of these, reported here, is on signature based security products,  the most widely deployed of all home computer security programs.

This class of security product based products includes anti-virus, anti-trojan and anti-spyware scanners. 

The tests I conducted were quite unlike the traditional tests of such products that focus on the adequacy of their signature detection.

In contrast, I wasn't interested in signature detection but rather how well the products were equipped to handle the latest generation of security tests.

In particular I was interested in the following questions:

  • Can the security product be easily terminated by a hostile malware program?

  • Can the program detect malware within archived and compressed executable files?

  • Can it detect process injection?

  • Does it monitor changes in the Windows startup folder and registry startup areas?

  • Does the product detect the presence of running rootkits?

  • How good is it's protection again a modern blended threat?

  • How well does the product protect against drive-by downloads?

These are really important questions yet they are ignored in most product reviews. A product may have outstanding malware detection but it's worth zip if the security program can be easily terminated by a PE encrypted malware program.

To answer these questions I used a number of different technical test procedures. Several of these were based upon the methodology devised by Michel Aparicio at his blog site: http://kareldjag.over-blog.com/10-category-69553.html  Full details of the technical tests can be found in the detailed product test results below.

Test Methodology

My short list of products to be tested included most of the most popular free and commercial anti-virus, anti-trojan and anti-spyware scanners.

The programs tested were the most recent versions available at that time; June 2006. With a few minor exceptions, all programs were tested using their default configurations. Where exceptions were made, they have been noted in the results.

Two products, Ewido and CounterSpy had betas of new versions available so I tested the betas as well as the current products. One product,  Windows Defender was only available in Beta form.

In total I ended up with 16 products. Each was tested on a virtual PC running on a VMWare workstation.  Most products were tested using an unpatched version of Windows XP.  The beta products were tested with unpatched copy of Windows XP SP2 as this was a minimum requirement.

Here's the list of products tested. You can click on the product to get the detailed test results.

Note: Although I tested the most recent versions available at the time of testing in June, almost all products will have been updated to some degree since then. At the time of writing,  August 2006, Ewido V4 has moved out of beta, SpySweeper V5 has been released,  Spyware Doctor is up to V4 and CounterSpy V2 is now at a much more advanced beta version. My test results may have reduced applicability to these later versions.  

Can the security product be easily terminated by a hostile malware program?

In this test I tried terminating the product's monitor or core processes using several different termination tools.

 
Application

Termination Agent

  Windows Task Manager Diamond Advanced Program Termination IceSword/DarkSpy
Ad-Aware Pro V1.6 Fail Fail Fail
Avast! Home V4.7 Fail Fail Fail
AVG Anti-Virus Free V7.1 Fail Fail Fail
BitDefender Pro V9.095 Fail Fail Fail
CounterSpy V1.5 Fail Fail Fail
CounterSpy V2.0.122 beta Fail Fail Fail
Ewido v3.5 Resistant Resistant Fail
Ewido V4 beta Resistant Resistant Fail
Kaspersky AV V6.0.0 Resistant Resistant Fail
NOD32 V2.51 Resistant Resistant Resistant
Norton Antivirus 2006 Resistant Resistant Fail
SpyBot S&D V1.4 Fail Fail Fail
Spyware Doctor V3.6 Fail Fail Fail
Trojan Hunter V4.5 Fail Fail Fail
WebRoot SpySweeper V4.5 Fail Fail Fail
Windows Defender V1.1.1051 Fail Fail Fail

As you can see most products were easily terminated using Windows Task Manager, a result that can only be described as feeble.

Ewido, Kaspersky and Norton put up decent fights. If I terminated their processes the products would restart them again. However with the heavyweight tools I was able to delete the source files so they could not be restarted..

Only NOD32 was resistant to all three methods of termination. Even so. I was still able to terminate NOD32 by forcing a reboot and deleting some of its key files during the boot process.

Can the program detect malware within archived and compressed executable files?

 1. Archive Detection

I tested detection within archives using by embedding a malware product that I knew the product could detect within each of 11 different archive types: .7z, .bh, .cab,  .bz2, .gz, .jar, .lha, .rar, .tar, .yz1 and .zip.

I used installation default settings in all cases but note that some products such as  SpySweeper and Ad-Aware will only scan archives if you change the default settings. This possibility was not tested.

 
  7z bh cab bz2 gz jar lha rar tar yz1 zip
Ad-Aware Pro V1.6                      
Avast! Home V4.7 Y   Y Y Y Y Y Y Y   Y
AVG Anti-Virus Free V7.1     Y Y Y Y Y Y Y   Y
BitDefender Pro V9.095 Y   Y   Y Y Y Y Y   Y
CounterSpy V1.5                      
CounterSpy V2.0.122 beta                      
Ewido v3.5     Y     Y   Y     Y
Ewido V4 beta     Y     Y   Y     Y
Kaspersky AV V6.0.0     Y Y Y Y Y Y Y   Y
NOD32 V2.51     Y Y Y Y Y Y     Y
Norton Antivirus 2006     Y   Y     Y     Y
SpyBot S&D V1.4                      
Spyware Doctor V3.6                      
Trojan Hunter V4.5               Y     Y
WebRoot SpySweeper V4.5                      
Windows Defender V1.1.1051       Y Y Y Y Y Y   Y

Three aspects of the results are notable. First, no product scanned within all the archives. Second no product scanned the .bh and .yz1 archive types. Finally several products didn't scan even the most common archives such as .zip, .rar and .cab.

If a program doesn't scan with a particular type of archive file it can never detect  any malware that is within such a archive. It should however be capable of detecting the product once it is extracted from the archive.

However many folks routinely scan downloaded files before installing as a precaution. If that file happens to be archived then there are good chances that a scan will not reveal anything. If you use Ad-Aware, CounterSpy, SpyBot, Spyware Doctor or SpySweeper with their default settings, you are wasting your time. None of these products scan within any archive.

Compressed Executable Detection

To test detection within compressed executables I packed a malware program that I knew the product could detect using 11 different packers: WinZip SEA, WinRar SEA, Morphine, NSPack, Yoda, UPX, Thermida, Petite, TELock, Mew and FSG.

Some of these packers can produce polymorphically encrypted files executable files. That is, each file produces is different.  Detecting these files is a challenge for most security products yet they are widely used by hackers.
  zip rar mor nsp yod upx ther pet te mew fsg
Ad-Aware Pro V1.6                      
Avast! Home V4.7 Y Y                  
AVG Anti-Virus Free V7.1 Y Y Y Y Y Y   Y   Y Y
BitDefender Pro V9.095 Y Y Y Y Y Y   Y Y Y Y
CounterSpy V1.5                      
CounterSpy V2.0.122 beta                      
Ewido v3.5 Y Y Y   Y Y   Y   Y Y
Ewido V4 beta Y Y Y   Y Y   Y   Y Y
Kaspersky AV V6.0.0 Y Y Y Y Y Y   Y   Y Y
NOD32 V2.51 Y Y Y Y   Y   Y   Y Y
Norton Antivirus 2006 Y Y Y Y Y Y     Y Y Y
SpyBot S&D V1.4                      
Spyware Doctor V3.6                      
Trojan Hunter V4.5           Y          
WebRoot SpySweeper V4.5                      
Windows Defender V1.1.1051 Y Y                  

As with archived malware no security product detected within all the compressed executables. Some, notably Ad-Aware, SpyBot, Spyware Doctor, Trojan Hunter and SpySweeper were very poor in the scope.  No product was able to detect malware that was hidden by the commercial packer Thermida.

The inability of a security product to scan within compressed executables is more serious than not being able to scan within archives. Yes, the real time monitors in some products may catch the malware when it is executed but if the malware first terminates the security software then an infection is inevitable.

Again note that a precautionary scan of a downloaded file is useless if the security scanner doesn't scan within the packer used to produce the downloaded executable file. In particular note that If you use Ad-Aware, CounterSpy, SpyBot, Spyware Doctor or SpySweeper with their default settings, you are wasting your time. None of these products scan within any packed file.

Can the Security Product Detect Process Injection?

I used the ZapAss test program that injects an implant into a running process and then downloads a file using that process.

None of the security  products tested  warned of the process injection. Simple as that.  Better get out your IDS  program ;>)

Does it monitor changes in the Windows startup folder and registry startup areas?

To pass this test the program had to warn if changes were made in any of several different startup area.

No product passed the test.  Definitely get out your IDS  program ;>)

Does the product detect the presence of running rootkits?

In this test I loaded the Hacker Defender and FuTo rootkits while the security program and its monitor were deactivated. I then enabled the monitor and did a system scan.  To pass the test both rootkits had to be detected.

  Hacker Defender FuTo
Ad-Aware Pro V1.6
Avast! Home V4.7
AVG Anti-Virus Free V7.1 Yes
BitDefender Pro V9.095
CounterSpy V1.5
CounterSpy V2.0.122 beta
Ewido v3.5
Ewido V4 beta
Kaspersky AV V6.0.0 Yes
NOD32 V2.51 Yes
Norton Antivirus 2006 Yes
SpyBot S&D V1.4
Spyware Doctor V3.6 Yes Yes
Trojan Hunter V4.5
WebRoot SpySweeper V4.5 Yes, if enabled Yes, if enabled
Windows Defender V1.1.1051

Only Spyware Doctor and  SpySweeper detected both rootkits though in the later case, the rootkit detection option needed to be enabled. 

How good is it's protection again a modern blended threat?

This test involved running the DFK Threat Simulator, a sophisticated blended threat simulation that disables your defenses, bypasses your firewall, installs a cleverly disguised trojan, a virus and a keylogger all masked with a rootkit.

Ad-Aware Pro V1.6 Fail
Avast! Home V4.7 Pass
AVG Anti-Virus Free V7.1 Fail
BitDefender Pro V9.095 Conditional Pass
CounterSpy V1.5 Fail
CounterSpy V2.0.122 beta Fail
Ewido v3.5 Conditional Pass
Ewido V4 beta Conditional Pass
Kaspersky AV V6.0.0 Conditional Pass
NOD32 V2.51 Conditional Pass
Norton Antivirus 2006 Conditional Pass
SpyBot S&D V1.4 Fail
Spyware Doctor V3.6 Fail
Trojan Hunter V4.5 Fail
WebRoot SpySweeper V4.5 Fail
Windows Defender V1.1.1051 Fail

The only products that passed this deadly but revealing test were those that detected the test program by signature. I've rated this as a conditional pass. I suspect that had the programs not detected the test by signature then all would have failed as they were all included in the security program "kill list" embedded in the test program. 

How well does the product protect against drive-by downloads?

Here I browsed with Internet Explorer to three known drive-by download sites. These sites use flaws in Windows and Internet Explorer to download malware without any user action or knowledge. Typical exploits include the well known iFrame and WMF exploits though the sites will repeatedly try a sequence of exploits if not initially successful. If finally successful, the sites will download multiple malware products, often running into tens of megabytes.

After browsing I ran HijackThis and WhatChanged reports to see if there was an active infection. Because if the possibility of rootkit infection stealthing malware I also scanned with BlackLight and RootkitRevealer.

Ad-Aware Pro V1.6 Fail Fail Pass
Avast! Home V4.7 Pass Pass Pass
AVG Anti-Virus Free V7.1 Pass Pass Pass
BitDefender Pro V9.095 Pass Fail Fail
CounterSpy V1.5 Fail Fail Fail
CounterSpy V2.0.122 beta Pass Fail Fail
Ewido v3.5 Fail Fail Pass
Ewido V4 beta Fail Fail Fail
Kaspersky AV V6.0.0 Pass Pass Pass
NOD32 V2.51 Pass Pass Pass
Norton Antivirus 2006 Pass Pass Pass
SpyBot S&D V1.4 Pass Pass Fail
Spyware Doctor V3.6 Fail Fail Fail
Trojan Hunter V4.5 Fail Fail Fail
WebRoot SpySweeper V4.5 Cond'l Pass Cond'l Pass Cond'l Pass
Windows Defender V1.1.1051 Fail Fail Fail

SpySweeper has been given a conditional pass rating as access to the bad sites was blocked by SpySweeper.

Conclusions

Having looked at the results you have probably already concluded that most of the products failed most of the tests and alas, this is not far from the truth.

I'm resisting making more specific conclusions as this test is only the first of several I'll be conducting in the second half of 2006. In the coming months I'll be looking at virtualization products, IDS/IPS utilities and some other categories as well.

By the time this series is completed, I'll have some specific recommendations for you on the best way to protect your computer against the latest generation of threats. These recommendations will be based on facts rather than vendor hype or commercial affiliation.

Even now, two things are already clear to me.

First, it's almost impossible to defend your PC from a modern malware program that is allowed to run on your PC with full admin privileges. The problem here is not with the security programs. The problem is with Windows.

Second, it seems to me that virtualization techniques such as those used by VMWare, Sandboxie and several other products may offer the best option for preventing infection. Certainly the layering of defenses using multiple products is another option but I suspect the cost in terms of complexity, user confusion and processing overhead may make this a less attractive option that protecting the host PC through virtualization.

But this is speculation. When this series is completed we will (perhaps) know the real answer.

I'll be presenting the results of my next series of tests in Support Alert Newsletter. If you want to receive these results as they are published, you may wish to subscribe. It's free.

If you feel this article is of value, please post it to one or more of the following:  Digg it!   Del.icio.us   Furl

Gizmo

Ian "Gizmo" Richards
Editor
Support Alert Newsletter
http://www.techsupportalert.com

 

August 2006

 

FREE: Gizmo's famous "The 46 Best Ever Freeware Utilities" report.   Click here  to see it now

Home Shareware Free support Paid Support Troubleshooting Tips & tricks Security Resources  Scripts & drivers   Tutorials  
 Contact   

Copyright © techsupportalert.com 2006