If you are like me, you often find that you create duplicate files as you rename or use files in another context, or just backup more than you had intended. Duplicate File Detectors help you find and/or delete these duplicates and regain the storage space they previously occupied.
Judging from some of the recent comments, it would appear that some cautions should be noted before using this type of software. One might also note that with the cost per gigabyte rapidly decreasing (As of Oct 2011, 500 GB drives can be found for $39), redundancy is no longer such a bad thing.
Don't ever just blindly run the program, find all the duplicates and then click "remove". This can be disastrous in the sense that even though filenames are the same, sometimes the content is different. For example, you may have multiple icons with the same name, but of differing sizes, e.g., 256x256, 128x128, etc. Or, you may have a photograph that has been modified when placed in a different folder but retains the same filename.
Don't scan too much at once. A complete scan of drive C: would not be smart. However, scanning a few related folders at one time is much more efficient and the results less overwhelming.
Despite the best software, human intelligence may still have to be used. It is rumored that Abraham Lincoln stated that common sense was very uncommon. It is best, like eating an elephant, to take one bite/one small scan at a time.
OK, rant finished.
Go straight to the Quick Selection Guide
Over the past few days, I've had the opportunity to take a look at a program by the name of SearchMyFiles by NirSoft. It didn't take long to realize just how well this program works. It features a clean and simple layout and the ability to perform both standard searches or duplicate file searches with as little or as many filters as you want. For standard searches, the plentiful search filters allow for some effective searching. However, I didn't find the filters to be as useful when searching for duplicate files though. Despite that, I was still impresses with the numerous search filters such as searching for files made within specific time frames, a certain file size range, certain file extensions, and so much more. Add on to this that the program can be used on of a USB drive, and you have a program that is not only useful, but flexible too. The only negative I could muster up on this one was that the search results are in a seperate window. Having to switch back and forth between search options and search results was a little bit of a drag. Although not a big deal, it would be nice to have everything in the same window. In my opinion, NirSoft has really hit the nail on the head with this one. I've tried the other duplicate file detector programs here and I can honestly say that this one is the best one here thus far.
The following is an example of why you may require duplicate file software: you have 50 duplicates and you need to decide what to do with each file, but what you get are 50 pop-ups, one for every file. Duplicate Cleaner Free has a very streamlined user interface divided into tabs. The scanning job sequentially progresses to the next tab. It also scans for files matching their CRC checksums. Scanning is very fast in comparison to Clone Spy.
Scan results are shown as a list in the results tab, where you can select any file by looking at its name, size, date, location, or use ‘Selection Assistant’.
The Selection Assistant selects files by date, smaller/greater size, or by having a specific string in file name/file path. It offers image preview of common image formats, and has a music search mode which can involve bit rate and size. It can also create hard links of the deleted duplicate files to the single file which you opt to keep. Hard linked files behave like any standard files, but are cross-linked to one single file. Read further details about hard links before using this option.
Auslogics Duplicate File Finder is another option. It is fast and easy to use.
Anti-Twin This very powerful program performs well and adds the ability to search unicode files. It will indeed be the No. 1 choice for many users. It has many great features and has many helpful hints. It is speedy, efficient and the unicode capabilities are a definite plus.
CloneSpy is the one of the oldest programs in its class. It's certainly not the fastest, but it's definitely the smartest. You can specifically select what is to be scanned and this can include multiple drives, multiple folders, or just individual folders. It has the ability to detect duplicates by CRC, by file name, by CRC and file name, and by file name and size. It will also detect zero length files. The program does not need to be installed but can be run from the executable so it's a good candidate for your USB Flash drive toolkit. It also throws up duplicates in a way that at least makes you think about what you are deleting. Click-happy users may find this latter feature an annoyance; to me it's a safety feature.
- Fast Duplicate File Finder is fast and easy to use. It also features two specialty searches: Graphics and music. However, these functions are now limited in the free version.
- Exact Duplicate Finder is fast and easy to use, does a credible job and it has enough options to satisfy most users.
- Double Killer is fast and easy to use and does a credible job and it has enough options to satisfy most users.
- AllDup is yet another option. It is fast and easy to use, does a credible job and it has plenty of options.
Duplicate Cleaner Free
Auslogics Duplicate File Finder
Fast Duplicate File Finder
Exact Duplicate Finder