Best Free Duplicate File Remover

 
Introduction

If you are like me, you often find that you create duplicate files as you rename or use files in another context, or just backup more than you had intended. Duplicate File Detectors help you find and/or delete these duplicates and regain the storage space they previously occupied.

Judging from some of the recent comments, it would appear that some cautions should be noted before using this type of software. One might also note that with the cost per gigabyte rapidly decreasing (As of Oct 2011, 500 GB drives can be found for $39), redundancy is no longer such a bad thing.

Don't ever just blindly run the program, find all the duplicates and then click "remove". This can be disastrous in the sense that even though filenames are the same, sometimes the content is different. For example, you may have multiple icons with the same name, but of differing sizes, e.g., 256x256, 128x128, etc. Or, you may have a photograph that has been modified when placed in a different folder but retains the same filename.

Don't scan too much at once. A complete scan of drive C: would not be smart. However, scanning a few related folders at one time is much more efficient and the results less overwhelming.

Despite the best software, human intelligence may still have to be used. It is rumored that Abraham Lincoln stated that common sense was very uncommon. It is best, like eating an elephant, to take one bite/one small scan at a time.

OK, rant finished.

 
In a Hurry?

Go to details...  Go straight to the Quick Selection Guide

 
Discussion

Over the past few days, I've had the opportunity to take a look at a program by the name of SearchMyFiles by NirSoft. It didn't take long to realize just how well this program works. It features a clean and simple layout and the ability to perform both standard searches or duplicate file searches with as little or as many filters as you want. For standard searches, the plentiful search filters allow for some effective searching. However, I didn't find the filters to be as useful when searching for duplicate files though. Despite that, I was still impresses with the numerous search filters such as searching for files made within specific time frames, a certain file size range, certain file extensions, and so much more. Add on to this that the program can be used on of a USB drive, and you have a program that is not only useful, but flexible too. The only negative I could muster up on this one was that the search results are in a seperate window. Having to switch back and forth between search options and search results was a little bit of a drag. Although not a big deal, it would be nice to have everything in the same window. In my opinion, NirSoft has really hit the nail on the head with this one. I've tried the other duplicate file detector programs here and I can honestly say that this one is the best one here thus far.

 

Duplicate CleanerThe following is an example of why you may require duplicate file software: you have 50 duplicates and you need to decide what to do with each file, but what you get are 50 pop-ups, one for every file. Duplicate Cleaner Free has a very streamlined user interface divided into tabs. The scanning job sequentially progresses to the next tab. It also scans for files matching their CRC checksums. Scanning is very fast in comparison to Clone Spy.

Scan results are shown as a list in the results tab, where you can select any file by looking at its name, size, date, location, or use ‘Selection Assistant’.

The Selection Assistant selects files by date, smaller/greater size, or by having a specific string in file name/file path. It offers image preview of common image formats, and has a music search mode which can involve bit rate and size. It can also create hard links of the deleted duplicate files to the single file which you opt to keep. Hard linked files behave like any standard files, but are cross-linked to one single file. Read further details about hard links before using this option.  

 

Auslogics Duplicate File<br />             Finder Auslogics Duplicate File Finder is another option. It is fast and easy to use.

 

Anti-TwinAnti-Twin This very powerful program performs well and adds the ability to search unicode files. It will indeed be the No. 1 choice for many users. It has many great features and has many helpful hints. It is speedy, efficient and the unicode capabilities are a definite plus.

 

CloneSpyCloneSpy is the one of the oldest programs in its class. It's certainly not the fastest, but it's definitely the smartest. You can specifically select what is to be scanned and this can include multiple drives, multiple folders, or just individual folders. It has the ability to detect duplicates by CRC, by file name, by CRC and file name, and by file name and size. It will also detect zero length files. The program does not need to be installed but can be run from the executable so it's a good candidate for your USB Flash drive toolkit. It also throws up duplicates in a way that at least makes you think about what you are deleting. Click-happy users may find this latter feature an annoyance; to me it's a safety feature. 

 

Other Duplicate File Removers
  • Fast Duplicate File Finder is fast and easy to use. It also features two specialty searches:  Graphics and music. However, these functions are now limited in the free version.
  • Exact Duplicate Finder is fast and easy to use, does a credible job and it has enough options to satisfy most users.
  • Double Killer is fast and easy to use and does a credible job and it has enough options to satisfy most users.
  • AllDup is yet another option. It is fast and easy to use, does a credible job and it has plenty of options.
 
Quick Selection Guide

SearchMyFiles
5
 
Gizmo's Freeware award as the best product in its class!

Runs as a stand-alone program on a user's computer
Easy to use, plenty of search options, searches run quickly
Search results appear in separate window
http://www.nirsoft.net/
2.40
94.64 KB
32 and 64 bit versions available
Unrestricted freeware
This product is portable.
Windows 2000 up to Windows 7

View the version history here

Duplicate Cleaner Free
4
 
Runs as a stand-alone program on a user's computer
Fast scan, selection assistant, music search mode
Installer is bundled with unwanted components. Requires Microsoft .NET Framework version 2+ to be installed
3.1.6
1.25 MB
Feature limited freeware
There is no portable version of this product available.
Windows 98/ME/2000/XP/Vista/7

Will try to install toolbars and other unwanted components.

Auslogics Duplicate File Finder
4
 
Runs as a stand-alone program on a user's computer
Fast scan, clean interface
3.1.0.0
4.6 MB
32 and 64 bit versions available
Free for private use only
There is no portable version of this product available.
Windows XP/ 2003/ Vista/ 2008/ 7/ 8
Anti-Twin
4
 
Runs as a stand-alone program on a user's computer
Easy to use, search process is speedy, system files can be excluded, helpful search suggestions, can search images, performs unicode searches
1.8d
889.9 KB
Free for private use only
A portable version of this product is available from the developer.
Windows 95/98/Me/2000/XP/Vista/7

Anti-Twin is also available for Linux and Mac via Wine: http://www.winehq.org/
v1.8d released 16 May, 2010

CloneSpy
4
 
Runs as a stand-alone program on a user's computer
Detect duplicates by CRC, by file name, by CRC and file name, and by file name and size
A bit slow, a lot of confirmation clicks
http://www.clonespy.com/
3.01
2.56 MB
Unrestricted freeware
There is no portable version of this product available.
Windows 98se/NT/ME/2000/XP

v2.7 released 15 December, 2012
View the version history here

Fast Duplicate File Finder
3
 
Runs as a stand-alone program on a user's computer
Fast scan, selection assistant, music search mode
3.7.0.1
2.14 MB
Feature limited freeware
There is no portable version of this product available.
Windows 98/ ME/ 2000/ XP/ Vista/ 7/ 8

Windows 95 installations are currently untested/unsupported.

Exact Duplicate Finder
3
 
Runs as a stand-alone program on a user's computer
Fast scan, adequate search options, good display of results
0.9.6.17
104 KB ZIP
Unrestricted freeware
There is no portable version of this product available.
Windows 98/ ME/ 2000/ XP/ Vista/ 7/ 8

v0.9.6.17 released 18 March, 2013

Double Killer
3
 
Available either as a web service or stand alone program
Fast scan, adequate search options, nice display of results
1.6.2.82
392 KB
Feature limited freeware
There is no portable version of this product available.
Windows 98/ME/2000/XP/Vista/7

v1.6.2.82 released 7 July, 2007

AllDup
3
 
Runs as a stand-alone program on a user's computer
Fast scan, good search options, nice display of results
http://www.alldup.info/
3.4.24
3.3 MB ZIP
Unrestricted freeware
A portable version of this product is available from the developer.
Windows 98/ME/2000/XP/Vista/7

v3.4.24 released 27 June, 2013

 
Editor

This software review is copy-edited by Victor Laurie. Please help edit and improve this article by clicking here.

Back to the top of the article.

 

Share this
2.79374
Average: 2.8 (543 votes)
Your rating: None

Comments

by ADW on 16. December 2014 - 23:11  (120134)

Has anyone come across a duplicate file finder that can be activated from the Windows Explorer Context Menu, or added to the 'Send To' menu, so that whichever folder you right-click on is automatically added to the program as the search folder?

by Remah on 7. December 2014 - 9:43  (120020)

DupeGuru is now free. It has a 64-bit version as well as Mac OS X and Linux (Ubuntu, Arch) versions.
http://www.hardcoded.net/dupeguru/

by Ghost_Archer on 2. December 2014 - 4:15  (119928)

Search my files lack a simple but useful function -- to search files in the same name, or same size, etc. It more focus on content comparison, maybe based on MD5 indexing first. However, sometimes two txt document are almost same, 1 emtpy line would create different md5.

Duplicate cleaner free and double killer supports name and size match. Glary utilities also has duplicate clean function, however, I am not sure it does content check. Of course, we can do a standard search using search my files, then sort the list by name or size to find the similar files, apparently it is awkward.

I used to use double killer till it started to have problem with my vista. It is a nice one but as the development stops at 2007, I am not sure it works with win7 and later windows, especially 64 bit system. Download.com indicates it supports up to vista, while softonic thinks it supports up to xp.

by CoreTM on 25. April 2014 - 20:16  (115892)

The new Exact Duplicate Finder removes duplicates faster.
http://www.techsupportalert.com/best-free-duplicate-file-remover.htm#Exa...

by sicknero on 25. April 2014 - 20:39  (115893)

I just gave that a quick test drive on a folder of 5000 pictures - it flagged 6 images which aren't even close to being matches.

by CoreTM on 25. April 2014 - 21:33  (115894)

Read the user's manual. Files in the list aren't identical to each other.

by sicknero on 25. April 2014 - 22:00  (115895)

Oops, yes that was a bit hasty of me sorry.

I see now what happened ... the test folder I ran it on contains quite a few similar images some of which are also dupicated, so when the program finished and presented the list, it looked as if it had flagged the similar images as duplicates.

Cheers : )

by inthefog on 10. March 2014 - 15:12  (114925)

Duplicate Cleaner Free no longer has any toolbar bundling - it's completely clean (and free).

by newsuperhuman on 30. December 2013 - 5:24  (113270)

Auslogics is fast? By what standard? I am in the process of deleting 50,000 duplicates in 2 folders on a 3rd generation i5 4 core Win 7 64 bit computer with 8mg RAM, with nothing else running. It's been over 5 hours and still has about 40% to go. If this is fast, what is slow?

by stuhindl on 16. March 2014 - 23:11  (115055)

I have tried Auslogics but I am not sure of it end of run interface seems limited. On the matter of speed or performance I run a number of "boxes' none of them the latest But I dont think it may matter much as file seaching by exact bits particuly large image or video files have to be by the very nature indepth and precise and this does take time over a large number of files Stuhindl

by ichabod on 18. November 2013 - 20:59  (112355)

Although this category title focuses on the *removal* of duplicate files, the current reviews would be better described as being for a Duplicate File "Finder" category. Each review should focus seriously on the removal process, since that is by far the most time-consuming for users' hands-on efforts.

In comments for a related article ("A Duplicate File Finder That's Simple To Use And Unbelievably Fast"), Century22 and 11bravo pointed out what may be the single most important feature to anyone facing a large number of duplicates: an automated way to protect from deletion one file in each group of duplicates and then to choose between quick, batch deletion of all duplicates that weren't so protected versus manual deletion of selected duplicates. Note that I'm referring only to true duplicates, for which the minimum requirement is that files be identical in content; it would be desirable to have the added option to require duplication in data such as date and name and perhaps other attributes. Without this combination of automated protection and batch deletion, the user is faced with doing that manually for each group of duplicates, which could take countless hours with a large number of duplicates.

The next editor for this software category would do well to add to each program's review a statement of whether the program offers such an automated deletion-with-protection feature, or requires manual selection of each deletion, which is probably more error-prone than a well-designed automated method.

by RandyDoqa on 16. November 2013 - 22:08  (112275)

Duplicate Cleaner Free is bundled with OpenCandy now.

by indnajns on 19. September 2013 - 16:20  (110873)

Anti-Twin turned out to be what I was looking for. Duplicate Cleaner Free and Auslogics wouldn't do networked drives. I'm not looking to compare audio, video, or family pics. I'm looking to get rid of the 400 copies of plain, old documents that have been created all across my network. We'd been trading core files around with a USB drive. Now we've got a NAS and everybody has their own private copy of every file from the past three years plus their own stuff that all needs to be merged back onto the NAS. I'm OCD, but that's tedious even for me!

Pros for Anti-Twin:
Works on networked drives
Compares by name, content, size, or date or a combination of these
Allows a % match. DCF didn't recognize a lot of these files as duplicates because several have been re-saved with nothing more than a carriage-return or a blank space difference. I don't care! Same date, same file name, same basic file size, same file for my purposes.
Clearly lists all duplicates in matched groups showing file path/name, date, and size.
Allows deletion by directory! After it finds the duplicates, I can check a single folder (like one of the many copies) and it will delete from the copy and leave my original alone.
Interface is clean and user-friendly. I was up and running within minutes, maybe even seconds. Intuitive checkboxes.

Thank you for this listing of duplicate finding programs, and especially for linking to Anti-Twin. I've spent many blurry-eyed hours comparing files. Now I may actually get this project done before my next birthday.

by sarahwilliams25 on 8. August 2013 - 14:59  (109984)

Who is rating those tools?
I am amazed that SearchMyFiles has 5 stars with that unusable ugly interface. I tried all those tools and I can say that the best in this list is "Fast Duplicate File Finder".
It is completely free for "100% Identical files" and "Similar File Names" modes. Actually I do not see even "Similar File Names" mode in the other tools - this is very hand if you have My CV.doc, My Cv (1).doc, My Cv 2013.doc named files which is quite common situation.
Yes FDFF has some small limitations in the free version - the mode for finding files with similar (not duplicate) content lists just 10 groups (which is even usable), but for the average user like me the free version is more than sufficient.

I love Gizmo's freeware, but in my opinion this article is not objective and requires revising.

Thank you for the great site though.

P.S. I just noticed that your version is also outdated - the latest version is 3.7.0.1

by AlphaTron on 23. September 2013 - 11:25  (110947)

DupliateMyFiles by Nirsoft.net works really well. It may be more geared for PC experts than for novice PC users because it doesn't give enough detail for some of the fields. It could use some better instructions but it's free. Maybe some technical writer will improve the instructions for free. I suck at writing or I would.

by MidnightCowboy on 8. August 2013 - 15:29  (109986)

There is currently no editor for this review. It was last updated over a year ago and won't be again until someone volunteers to take it over. Maybe you would be interested? MC - Site Manager.

by sarahwilliams25 on 8. August 2013 - 16:28  (109987)

Yes I see that other tools like Auslogics are listed with old versions too.
Thank you, I am flattered! I am not sure if I will have the sufficient time for this, but for sure I can try... I will at least correct the current listings.

by MidnightCowboy on 9. August 2013 - 0:17  (109999)

Thank you for considering us. The video on this page will give you more idea about what is involved with being a Gizmo's Freeware Editor. MC

http://www.techsupportalert.com/we-need-you.htm

by Anupam on 8. August 2013 - 17:05  (109988)

For correcting the listings, we have an archive manager in the team, who updates the articles without editors, and he will eventually get down to updating this one too, in time. But, what's really needed is an editor, who will maintain the category, testing out different software, add news ones if there are any, and update the article accordingly.

by Giovy on 8. August 2013 - 12:50  (109978)

This tool is the best one I've tried so far:

http://malich.ru/duplicate_searcher.aspx

As you can see, this magic free app not only deletes duplicate files but it's even able to replace duplicates files with NTFS hard links as well, a key feature missing in any tool mentioned above.

I'm amazed you didn't mention it!

by mjcummins on 16. May 2013 - 7:40  (107778)

I think there is an important item being overlooked in the comparision.

Won't affect the selection of Best in Class though.

I'm using Windows 7 64bit, and NAS4Free on a Microserver on the home network. I have a lot of files strewn across USB hard drive. A recent external drive failure caused me to get a NAS solution.

So this duplicate file finder mechanism is really important for home use with the amount of digital information normal families accumulate.

Well, neither "Duplicate Cleaner Free" or "Auslogics Duplicate File Finder" will touch the windows share from my NAS server.

"SearchMyFiles" performs flawlessly.... Love it.

So could TSA add network share capable as part of it's tests please.

Thanks for the awesome work, and awesome site.

Michael

by skan on 12. March 2013 - 18:07  (106180)

Can any of these applications look inside compressed files (rar, zip, cab...)?

by phantom.mojo on 30. March 2013 - 23:43  (106682)

Testing is almost complete. Will have answers for you shortly.

by Sigurros on 5. March 2013 - 9:45  (105958)

Duplicate Cleaner is clean and doesn't include a Bing Bar (or any toolbar) in the installer - you need to update your listing! My vote for the best though, as it has more features than most of the paid guys.

by phantom.mojo on 11. March 2013 - 3:17  (106109)

Thanks for your input. I'm still running through all the duplicate file detectors in this list. I will be sure to update all Duplicate Cleaner's particulars as soon as I get done testing.

Gizmo's Freeware is Recruiting!

Gizmos Needs YouShare your knowledge of free software with millions of Gizmo's readers by joining our editing team.  Details here.