I like the idea of portable applications. They don't need any installation, so they don't add anything to your registry and they don't install any new services or drivers. To run the program, just double-click it. To uninstall the program, just delete the file(s) that make up the application.
Sadly, not all programs are available in portable form. But one way around this is to use an application virtualizer, and one that's new to me is Cameyo. You can get a pretty good idea of how it works by watching the 4-minute video at http://www.youtube.com/watch?v=3cuLYaYlo4k. To summarise, you first start Cameyo, then install the program you want, and configure it the way you like it. Then tell Cameyo to package the application you just installed, which will result in a single .EXE file. This is your packaged, virtualised, portable application, ready to use on your PC or on any other. Though it's always best to only use packaged apps under the same version of Windows that was used to create the package in the first place.
Cameyo is free, runs on all recent versions of Windows, and is a 5 MB download. You can get it from www.cameyo.com. Note that your antivirus software may report a problem with the download. The company is aware of this occasional tendency to create false alarms, and swears that the program is clean.
My thanks to site reader camgfx for this great Hot Find.