Most USB storage devices use the FAT file system. This includes pen drives, and MP3 players. There's nothing wrong with FAT, and it's ideally suited to removable devices because, unlike NTFS, there's no cacheing of data happening behind the scenes. So there's less chance of losing information if you unplug your device at the wrong time.
But the FAT file system doesn't sort files in alphabetical order. So, unless you access the device via a system that does, the order in which the files are displayed becomes ever more random as you continually copy files to it, delete them, or rename them.
The problem doesn't show up in Windows, because the OS sorts filenames before it displays the contents of the device. But try a directory listing from a command prompt, and it's a different story. As is using the device in an MP3 player and expecting the tracks to play in order.
If this is a problem for you, the solution is to simply find a utility that can sort the contents of the drive for you. And one great such utility is called FAT Sorter. It's free, it runs on Windows XP and above, and you can get it from http://fenux.net/story/fatsorteroneohfour. It's only a 0.4 MB download, but you'll also need .NT 3.5 if you don't already have it.
My thanks to site user Keith for this recommendation. Apparently his daughter is using it to sort out her MP3 files and is very impressed!