uniq not working? yes it is

Remember this simple fact and you wont feel like an idiot (like i did about 10minutes ago). Lets say you have a text file with the following:

cats
cats

if you run that file through sort -u you will get the exact same output. Sound crazy? yeah thats what i thought, until you see the invisible. Whitespace. the second "cats" has a space after it. The space at the end makes that line completely different from the previous "cats".

Use the following line to eliminate the whitespace:
sed 's/ //g' filename

Now, run it through sort -u and be a happy camper.

kthnxbai