Here’s a nice command to get rid of line duplicates in a file. I’m sending the output to a temporary file then renaming it back to the original file. The reason is, it’s probably not a good idea to overwrite a file that you are editing. It’s much safer to send the output to another file.
$ awk '!seen[$0]++' backup.sh > backup01.sh $ mv backup01.sh backup.sh |