• Skip to main content

Uly.me

cloud engineer

  • Home
  • About
  • Archives

Clean Duplicates in File

December 9, 2018

Here’s a nice command to get rid of line duplicates in a file. I’m sending the output to a temporary file then renaming it back to the original file. The reason is, it’s probably not a good idea to overwrite a file that you are editing. It’s much safer to send the output to another file.

$ awk '!seen[$0]++' backup.sh > backup01.sh
$ mv backup01.sh backup.sh

$ awk '!seen[$0]++' backup.sh > backup01.sh $ mv backup01.sh backup.sh

Filed Under: Linux Tagged With: awk, duplicates

Search This Website

Subscribe Via Email

  • Home
  • About
  • Archives

Copyright © 2023