Uly.me

cloud engineer

  • Home
  • About
  • Archives
Home/Linux/Clean Duplicates in File

December 9, 2018

Clean Duplicates in File

Here’s a nice command to get rid of line duplicates in a file. I’m sending the output to a temporary file then renaming it back to the original file. The reason is, it’s probably not a good idea to overwrite a file that you are editing. It’s much safer to send the output to another file.

$ awk '!seen[$0]++' backup.sh > backup01.sh
$ mv backup01.sh backup.sh

$ awk '!seen[$0]++' backup.sh > backup01.sh $ mv backup01.sh backup.sh

Filed Under: Linux Tagged With: awk, duplicates

Have content delivered to your mail. Subscribe below.

About Me

I'm Ulysses, Cloud Engineer at Cardinal Health based in Columbus. This blog is about Linux and Cloud technology. When off the grid, I enjoy riding the electric skateboard. I've surfed, snowboarded and played the saxophone in the past. I hope you will find this website helpful. It's powered by WordPress and hosted on AWS LightSail.

  • Cloud
  • Linux
  • Git

Copyright © 2012–2021