Remove Duplicate In A File Posted on September 14, 2018 by Ulysses Here’s a Perl script for removing duplicates in a file. perl -ne 'print unless $dup{$_}++;' input_file > output_fileperl -ne 'print unless $dup{$_}++;' input_file > output_file