. To remove duplicate lines from a file use the command: uniq in_file out_file . uniq searches the in_file for consecutive duplicate lines - a line that is immediately followed by one or more lines that are identical to it - and removes them. The result is an out_file that contains no duplicate lines. Use the sort command if you want to remove duplicate lines that occur in different parts of a file. Examples .