If you have a file in which all lines are sorted (alphabetically or
otherwise), you can easily delete (consecutive) duplicate lines. Simply
open the file in your favorite text editor, and do a search-and-replace searching for ^(.*)(\r?\n\1)+$ and replacing with \1. For this to work, the anchors need to match before and after line breaks (and not just at the start and the end of the file or string), and the dot must not match newlines.
