Remove Duplicate Lines

Features

• Preserves original order by default
• Case-insensitive and whitespace-trimming modes
• Keep the first or last occurrence, or drop duplicates entirely
• Optional alphabetical or length sorting
• Reports exactly how many lines were removed

How to use it

1. Paste your list, one item per line.
2. Choose how strictly lines should be matched.
3. Decide which occurrence to keep.
4. Copy the deduplicated list or download it.

Choosing the right matching rules

Exact matching is the safest default but often misses what you meant. Trailing spaces are invisible and extremely common in lists pasted from spreadsheets or terminal output, so apple and apple  look identical while comparing as different strings. Leaving the trim option on catches those.

Case sensitivity depends entirely on your data. Email addresses are case-insensitive in the domain part and effectively so in practice, meaning User@Example.com and user@example.com reach the same inbox. Passwords, file paths on Linux, and most identifiers are case-sensitive, so folding case there would destroy meaning.

The third option deserves explanation. Keep the first occurrence and keep the last both leave you with one copy of every line. Remove all duplicated lines entirely does something different: it keeps only lines that appeared exactly once, discarding every line that was ever repeated. That is what you want when you are looking for the items unique to a list rather than a tidy version of it.