Text cleaner

Nothing happens until you ask

Every switch starts off, and with none of them on the output is character-for-character what you pasted. A cleaner that quietly rewrites text the moment it arrives is one you cannot hand anything you care about, so the untouched state is genuinely untouched.

The order is fixed, and it matters

Collapse repeated spaces, then trim the ends of lines, then remove blank lines, then remove duplicates. Applied in the order the switches were flicked, the results would differ, and not in a way anyone could predict.

The clearest case: a line containing three spaces is not blank. It only becomes blank once it has been trimmed, so removing blank lines before trimming would leave it standing. The same goes for duplicates, two lines that differ only in trailing whitespace are different strings until they have been trimmed, which is why a deduplication that “does not work” almost always starts working when trimming is turned on too.

What each step does

• Collapse repeated spaces turns runs of spaces and tabs into one space, within each line. Line breaks are untouched, so paragraphs survive.
• Trim the ends of lines removes leading and trailing whitespace from every line, leaving the middle alone.
• Remove blank lines drops empty lines and whitespace-only ones.
• Remove duplicate lines keeps the first of each repeated line and preserves the original order. It is case-sensitive, because case usually carries meaning in the lists people paste in.

Invisible differences

Windows line endings are normalised before anything else runs. Text copied from a Windows editor carries a carriage return before every newline, which makes two identical-looking lines compare as different and makes a deduplication look broken for no visible reason.

Where the mess comes from

Text out of a PDF, which arrives with line breaks in the middle of sentences and spacing that was really letter-spacing. Lists out of a spreadsheet or an email thread, which arrive with blank lines and repeats. Anything copied out of a web page, which brings the source’s indentation with it. All of it runs in your browser. Nothing is uploaded.