Hidden characters in text: where they hide and how to remove them

Hidden characters are invisible Unicode code points such as zero-width spaces, non-breaking spaces and directional marks. They look like an ordinary space or nothing at all, so the text appears fine while search, code and form validation treat it as different text. You find them by inspecting the code points, and you remove them by deleting or replacing those specific characters.

Remove hidden charactersInvisible junk in my text — free, runs in your browserOpen →

Where they come from

Most hidden characters arrive by copying, not by typing:

• Web pages. Sites use non-breaking spaces (U+00A0) to stop two words from splitting across a line, and zero-width spaces (U+200B) to mark where a long word may break. Copying a passage copies them too.
• Word processors. Soft hyphens (U+00AD) are inserted for hyphenation. They are invisible unless a line breaks at them. Some editors also insert non-breaking spaces automatically in certain places.
• Chat apps and email. Messages copied from chat or from an email client can carry zero-width joiners and direction marks, especially when the text mixes languages.
• Mixed-direction text. Arabic, Hebrew and mixed text use directional marks such as the left-to-right mark (U+200E), the right-to-left mark (U+200F) and the embedding and override characters (U+202A to U+202E). They control how the line is displayed, not what it says.
• Byte order marks. A byte order mark (BOM, U+FEFF) at the start of a file is a signature that some editors write. It is invisible, but it becomes a real first character when the file is read as plain text.
• AI chat windows. Text copied from some AI chat interfaces can contain these characters as well. The likely cause is how the text was rendered and copied, so do not read them as a sign of intent.

Why they cause problems

Each hidden character is a real code point, so it changes the text even when the page looks identical:

• Search fails. A zero-width space inside a word breaks the match, so a search for the word finds nothing.
• Code breaks. A non-breaking space in a source file is a syntax error in many languages, and the error often points to a line that looks perfect.
• Form validation rejects input. An email address with a zero-width space after the @ looks valid but fails the check.
• Counts drift. Character counts include or exclude these characters depending on the tool, so the same text can get two different totals.

Here is a small example. The word "invoice" has seven letters. With a zero-width space inside it, the same word is eight code points long, and it looks exactly the same on screen. A search for "invoice" will miss the second version.

How to see them

Looking at the text does not help, so use one of these:

1. Paste the text into a tool that lists the code points, such as remove hidden characters, which reports what it finds before anything is changed.
2. In a code editor, turn on the setting that shows whitespace or special characters. Non-breaking spaces then appear as a different mark from ordinary spaces.
3. In a programming language, print the code points of a suspicious string. In Python, for example, [hex(ord(c)) for c in text] lists each one, so a non-breaking space shows up as 0xa0 and a zero-width space as 0x200b.

How to remove them

Removal should be specific. Deleting every non-ASCII character would also destroy accents, quotes and letters from other languages. Instead:

• Replace non-breaking spaces with normal spaces, since they are almost always meant as spaces.
• Delete zero-width spaces, soft hyphens, byte order marks and stray direction marks.
• Keep accented letters, curly quotes and real punctuation.

Fix pasted text is built for text that came from somewhere else, so use it when the copy also has broken line breaks or odd spacing.

Edge cases

• Emoji sequences use zero-width joiners on purpose. Removing them splits one emoji into several symbols.
• A non-breaking space is correct in some typography, such as between a number and its unit, and in some French punctuation. Replace it only where you know it is a stray.
• Once the text is cleaned, compare it with the original to see exactly what changed.

Quick answers

Are hidden characters a sign that an AI wrote the text?

No. They come from web pages, word processors, chat apps and copy-paste in general. Their presence does not show who or what wrote the text.

Will removing every non-ASCII character fix the problem?

No. That would also remove accents, quotes and letters from other languages. Remove the specific invisible code points instead.

How can I tell if a string has hidden characters?

Compare its length with the number of characters you can see, or paste it into a tool that lists code points. If the counts differ, something invisible is inside.