My Words: tolerate typography when matching the writer's words

[kcarnold]

Makes My Words' text matching tolerant of typographic variation, so edits stop failing over hyphens, curly quotes, and spacing. Also fixes failure notices persisting long after the failure they describe.

Background

Every My Words tool that acts on the document — str_replace, insert … after, move, highlight — must locate a phrase the model produced inside text a host stores. Those two strings disagree routinely:

Source	Produces
Word autocorrect	en dash for -, curly ’ for '
Speech transcription	straight ASCII, and its own hyphenation
Paste	non-breaking and other exotic spaces
Typing habits	double space after a period

Matching was exact indexOf, so each of these read as "not found". The model then spent its turn re-viewing and retrying an edit whose target text was correct.

The word bank had the mirror-image problem in corpus.ts: tokenize treated well-being as a single atomic token, so a hyphenation disagreement rejected the writer's own words as not theirs — in both directions, since either side may be the hyphenated one.

Changes

New: frontend/src/utilities/textMatching.ts

A leniency ladder, applied most-literal-first:

Tier	Folds	Anchored
exact	nothing	—
typographic	dashes, quotes, exotic spaces canonicalized; invisibles (soft hyphen, zero-width, BOM) dropped	—
loose	+ case, collapsed space runs, hyphen-as-space	—
run-together	+ separators dropped entirely (email ↔ e-mail)	word boundaries

Each tier is exhausted across the entire search space before the next is tried, so an exact hit anywhere beats a folded hit elsewhere; leniency only rescues a search that would otherwise fail outright. Folds carry an index map back to the source, and callers slice the original text with it, so replaced or selected text is always the writer's real characters rather than the needle's spelling.

Consumers: interaction/ops.ts findSpan, and the standalone editor's selectPhrase in editor/editor.tsx. Word's selectPhrase was already lenient (ignorePunct/ignoreSpace/matchCase: false), which produced a specific failure: the reveal would highlight a phrase the subsequent apply then failed to find.

corpus.ts

An intra-word hyphen is now a word separator, and the corpus indexes both readings of a hyphenated compound (e-mail as e + mail, and as email). A proposal matching either reading validates.

The phrase rule is unchanged: well being must still appear contiguously in the corpus for well-being to be legal. What the AI gains is only the freedom to hyphenate or unhyphenate the writer's own adjacent words.

voice/session.ts, voice/VoiceSession.tsx

onNotice accepts null, which retracts the standing notice. The session retracts on writer speech and on a successful apply (documented as invariant 6). Previously a notice cleared only when replaced or manually dismissed, so a single word-bank rejection stayed on screen across many successful turns; a stale "that was blocked" is indistinguishable from a live one. Notices are still logged; only the strip is transient.

Design notes

Why the run-together tier is anchored. Dropping separators lets letters re-segment, so email matches inside the ache mail — a span no reader would call the same phrase, and splicing it rewrites the wrong text. That tier therefore requires the hit to begin and end on a word boundary in the source. The anchor is on that tier alone: literal substring matching is unchanged, so cat still matches inside concatenate as it always has.

Why the word bank draws the line at hyphens. In the locator, a bad match picks the wrong span. In the word bank, re-segmentation invents meaning: notable → not able is a claim the writer never made. So the closed-up form is offered only where a hyphen actually appears on one side or the other, never off a plain space — a hyphen is a mark the sources disagree about, a space is not. Both directions of the notable/not able case remain rejected, with tests.

The same rule makes a hyphen bind tighter than the glue-word rule: in-depth must be lifted whole rather than read as in (glue) plus a stray depth.

Why folds substitute by category instead of stripping non-alphanumerics. Stripping would break two things:

• Punctuation is editable content here — str_replace is routinely used to change a comma to a period. A locator blind to punctuation can't tell whether a trailing . falls inside the match, so the span it returns eats or drops a character of the writer's prose.
• It collapses the ladder. If the second tier already ignored all punctuation, nothing sits between byte-exact and maximally lenient, and a curly-quote mismatch would be resolved at the same leniency as ignoring a sentence boundary.

The disagreements are per-character glyph substitutions, so the fold is a same-length substitution. Substitutions derive from Unicode general categories (\p{Pd}, \p{Zs}, \p{Cf}) rather than a hand-listed table, so an unlisted variant can't go missing. Quotes stay enumerated because Pi/Pf don't distinguish single from double, and folding both to one character would make ' match ".

Span ends resolve to one past the last matched character, not to the next character the fold kept. Otherwise a match on e-mail in send an e-mail now returns a span of e-mail — the trailing space rides along because it was dropped by the fold.

Behavior change: move

move re-inserted op.phrase. Under exact matching that equalled the text it cut; under lenient matching it need not, so a move could silently restyle the text it relocated. It now re-places the characters it actually removed.

Incidental

• frontend/src/pages/my-words/CLAUDE.md (new) — call-path index for the feature: the two turn-loops, the shared edit path, and a table of the four places text is matched against the document. Those four have to stay in agreement, and nothing named them in one place. Notes that Google Docs' selectPhrase delegates to Apps Script and is not folded, and that MockEditor always succeeds, so a miss there is invisible in tests.
• corpus.ts contained two literal NUL bytes (the source-boundary sentinel), which made git treat the file as binary. They are named escapes now, so it diffs as text.
• docs/my-words-interaction-design.md — §4.5 and §4.6 record the notice lifetime and the typography/vocabulary split. Prior §4.5 is renumbered to §4.7; no existing cross-references pointed at it.

Testing

• src/utilities/__tests__/textMatching.test.ts (new) — tier ordering, index-map round-trips and monotonicity, span-end resolution, the word-boundary anchor (including that it doesn't retroactively constrain the exact tier), and that newlines are never folded into spaces.
• ops.test.ts — hyphenation in both directions, dash family, curly apostrophes, source-slicing, paragraph scoping under lenient matching, move fidelity, and that a genuine miss still throws.
• corpus.test.ts — hyphenation both ways, hyphen chains, closed vs. hyphenated compounds both ways, the notable/not able guard in both directions, and hyphen-over-glue binding.
• voiceControl.test.ts — invariant 6: retraction on speech and on a landed edit, notices still reaching the log, and a fresh notice standing until superseded.

267 unit tests pass; tsc --noEmit clean. The one eslint error and five prettier warnings present in the tree are pre-existing on main, in files this PR does not touch.