Skip to content

Keep a correction's meaning in Deep's check - #50

Closed
purinda wants to merge 2 commits into
deep-check-speech-formsfrom
deep-check-correction-meaning
Closed

purinda wants to merge 2 commits into
deep-check-speech-formsfrom
deep-check-correction-meaning

Conversation

@purinda

@purinda purinda commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Summary

Deep's check (SelfRepair, Swift and Rust) accepted answers that resolve a self-correction the wrong way while keeping every word in its place: "book the blue room sorry the green room for friday" → "Book the blue room for Friday." or "Book the blue green room for Friday." It now reads what a correction means:

  • The phrase's key word ("green") stays, as itself or a word like it ("busses" → "buses"), never as another word ("trains").
  • The corrected words that key word takes back ("blue") are never written again.
  • A phrase that says the corrected words again after a new word corrects the word before them too ("The billing service goes live. Sorry, I mean the login service." → "The login service goes live.").
  • A fact or a name never leaves the one of its sort just before the words it corrects. "three servers, sorry, four" corrects "three", and "Invite Sam to the launch. Sorry, Priya." is never "Invite Sam and Priya to the launch."
  • Misheard cues ("know" for "no") and misheard small words ("thee") say nothing a correction says.
  • "a.m." and "p.m." end a sentence only before a capital.

It also fixes a crash. A second correction could take a cue out of a phrase an earlier one had moved ("I'll call no one now. Scratch that. I'll email no one instead."). The phrase then kept its old length and the alignment read past the end of the words.

Stacked on #49.

Test plan

  • Swift CleanupTests: 190 pass, including the guard fixture verdicts. lt-cleanup: 200 pass. clippy and rustfmt are clean.
  • Guard fixtures: three verdicts change, each to rejecting a wrong answer.
  • Replay through the Rust check:
    • No right answer among 19,826 training and test targets is newly rejected. Six wrong ones are.
    • The crash fix changes no verdict on those targets or on 3,566 recorded answers from two Deep adapters.

Corrections that take back too little or too much (second commit)

Two more kinds of wrong answer passed this check and the check 1.1.0 ships with. Retrained Deep adapters write them:

  • A new instruction spliced into the old one. "Ship it to Prague, scratch that, hold it until September." → "Ship it to hold it until September." Now a phrase that opens the way its sentence did ("Ship it" → "hold it") starts that sentence again, and takes it back from its first word that carries meaning.
  • More taken back than the words said again. "the physio team sorry not physio nursing will join the call at noon" → "The nursing will join the call at noon." After "not", a correction now takes back exactly the words said again.

Through the Rust check, against the first commit:

  • 19,826 training and test targets: one right answer is newly rejected. It is a false start whose restated name leaves a verb behind: "Nia knows sorry, not Nia Matteo will lead …". It falls back to Standard.
  • 6,124 recorded answers from three Deep adapters: ten newly rejected, all of them wrong. None is newly accepted.
  • Guard fixtures: 68 new cases (the inputs of SelfRepairTests' correction tests), and no earlier verdict changes. Swift CleanupTests 194, lt-cleanup 202. clippy and rustfmt are clean.

Still accepted: a correction that takes back a whole sentence so far without saying any of it again: "insurance for ferries no wait boats went up again" → "Boats went up again." Without capitals it has the same shape as a right false start ("alice knows sorry tara will lead the design review" → "Tara will lead the design review."), so no rule here can tell them apart.

🤖 Generated with Claude Code

Deep's check (SelfRepair, in Swift and Rust) accepted answers that
resolve a self-correction the wrong way while keeping every word in its
place: "book the blue room sorry the green room for friday" → "Book the
blue room for Friday." or "Book the blue green room for Friday." It now
reads what a correction says:

- The phrase's key word, the first word that says something new
  ("green"), stays, as itself or a word like it ("busses" → "buses"),
  never as another word ("trains").
- The corrected words that key word takes back ("blue") are never
  written again.
- A phrase that says the corrected words again after a new word corrects
  the word before them too ("The billing service goes live. Sorry, I
  mean the login service." → "The login service goes live.").
- A fact or a name never leaves the one of its sort just before the
  words it corrects: "three servers, sorry, four" corrects "three", and
  "Invite Sam to the launch. Sorry, Priya." is never "Invite Sam and
  Priya to the launch."
- Misheard cues ("know" for "no") and misheard small words ("thee") say
  nothing a correction says.
- "a.m." and "p.m." end a sentence only before a capital.

It also fixes a crash. When a second correction took a cue out of a
phrase an earlier one had moved ("I'll call no one now. Scratch that.
I'll email no one instead."), the phrase kept its old length and the
alignment read past the end of the words. A moved phrase now ends where
its words stop following each other.

Guard fixtures: three verdicts change, each to rejecting a wrong answer
("cars, sorry, busses" → "trains"; "say sorry to Jo" → "say it to Jo";
"Invite Sam and Priya to the launch"). Through the Rust check, no right
one of 19,826 training and test targets is newly rejected; six wrong
ones are (a second cue dropped, a date correction that keeps the day it
retracts). The crash fix changes no verdict on those targets or on 3,566
recorded answers of two Deep adapters. The Swift CleanupTests (190) and
lt-cleanup's 200 tests pass; clippy and rustfmt are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two kinds of wrong answer passed Deep's check (SelfRepair, in Swift and
Rust), including the check 1.1.0 ships with:

- A new instruction spliced into the old one: "Ship it to Prague,
  scratch that, hold it until September." → "Ship it to hold it until
  September." The check read the phrase as standing in for "Prague",
  and its grammar fix may add "it" and "to". Now a phrase that opens the
  way its sentence did starts that sentence again, and takes back from
  the sentence's first word that carries meaning. The phrase's first
  word is that word ("Book …. Scratch that. Book …"); or carries meaning
  in its place, with its second the word said next ("Ship it" → "hold
  it"); or follows a word that says nothing new ("just ship it").
- More taken back than the corrected words said again: "the physio team
  sorry not physio nursing will join the call at noon" → "The nursing
  will join the call at noon.", and "Kofi's brother, no wait, not
  brother, cousin, …" → "Cousin is hosting …". After "not", a
  correction now takes back the words said again, and the words beside
  them stay.

A retrained Deep adapter wrote the first in all four of Standard's cases
of that form. The shipped adapter and a retrained one wrote the second.

Still accepted: a correction that takes back a whole sentence so far
without saying any of it again ("insurance for ferries no wait boats
went up again" → "Boats went up again."). Without capitals it reads
like a false start ("alice knows sorry tara will lead the design
review" → "Tara will lead the design review."), which must pass.

Through the Rust check, one right answer among 19,826 training and test
targets is newly rejected: a false start whose restated name leaves a
verb behind ("Nia knows sorry, not Nia Matteo will lead the design
review."). Of 6,124 recorded answers from three Deep adapters, ten are
newly rejected, all of them wrong, and none is newly accepted.

Guard fixtures: 68 new cases, the inputs of SelfRepairTests' correction
tests, and no earlier verdict changes. The Swift CleanupTests (194) and
lt-cleanup's 202 tests pass; clippy and rustfmt are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@purinda

purinda commented Oct 3, 2026

Copy link
Copy Markdown
Member Author

Superseded by #58, which merged this work into main with the other changes behind candidate E. It shipped in 1.2.0. (GitHub closed this PR when its base branch was deleted.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant