Read a code span that survives a line break as one identifier - #13
Open
aaddrick wants to merge 1 commit into
Open
Read a code span that survives a line break as one identifier#13aaddrick wants to merge 1 commit into
aaddrick wants to merge 1 commit into
Conversation
INLINE_CODE was anchored to the line. A span that wraps -- ordinary in
files hard-wrapped at 80 columns, and rendered by Markdown as a single
identifier -- is therefore not matched at all, and the regex pairs that
span's closing backtick with the next span's opening one instead. The
census gains a phantom entry made of the prose between them and loses the
two real identifiers it existed to pin.
Demonstrated by reverting the pattern with the new tests in place:
AssertionError: Counter({' and check for ': 1})
!= Counter({'claude plugin list': 1, 'enabled': 1})
A translation cannot reproduce that phantom entry without leaving the
clause in English, so the multiset test demands the one thing it exists to
forbid. The sibling repository slushpile hit this on a troubleshooting page
and two translators duly shipped "and check for" mid-sentence in Portuguese
and Vietnamese before it was found.
No span in these six READMEs wraps today, so nothing here is currently
mispinned -- verified by comparing the old and new census across all six:
32 spans each, identical under both. That is the reason to fix it now
rather than a reason not to. The gate is exact by accident, and the
accident ends the first time somebody rewraps a paragraph, in a commit that
looks like prose editing.
The pattern now allows a newline and the census normalizes whitespace, so a
translation that fits the identifier on one line matches an English one that
does not. Fences are stripped first: without that, a newline-tolerant
pattern pairs the backticks of two fence delimiters and swallows the block
as a single span. Fence content is compared verbatim by its own test, so
the census loses nothing.
CensusTest asserts on constructed strings rather than on the READMEs,
because the failure is one no current README exhibits and a gate whose only
evidence is that the tree happens to pass it cannot be seen to work.
Claude-Session: https://claude.ai/code/session_01TfLpzhGer85CKJ2fZpvLN8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
INLINE_CODEintests/test_readmes.pywas anchored to the line. A span that wraps — ordinary in files hard-wrapped at 80 columns, and rendered by Markdown as a single identifier — is therefore not matched at all, and the regex pairs that span's closing backtick with the next span's opening one instead. The census gains a phantom entry made of the prose between them, and loses the two real identifiers it existed to pin.Demonstrated by reverting the pattern with the new tests in place:
A translation cannot reproduce that phantom entry without leaving the clause in English, so the multiset test ends up demanding the one thing it exists to forbid. The sibling repository slushpile hit this on a troubleshooting page and two translators duly shipped "and check for" mid-sentence in Portuguese and Vietnamese, each correctly reporting they had no legal alternative.
Nothing here is currently mispinned
Verified by computing the old and the new census across all six READMEs: 32 spans each, identical under both. No span in these files wraps today.
That is the reason to fix it now rather than a reason not to. The gate is exact by accident, and the accident ends the first time somebody rewraps a paragraph — in a commit that looks like prose editing and that nobody would think to check the census against.
The fix
The pattern now allows a newline, and the census normalizes whitespace so a translation that fits the identifier on one line matches an English one that does not.
Fences are stripped first, and that half is load bearing: without it, a newline-tolerant pattern pairs the backticks of two fence delimiters and swallows the whole block as a single enormous span. Fence content is compared verbatim by its own test, so the census loses nothing by not reading it.
The new tests assert on constructed strings
CensusTestdoes not read the READMEs, because the failure is one no current README exhibits. A gate whose only evidence is that the tree happens to pass it cannot be seen to work — which is exactly how this one was silently exact for the wrong reason. One assertion does readREADME.md, to catch a helper that returned nothing and would otherwise pass every other case.Gates
All four green:
sync_style --check,check_configs,run_evals validate, and 80 tests. The diff is one file.https://claude.ai/code/session_01TfLpzhGer85CKJ2fZpvLN8