W5-A: plan-driven prompt rendering, bounded codebook search, one JSON serializer, nonce delimiter - #38
Merged
Conversation
Findings 39, 1, 44 (GPT Pro round 2):
- 39: jevmlx/json_text.py — one canonical JSON serializer
(ensure_ascii=False) for every prompt and candidate path; schema.py's
labels candidates used the stdlib default (escaped '\u00e9') while the
prompt showed real characters. Engine re-exports it; openai_slots uses
it for rows/keys.
- 1: to_schema_str('slots', tokenizer=...) renders the DISPLAYED aliases
from the compiled slot plan (p['fields'][name]['aliases']) — the
independent _alias_code(i) rendering path is gone from prompts, so
prompt and scorer cannot disagree (the search picking digits is now
visible in the prompt). to_alias_schema_str takes the tokenizer;
alias_for_index remains only for the OpenAI adapter's per-field rows
(documented). Engine _prefill, openai_slots (threaded tokenizer through
decide_openai/_scalar_messages/_decide_multi_field/_user_content), cli.
- 44: _context_block/_context_nonce — both fences carry a sha256-derived
nonce absent from the context (a context containing 'CONTEXT>>>' can no
longer close the block); used by the naive path, the parallel prefill,
and the OpenAI adapter.
- PROMPT_VERSION v7 -> v8 (prompt text changed).
Every fix's test runs against 1f9f453 behavior (version pins, delimiter
shape, block-without-tokenizer raises). 548 passed (not slow), ruff clean.
Finding 2: _greedy_pick is now a bounded backtracking search over the
prefix-conflict graph of pre-tokenized candidate rows. Greedy committed to
A, rejected B/C/D, and raised SchemaCompileError even though {B, C, D} was
a valid prefix-free set; the search now explores the backtracking tree
(20k-node cap per pool, tighter for n>16; remaining-feasible prune) and
scores every COMPLETE set with the existing lexicographic objective
(trie nodes, max depth, max candidate tokens, length variance, code
length). Early exit on a single-branch-node set keeps the 26-choice
compile under a second (the C(36,26) tree never runs to its cap);
determinism unaffected (fixed pool priority order).
tests/test_w5a.py: 10 tests covering findings 1, 2, 39, 44 — verified to
FAIL on origin/main 1f9f453 (9 failed there before this branch's schema
fixes; 10/10 after strengthening) and PASS here:
- prompt shows the searched codebook (digits visible when the search
picks digits), plan/prompt alias sets identical, tokenizer required
- A-prefix-of-B/C/D backtracks to {B, C, D} and compiles
- 'é' verbatim in serializer, block, and candidates (no \u00e9)
- fake CONTEXT>>> cannot close the nonce-delimited block
- PROMPT_VERSION v8
558 passed (not slow), ruff clean.
This was referenced Sep 18, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes GPT Pro round-2 findings 1, 2, 39, 44 (scope per W5-A: schema.py prompt rendering + codebook, engine _prefill delimiter). Branch off origin/main 1f9f453. PROMPT_VERSION v7 → v8.
Finding 1 — the prompt and the compiled codebook disagreed (P0)
to_schema_str(mode="slots", tokenizer=...)now renders the displayed aliases from the compiled slot plan (plan['fields'][name]['aliases']). The independent_alias_code(i)prompt path is deleted — the tokenizer-free slot rendering raises instead of drifting. When the search picks digits, the prompt now shows0)", 1)"..., so the model is taught the protocol the scorer judges.to_alias_schema_str(tokenizer)takes the tokenizer (engine_prefill,openai_slots— threaded throughdecide_openai/_decide_scalar_field/_decide_multi_field/_user_content, cli).alias_for_indexsurvives only for the OpenAI adapter's per-field rows (documented at the definition).Finding 2 — greedy codebook search rejected satisfiable fields (P0)
_greedy_pick→ bounded backtracking over the prefix-conflict graph of pre-tokenized candidate rows. The failing tokenization (A a token-prefix of B/C/D, mutually prefix-free B/C/D, n=3) no longer raises: the search finds {B, C, D}. Every complete set is scored with the existing lexicographic objective (trie nodes → max depth → max tokens → length variance → code length). Bounds: 20k visited nodes per pool (2k for n>16), remaining-feasible prune, and an early exit when a set reaches the objective's first-component minimum (1 branch node) — keeps the 26-choice compile <1s. Deterministic (fixed pool priority order).Finding 39 — one canonical JSON serializer
New
jevmlx/json_text.py(json_text,ensure_ascii=False) used by every prompt and candidate path: schema.py labels candidates previously used stdlibjson.dumps(escaped\u00e9) while the prompt showed the real character — scorer judged tokenizations the model never saw. All prompt/candidate call sites in schema.py, engine.py, openai_slots.py converted.Finding 44 — ambiguous context delimiter
_context_block(context)/_context_nonce(context): both fences carry a sha256-derived 16-hex nonce of the context (<<<CONTEXT:C<nonce>…CONTEXT:C<nonce>>>), so a context containingCONTEXT>>>can no longer close the block early. Used by the naive path, the parallel prefill, and the OpenAI adapter. Formatting correctness, not a security boundary.Verification
tests/test_w5a.py: 10 tests, all FAIL on 1f9f453 (checked by checking out base), all PASS here — covering each finding's exact failing case.Notes