Skip to content

W5-A: plan-driven prompt rendering, bounded codebook search, one JSON serializer, nonce delimiter - #38

Merged
bnsd55 merged 4 commits into
mainfrom
w5a-prompt-plan
Sep 18, 2026
Merged

bnsd55 merged 4 commits into
mainfrom
w5a-prompt-plan

Conversation

@bnsd55

@bnsd55 bnsd55 commented Sep 18, 2026

Copy link
Copy Markdown
Owner

Fixes GPT Pro round-2 findings 1, 2, 39, 44 (scope per W5-A: schema.py prompt rendering + codebook, engine _prefill delimiter). Branch off origin/main 1f9f453. PROMPT_VERSION v7 → v8.

Finding 1 — the prompt and the compiled codebook disagreed (P0)

to_schema_str(mode="slots", tokenizer=...) now renders the displayed aliases from the compiled slot plan (plan['fields'][name]['aliases']). The independent _alias_code(i) prompt path is deleted — the tokenizer-free slot rendering raises instead of drifting. When the search picks digits, the prompt now shows 0)", 1)"..., so the model is taught the protocol the scorer judges. to_alias_schema_str(tokenizer) takes the tokenizer (engine _prefill, openai_slots — threaded through decide_openai/_decide_scalar_field/_decide_multi_field/_user_content, cli). alias_for_index survives only for the OpenAI adapter's per-field rows (documented at the definition).

Finding 2 — greedy codebook search rejected satisfiable fields (P0)

_greedy_pick → bounded backtracking over the prefix-conflict graph of pre-tokenized candidate rows. The failing tokenization (A a token-prefix of B/C/D, mutually prefix-free B/C/D, n=3) no longer raises: the search finds {B, C, D}. Every complete set is scored with the existing lexicographic objective (trie nodes → max depth → max tokens → length variance → code length). Bounds: 20k visited nodes per pool (2k for n>16), remaining-feasible prune, and an early exit when a set reaches the objective's first-component minimum (1 branch node) — keeps the 26-choice compile <1s. Deterministic (fixed pool priority order).

Finding 39 — one canonical JSON serializer

New jevmlx/json_text.py (json_text, ensure_ascii=False) used by every prompt and candidate path: schema.py labels candidates previously used stdlib json.dumps (escaped \u00e9) while the prompt showed the real character — scorer judged tokenizations the model never saw. All prompt/candidate call sites in schema.py, engine.py, openai_slots.py converted.

Finding 44 — ambiguous context delimiter

_context_block(context) / _context_nonce(context): both fences carry a sha256-derived 16-hex nonce of the context (<<<CONTEXT:C<nonce>CONTEXT:C<nonce>>>), so a context containing CONTEXT>>> can no longer close the block early. Used by the naive path, the parallel prefill, and the OpenAI adapter. Formatting correctness, not a security boundary.

Verification

  • tests/test_w5a.py: 10 tests, all FAIL on 1f9f453 (checked by checking out base), all PASS here — covering each finding's exact failing case.
  • Full suite: 558 passed (not-slow), ruff check + format clean.
  • Perf guard kept: 26-choice codebook compiles <1s (existing test updated only where behavior was the bug's).

Notes

  • Findings verified in code before fixing; none of the four was wrong.
  • No fallbacks/dual paths: the old rendering and delimiter are gone, not gated.
  • Out of scope here (W5-B/C per the wave plan): findings 3–13, 18–28, set solver, decide_many parity.

Findings 39, 1, 44 (GPT Pro round 2):

- 39: jevmlx/json_text.py — one canonical JSON serializer
  (ensure_ascii=False) for every prompt and candidate path; schema.py's
  labels candidates used the stdlib default (escaped '\u00e9') while the
  prompt showed real characters. Engine re-exports it; openai_slots uses
  it for rows/keys.
- 1: to_schema_str('slots', tokenizer=...) renders the DISPLAYED aliases
  from the compiled slot plan (p['fields'][name]['aliases']) — the
  independent _alias_code(i) rendering path is gone from prompts, so
  prompt and scorer cannot disagree (the search picking digits is now
  visible in the prompt). to_alias_schema_str takes the tokenizer;
  alias_for_index remains only for the OpenAI adapter's per-field rows
  (documented). Engine _prefill, openai_slots (threaded tokenizer through
  decide_openai/_scalar_messages/_decide_multi_field/_user_content), cli.
- 44: _context_block/_context_nonce — both fences carry a sha256-derived
  nonce absent from the context (a context containing 'CONTEXT>>>' can no
  longer close the block); used by the naive path, the parallel prefill,
  and the OpenAI adapter.
- PROMPT_VERSION v7 -> v8 (prompt text changed).

Every fix's test runs against 1f9f453 behavior (version pins, delimiter
shape, block-without-tokenizer raises). 548 passed (not slow), ruff clean.
Finding 2: _greedy_pick is now a bounded backtracking search over the
prefix-conflict graph of pre-tokenized candidate rows. Greedy committed to
A, rejected B/C/D, and raised SchemaCompileError even though {B, C, D} was
a valid prefix-free set; the search now explores the backtracking tree
(20k-node cap per pool, tighter for n>16; remaining-feasible prune) and
scores every COMPLETE set with the existing lexicographic objective
(trie nodes, max depth, max candidate tokens, length variance, code
length). Early exit on a single-branch-node set keeps the 26-choice
compile under a second (the C(36,26) tree never runs to its cap);
determinism unaffected (fixed pool priority order).

tests/test_w5a.py: 10 tests covering findings 1, 2, 39, 44 — verified to
FAIL on origin/main 1f9f453 (9 failed there before this branch's schema
fixes; 10/10 after strengthening) and PASS here:
- prompt shows the searched codebook (digits visible when the search
  picks digits), plan/prompt alias sets identical, tokenizer required
- A-prefix-of-B/C/D backtracks to {B, C, D} and compiles
- 'é' verbatim in serializer, block, and candidates (no \u00e9)
- fake CONTEXT>>> cannot close the nonce-delimited block
- PROMPT_VERSION v8

558 passed (not slow), ruff clean.
@bnsd55
bnsd55 merged commit b7e3031 into main Sep 18, 2026
2 checks passed
@bnsd55
bnsd55 deleted the w5a-prompt-plan branch September 18, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant