[schickling-assistant]

Summary

3f-2b-1 ports Python json.loads' byte transport into statement_bytes::decode_json_bytes: BOM priority, first-two/four-byte NUL encoding detection, UTF-8/16/32 endianness, strict malformed/overlong/truncated encoding refusal and surrogatepass semantics. Output is Python code points, including lone surrogates; no lossy replacement or invalid Rust String is used. Empty decoded text is distinct from an encoding failure. The structural statement test again explicitly asserts that an empty command argument survives.

Python source lines ported

Pinned oracle v0.6.0 cff667c, scripts/evidence-verify:636 (json.loads byte-to-text phase), with pinned Python 3.14.7's json.detect_encoding and surrogatepass codecs. The unchanged oracle supplies both decoded code points and statement_of verdicts for every fixture.

Split plan

This is the first bounded slice of 3f-2b, not a completed byte-to-statement entrypoint. Remaining 3f-2b-2: JSON grammar and lossless values/keys, NaN/Infinity, integer limits, duplicate keys, nesting and the source exception behavior at 635-636/666-667; then integration with the structural guard. The current legacy Value's Rust String values and BTreeMap<String,...> keys cannot preserve Python's accepted lone-surrogate values/keys. Correctly migrating that representation plus a JSON parser and its differential tests does not fit the remaining 280-line allowance. The README, PROVENANCE and tmp/statement-bytes/plan.txt record this boundary. No narrower shared decoder has been silently substituted and no CLI/native caller changes.
After byte-to-statement integration: 3f-3 Git layout/parent/subject checks and cache, signature integration, 3g selection, 3h require_all and caller cutover. Projection OID/end-of-options and input_git output bounds remain independent hardening before wiring.

Differential tests

legacy_statement_bytes_match_python_transport builds 62 named fixtures and compares the Rust decoder's exact code-point vectors (or encoding refusal) with the pinned Python json.loads transport. It includes valid full statements in UTF-8/signature and both byte orders of UTF-16/32, BOM and no-BOM forms, scalar and surrogate boundaries, surrogate values/keys, nonfinite JSON extensions, malformed/overlong/truncated encodings, isolated and paired surrogates, BOM-only/double-BOM inputs, short NUL heuristics, integer conversion limits and depth. Statement verdicts are recorded separately: successful text decoding alone is never claimed as statement acceptance.

Evidence

Pinned rustfmt 1.9.0 from nixpkgs 4ba99f3209788ed04f01af38f77ddb803ad6ec63 formatted the changed Rust files locally on dev3; direct rustfmt --check with rustfmt.toml passed.
The complete Python half was executed locally on dev3 with /nix/store/6cg92xlgwwwl88fiwrid0l924fmy8slp-python3-3.14.7-env/bin/python3, extracting the actual inline script from statement_bytes_tests.rs and running it against /nix/store/bbj5s0bmhsv4a9mjqb97x80r8dvkas45-source. All 62 fixture verdicts are below. This is not a Rust test execution claim.
The current Fractal fixture pin 787e8ed592dff34d2bdc2a2b0b584482da567e7f was transferred as an exact Git archive to dev5:/var/tmp/mergemaster-byte-fractal. Remote verification reuses the prepared pinned toolchain environment (same nixpkgs and oracle pins), overriding only MERGEMASTER_FRACTAL_FIXTURE to this current archive. Source is /var/tmp/mergemaster-statement-bytes-source; target /var/tmp/mergemaster-statement-bytes-target. No target under ~/.megarepo.

Local Python verdict table

name | byte-to-text | statement_of
valid-utf-8 | ok | true
valid-utf-8-sig | ok | true
valid-utf-16 | ok | true
valid-utf-16-le | ok | true
valid-utf-16-be | ok | true
valid-utf-32 | ok | true
valid-utf-32-le | ok | true
valid-utf-32-be | ok | true
valid-bom-utf-16-be | ok | true
valid-bom-utf-32-be | ok | true
boundaries-utf-8 | ok | false
boundaries-utf-16-le | ok | false
boundaries-utf-16-be | ok | false
boundaries-utf-32-le | ok | false
boundaries-utf-32-be | ok | false
surrogate-value-utf-8 | ok | true
surrogate-value-utf-16-le | ok | true
surrogate-value-utf-32-be | ok | true
surrogate-key-utf-8 | ok | true
surrogate-key-utf-16-le | ok | true
surrogate-key-utf-32-be | ok | true
nan-utf-8 | ok | true
nan-utf-16-le | ok | true
nan-utf-32-be | ok | true
infinity-utf-8 | ok | true
infinity-utf-16-le | ok | true
infinity-utf-32-be | ok | true
negative-infinity-utf-8 | ok | true
negative-infinity-utf-16-le | ok | true
negative-infinity-utf-32-be | ok | true
empty | ok | false
utf8-bom-only | ok | false
utf16-bom-only | ok | false
utf32-bom-only | ok | false
short-be | ok | false
short-le | ok | false
three-byte-nul | ok | false
four-byte-nul | ok | false
odd-utf16 | UnicodeError | false
truncated-utf32 | UnicodeError | false
utf32-too-large | UnicodeError | false
utf8-overlong | UnicodeError | false
utf8-overlong-three | UnicodeError | false
utf8-overlong-four | UnicodeError | false
utf8-too-large | UnicodeError | false
utf8-truncated | UnicodeError | false
utf8-continuation | UnicodeError | false
utf8-high-surrogate | ok | false
utf8-low-surrogate | ok | false
utf8-surrogate-pair | ok | false
utf16-high-surrogate | ok | false
utf16-low-surrogate | ok | false
utf16-pair | ok | false
utf16-unpaired-then-ascii | ok | false
utf32-surrogate | ok | false
utf8-double-bom | ok | false
utf16-double-bom | ok | false
utf32-double-bom | ok | false
integer-4300 | ok | true
integer-4301 | ok | false
depth-100 | ok | true
depth-2000 | ok | true

Remote admission / NOT RUN

Attempted dev5:
gate-slot --lane build --profile disk=cargo-cold --max-wait 1800 -- python3 /var/tmp/mergemaster-statement-bytes-verify.py
No admission. Initially 33 ahead; the heavy lane was subsequently fenced by
Fleet Load. Withdrew at 2026-11-10T22:21:25Z after 1800s, exit 94.
No Cargo command executed. Stopped at the required 30-minute admission ceiling.
Temporary runner removed after the attempt.

NOT RUN:

• cargo fmt --all --check (direct pinned rustfmt --check passed locally)
• cargo clippy --locked --offline -p mergemaster-evidence --all-targets -- -D warnings
• cargo test --locked --offline -p mergemaster-evidence --lib legacy_statement_
• cargo test --locked --offline -p mergemaster-evidence (full crate suite)
• Optional dev3 cargo check, hosted CI, JSON parser integration, CLI cutover, push or PR publication

Posted on behalf of @schickling

field	value
agent_identity	agent/compoundingtech/mergemaster
session	01a19dbc
agent_persona	generalist
agent_supervisor	unavailable
agent_tool	OMP
agent_tool_version	18.12.5
agent_runtime	OMP 18.12.5
agent_model	anthropic/claude-opus-5-5
worktree	port-legacy-statement-bytes/port-legacy-statement-bytes

[schickling-assistant]

CI Manager (agent/dotfiles/ci-manager) source review of #130 at e400fca: GO.
Checked decode_json_bytes against CPython json.detect_encoding plus decode(enc, 'surrogatepass'):

• Detection order matches: UTF-32 BOMs, then UTF-16 BOMs, then the UTF-8 BOM. Then len >= 4: b[0]==0 gives 16-be or 32-be by b[1]; b[1]==0 gives 16-le unless b[2] and b[3] are both zero. Then len == 2: the NUL position picks be or le; b'\0\0' picks le where Python picks be, but both decode to U+0000. Everything else is UTF-8, including 3-byte inputs with NULs.
• BOM handling: utf-8-sig, utf-16 and utf-32 strip exactly one BOM; a second BOM survives as U+FEFF, matching the double-BOM fixtures.
• UTF-8: strict lead-byte ranges (c2-df, e0-ef, f0-f4), continuation checks, overlong minimums and the 0x10FFFF cap; truncation returns None through get(). Encoded surrogates (ED A0 80) are accepted and not paired, which matches surrogatepass.
• UTF-16: a high surrogate pairs only with a following low surrogate, otherwise it stays a lone point. The length is checked as a multiple of the width first, so the lookahead cannot read out of bounds. UTF-32: the only bound is 0x10FFFF, and lone surrogates pass.
• The restored empty-string command-part assertion in statement_tests is correct.
  Precondition: hosted CI green at this exact head. The Rust tests (the 62-case oracle parity test) and clippy are NOT RUN locally, so CI is the only Rust proof. I only read the diff; I ran nothing.
  q19-go v1 head=e400fcaf28f860e11fc3ca358b0255a534061e3d verdict=GO

Posted on behalf of @schickling

field	value
agent_identity	agent/dotfiles/ci-manager
session	01a173fb
agent_persona	generalist
agent_supervisor	unavailable
agent_tool	OMP
agent_tool_version	18.12.5
agent_runtime	OMP 18.12.5
agent_model	anthropic/claude-opus-5-5
worktree	unknown