[Birfy]

Closes #252. Part of #221.

What

Next offline slice of the Google RRSI port (pinned at be50316e1db05914068a973f322770ef08ed7ba1), the evidence the proposer and pruner read:

• examples/rrsi/components.py, adapted from rrsi/components.py: K, K_STR, text_only, classify_diff, has_evidence, normalize, novelty.
• examples/rrsi/history.py, adapted from rrsi/history.py: the per-edit JSONL History (append_candidate, tried, attempted, accepted_edits, yield_g, prune_set, render, replace_round) and stall_flag / exploration.
• selection.py takes STRUCTURAL_COMPONENTS from K_STR instead of its own copy. Same set, so no behaviour change, and a test pins novelty parity with selection._novelty.

Apache-2.0 notices are kept on both adapted files.

Upstream behaviour preserved (and documented in docs/algo-rrsi.md)

• A declared non-structural tag (config, control_flow, ...) survives normalize only if a domain signal backs it, because the generic signals only cover structural components.
• yield_g is -inf for a tried component with nothing inside the prune window, so it is in B_t.
• An empty candidate writes one placeholder record (C1, no component, bundle = 0). stall_flag is 0 until w rounds exist.

Defensive boundaries (AgentDescent convention, not upstream parity)

• Non-finite or non-numeric delta_S / delta_C / S / C raise ValueError. Upstream would write non-standard NaN JSON.
• A torn final line from a crash mid-append is ignored by records() and dropped by the next append. A malformed line anywhere else still raises with its line number.
• replace_round rewrites through a temp file and os.replace.

Tests

tests/test_rrsi_history.py (24 cases) covers bundle fan-out, the placeholder record, measured vs attempted, non-finite rejection, torn-line recovery, mid-file corruption, atomic replace_round, the yield window and B_t with accepted machinery, render's cap on unmeasured records, stall and exploration text, and normalize evidence checks.

pytest -n 8 full suite passes locally (Python 3.13).

Not in scope

The critic, proposer, pruning execution, the loop, the CLI and evolve() wiring all stay on #221. Nothing consumes the history yet.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-14T06:21:42.044402Z	72095f5	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72095f53bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Let text-only diffs override declared structural tags

When a prompt string or comment mentions a generic signal such as SkillRegistry, text_only(diff) and classify_diff(diff) correctly identify it as a prompt edit, but this branch calls has_evidence first and retains a declared skill tag. That lets a prose-only change satisfy a reserved structural exploration slot and corrupt the tried and novelty summaries that normalization is intended to protect; check the text-only case before accepting the declaration.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Decode torn UTF-8 tails only after isolating the final line

If a crash interrupts an ensure_ascii=False append in the middle of a multibyte character, decoding the entire file here raises UnicodeDecodeError before the torn-final-line recovery logic runs. Consequently records() and every summary method fail on a crash state that this implementation promises to ignore, even though the next append() can repair it; split the byte data first and decode the potentially torn tail separately.

Useful? React with 👍 / 👎.

[Birfy]

Automated review: request changes (one small fix)

GitHub doesn't allow a request-changes review on your own PR, so I'm posting it as a comment. Reviewed at 72095f5.

Overall: this is a careful, faithful slice. I diffed components.py and history.py against upstream rrsi at be50316e. The logic is unchanged apart from typing (3.9-compatible), K/K_STR becoming tuples, and the documented defensive boundaries. The Apache-2.0 headers match the existing selection.py and schedule.py convention. Moving selection.STRUCTURAL_COMPONENTS onto K_STR, with a parity test, is the right dedup. The docs and the parity table in algo-rrsi.md read clearly and stay honest about this being an unwired microport. Locally, the three RRSI test files and the full pytest -n 8 suite pass. CI: docs build is green, the test matrix was still pending when I looked.

Blocking: torn-tail recovery fails when the cut lands inside a multibyte UTF-8 character

The PR promises that "a torn final line from a crash mid-append is ignored by records()". But records() decodes the whole file with read_text(encoding="utf-8") before it looks at lines. Records are written with ensure_ascii=False, and hypotheses, details and diffs will often hold non-ASCII text (Chinese, emoji, or arbitrary code). If a crash cuts the tail mid-character, records() raises UnicodeDecodeError instead of skipping the tail. _drop_torn_tail() already handles this case on the append path, so the two paths disagree.

Repro:

h = History(p)
h.append_candidate(1, "v", [{"id": "e1", "component": "skill", "hypothesis": "技能"}],
                   "ACCEPTED", 0.1, 1.0, True, 0.5, 10.0, None)
line = json.dumps({"t": 2, "hypothesis": "中文假设"}, ensure_ascii=False).encode()
open(p, "ab").write(line[: line.index("中".encode()) + 1])   # cut mid-character
h.records()   # UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 ... unexpected end of data
h.append({"t": 3}); h.records()   # works: the append path drops the tail

Suggested fix: read bytes, split on b"\n", decode each line, and treat UnicodeDecodeError like JSONDecodeError. Skip it if it's the unterminated last element, and raise the same ValueError(f"{path}:{lineno}: malformed history record") anywhere else, so mid-file corruption also gets a line number instead of a bare decode error. Please add a regression test next to the existing torn-line cases.

Non-blocking nits

1. replace_round silently changes the file to mode 0600. tempfile.mkstemp creates the temp file as 0600, and os.replace keeps that, so a 0644 history becomes owner-only after the first re-adjudication (verified locally). Consider copying the original mode (os.chmod(tmp, stat.S_IMODE(os.stat(path).st_mode)) when the file exists) before the replace. An optional fh.flush(); os.fsync(fh.fileno()) before os.replace would also make the "atomic" claim hold across power loss, not just process crashes.
2. _finite rejects numpy scalars. np.float32(0.1) raises delta_S must be a number or None, because only int/float are accepted (np.float64 passes, since it subclasses float). Scores often come from numpy. Accepting numbers.Real (still excluding bool) would avoid a surprising failure once the loop is wired up. Fine to defer until something consumes the history.
3. records() can return non-dict JSON values (e.g. a stray [] line), and the summaries would then hit AttributeError on .get. Upstream behaves the same way, so this is optional: maybe raise the same malformed-record ValueError for non-objects.

Once the UTF-8 torn-tail case is handled and tested, this looks good to merge from my side.

[Birfy]

Addressed the review and both Codex findings in 7dee36d (on top of 10c5504, which merges current main and resolves the CHANGELOG conflict with #247):

• Blocking (torn UTF-8 tail). History.records() now reads bytes, splits on b"\n" and decodes per line. A UnicodeDecodeError / JSONDecodeError on the unterminated last element is skipped as a torn append; anywhere else it raises ValueError("<path>:<line>: malformed history record"). Regression: test_a_torn_tail_cut_inside_a_multibyte_character_is_ignored (your repro, plus a following append) and test_invalid_utf8_or_non_object_before_the_last_line_raises.
• Nit 1. replace_round fsyncs the temp file and copies the existing file's mode before os.replace (test_replace_round_keeps_the_file_mode: 0644 survives).
• Nit 2. _finite accepts numbers.Real (still excluding bool), so np.float32 / np.int64 work; NaN is still rejected (test_numpy_scalars_are_accepted_as_measurements, importorskip("numpy")).
• Nit 3. A non-object JSON line ([], 3) is reported as a malformed record instead of hitting AttributeError later.
• Codex P2 (text-only diffs). normalize() now returns prompt for a text_only diff before honouring a declared tag, so prose mentioning SkillRegistry cannot claim a structural slot. This is a deliberate deviation from upstream, documented in the module docstring and the parity table in docs/algo-rrsi.md (test_a_text_only_diff_cannot_keep_a_structural_declaration).

All 7 new tests fail against the previous examples/rrsi code and pass now. The three RRSI suites pass. Full pytest -n 8 passes too, apart from test_era_algotune.py::test_the_reference_scores_about_one_through_the_sandbox, which measured a speedup of 0.04 under 8-way parallel load and passes on its own. It's a timing flake unrelated to this change.

[Birfy]

Automated re-review at 7dee36d: LGTM (posted as a comment because GitHub blocks self-approval)

Verdict: ready to merge. Exact-head CI is green on Python 3.9–3.13 and the docs build, and the branch contains current main via 10c5504, with no conflicts.

• Blocking item resolved. records() splits bytes before decoding, so a tail cut inside a multibyte character is skipped exactly like any other torn line. The original repro passes as test_a_torn_tail_cut_inside_a_multibyte_character_is_ignored, including the following append. Corruption before the last line (invalid UTF-8, invalid JSON, or a non-object line such as [] or 3) raises ValueError with path:line.
• Nits resolved. replace_round keeps the file mode and fsyncs before os.replace. _finite accepts numbers.Real (numpy scalars), still rejects bool and NaN.
• Codex P2 resolved. A text-only diff is now prompt before a declared structural tag is honoured. This is a deliberate, documented deviation from upstream: it's listed in the components.py docstring and the docs/algo-rrsi.md parity table.
• Tests. All 7 new regressions fail against the previous examples/rrsi code. The three RRSI suites pass.

Note for later: #254 adds its own novelty / K usage on top of the same RRSI area. Whichever of #253 and #254 lands second should reconcile with components.normalize's text-only rule.

[Birfy]

Independently verified the reviewed repairs at 7dee36dfb102414ba10ec257b182d6ec0a9f05ff: 79 focused history/schedule/selection tests and 16 additional offline probes pass on Python 3.12.14. All seven new regression cases fail against old 72095f53 for the intended defects.

The additional checks cover every partial-byte cut within a Chinese character and emoji in a writer-generated tail, successful following-append recovery, and line-numbered rejection of earlier invalid UTF-8/JSON/non-object records. Text-only comments containing each structural signal normalize to prompt; actual structural edits remain structural. Rewrite controls preserve 0640/0644 modes, call fsync before replace, and retain the original bytes while cleaning the temporary after injected fsync failure. NumPy scalar tests ran without skipping.

Exact-head tests and docs are green, including Python 3.9–3.13 wheel/install smoke. The normalization change remains an explicitly documented upstream deviation. This is scoped verification of these fixes; #254 integration reconciliation and arbitrary malformed-input behavior are not certified here. No provider calls or formal approval.

[Birfy]

Merged current main (includes #248 trajectory ledger) into this branch with a clean auto-merge of CHANGELOG.md only. No conflict resolution beyond the merge commit. Local suite was green on the merge head before push; CI should re-run on the new head.

[Birfy]

Re-check at latest head

Verdict: LGTM (Closes #252)

Confirmed the earlier blocking finding is on the branch: 7dee36d makes History.records() split bytes before UTF-8 decode, so a torn tail cut inside a multibyte character is ignored like any other torn line; mid-file invalid UTF-8 / non-object raises ValueError with a line number. Regression test_a_torn_tail_cut_inside_a_multibyte_character_is_ignored is present.

Re-verified on head 209cdebc (post–main merges):

• pytest tests/test_rrsi_history.py → 31 passed (includes torn / multibyte / invalid-utf8 cases)
• CI: tests 3.9–3.13 + docs build SUCCESS

Ready to merge from our side.

(Posted as COMMENT — author account cannot APPROVE.)