Skip to content

feat(benchmark): add frozen P1 preparation port - #530

Merged
zengchuang-hw merged 5 commits into
mainfrom
codex/benchmark-430-production-prepare
Nov 20, 2026
Merged

zengchuang-hw merged 5 commits into
mainfrom
codex/benchmark-430-production-prepare

Conversation

@zengchuang-hw

@zengchuang-hw zengchuang-hw commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Governing Issue and delivery slice

Refs #430
Refs #371
Refs #434
Refs #490

实现 D023 / t108-v4 首个 P1 家族的 Benchmark 准备端口:冻结已配置的 task/rules、核对实际 worker/resource 字节,向 Apps 交付 PreparedEvaluation、规范协议字节和按 role 读取的资源闭包。

限定交付与作者授权明确允许 Apps 作者承担此 PR;执行记录。作者为 zengchuang-hw / Codex;Dong1017 保留 #430/#434 的交付责任与接口 owner,不涉及 S3 整体 ownership。

Base 493416520f537590eb8f2a4e74f276f82e488307,head 0a82e7b5ad7115f1b2e81818bcf227129208e4ec,5 个文件。此 PR 是 #430 的准备/编码/校验切片;#429 execution 和 #434 assess 分别推进,#528 的 Apps 目录合入顺序保持原计划。

Changed artifacts and evidence

  • 新增 benchmark.preparation.PreparationService.prepare:只接受已授权配置的 p1-fixture@1 / p1-rules@1 / evaluate / p1;PrepareRequest 无法替换冻结的 task rules、worker 或文件读取权限。实际 code resource 的固定摘要必须与 recipe.code_sha256 一致,prepare 时逐项验证大小及 SHA-256。
  • 新增纯 encode / decode / validate_prepared 和明确类型位置的两级身份投影;外层 protocol/resource 定位不影响 prepared_id,内层协议定位改变字节摘要及 prepared_id,ref-shaped Parameters 的位置仍是语义。
  • 完整资源闭包包括 resources 与 baseline/reference/quality-reference,按唯一 role 排列;资源 opener 由可信宿主注入,二进制流由读者关闭。返回的闭包能重新打开资源,Apps 独立验证并分配最终定位、归档后发布 Ref。
  • B 自有 PreparationError 返回便携 PublicError wire;已声明的缺失、摘要不符、未知版本和非法请求结构化拒绝。宿主 capability 的编程异常继续传播,不伪装成端口拒绝。
  • 新增 real Apps T36、跨进程归档重读、两层重定位、caller mutation、冻结规则、资源消失/损坏和 closed-shape 回归;补 Python operation 文档。Benchmark 不导入 Apps/Harness,未修改既有消费者或冻结 fixtures。

Checks and dependencies

Windows / Python 3.12。完整 229 项、schema 和独立发行物检查实际针对初始提交 8980ff6e;当前 3a19e5ae 将 expected_candidate 校验前移到资源读取前,同一校验 helper 继续被纯 codec 复用,新增两项零读取拒绝回归。当前增量相关 Benchmark / Apps prepare 共 51 passed,全仓 Ruff / whitespace 通过。随后 73d1e446 将错误的 text-stream capability 作为宿主 TypeError 传播并验证临时 source-location 别名按 role 归档;当前相关端口 53 passed,全仓 Ruff / whitespace 通过。a1d441ae 进一步明确 stream 所有权、pin 可改动的临时位置,以及 encode 本身拒绝非法形状/路径的独立断言;53 项相关回归再次通过。完整 CI 在 a1d441ae 发现唯一失败:新增测试未列入两个 canonical Apps 模块的 consumers(Python 3.10:1 failed、5553 passed、31 skipped、672 subtests)。当前 0a82e7b5 已补登记,连同迁移检查的本地回归 78 passed;没有放宽或跳过检查。当前 0a82e7b5 的 Ready run 44659195558 必需检查全部成功:full-check 3.10/3.11 各 5615 passed、31 skipped、672 subtests passed(各保留 1 条现有 zip warning);两项安装、浏览器与引用检查成功,两宿主 advisory 也成功,knowledge-model-policy 为 success。先前失败记录保留,原内容验证记录单列:

检查 实际结果
新 prepare、既有 Apps prepare、模块边界、import gate、永久隔离策略及 contracts 包 229 passed、330 subtests passed
全仓 Ruff / whitespace 通过
选定协议库的 PrepareRequest / ProtocolV1 / PreparedEvaluation 定义 三类实际对象通过 jsonschema 验证;显式选择各自 $defs
独立 sdist → wheel → 新 venv,checkout 外以 -I 执行安装检查 通过;930 个资源、128 条记录;资源摘要 f4fe59130727ef9f80d42ef6e416aaea10ca27b0a56b93339c72e82ae77f1879
新 wheel 中 import preparation 公共端口 通过

主要节点:tests/test_benchmark_preparation.py、tests/test_apps_prepare.py、tests/test_module_boundaries.py、tests/test_target_module_import_gate.py、tests/test_first_wave_isolation_policy.py、tests/test_contracts_package.py。使用 python -m pytest -o addopts= -q ... --tb=short。未修改 CI lanes 或 numeric gates。

Limits and acceptance

可信宿主必须提供已审定的 P1 ProtocolV1 和实际 worker/resource capability;本 PR 不自动选择历史文档 fixture 或从 PrepareRequest 获取 task 定义。新端口会冻结/校验配置,不会安装或注册该 worker 到 Harness target,也不执行它。历史 golden 的代码摘要仅作兼容编码对照,不表示该可执行文件已经可用。现有 packaged engineering-task registry 不变。首个任务的配置说明见 docs/contracts/production-prepare.md。

本 PR 验证真实 B prepare 代码与 Apps 归档消费方的 T36;T37/T40 仍依赖 #429,T35/T39/V23 离线判定依赖 #434。三端口齐备后再跑共享 CPU 生产联调。未声明模型/服务收益,#430 和 #371 的整体接受另行确认,使用非关闭关联。

保持 Ready。Dong1017 作为本切片交付责任人进行技术/接口检视;依照仓库独立性规则,其意见不计本 PR 的独立审批票。独立正式批准应来自两名非作者、非责任 parent 的 write+ 评审(请求 david6666666 和 amy-why-3459),首个非空行 LGTM。必需 CI 与两份有效批准齐备后再集成;host-compatibility 按 #508 为 advisory。

Agent-assisted-by: Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Dong1017
Dong1017 previously approved these changes Nov 20, 2026

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 4934165 and head 8980ff6. This is the first review of the full diff, which was seeded complete. There is no prior formal review and no outstanding CHANGES_REQUESTED.

Verified by reading the diff (preparation.py, tests/test_benchmark_preparation.py, docs/contracts/production-prepare.md, docs/operations.md):

  • Request boundary: requests are closed-shape and version/task/operation/scope checked. comparison_conditions must be {}. Request pins cannot override a frozen role's hash, format or required use (IdentityMismatch). Pins for roles that exist only as baseline/reference/quality-reference are refused.
  • Frozen rules: worker recipe, candidate contract, evidence rules and claim registry are checked strictly against the accepted P1 revisions. Unknown revisions give UnsupportedRuleVersion. The constructor also requires that the worker_role resource digest equals worker_recipe.code_sha256.
  • Identity:
    • prepared_id excludes itself, created_at, display_notes and the outer protocol and resource locations.
    • protocol_digest projects locations only on declared ArtifactRefs, so ref-shaped Parameters stay semantic.
    • Both are covered by tests, including relocation, an internal-location move and a Parameters change.
  • Pure codecs: encode, decode and validate_prepared read no artifacts. They require canonical bytes and recheck the protocol/prepared binding, including the sha256, size and media type of the protocol ref.
  • Failure mapping:
    • Declared rejections return a PublicError wire dict at stage prepare.
    • A missing resource gives ArtifactUnavailable. A byte or metadata mismatch gives ArtifactIntegrityError.
    • Host-capability programming errors are not swallowed (tested).
    • Streams are closed through closing, and the returned reader accepts only roles in the preparation's closure.
  • Isolation: the frozen protocol is held as bytes and decoded per call, so caller mutation cannot change it (tested). Benchmark imports neither Apps nor Harness. Only the tests import Apps.
  • Docs: they state the scope limits clearly. The H evaluation-mode path and production offline assessor stay under #429 and #434, and there is no default backend.

Not verified here:

  • I did not run the tests, and no CI results were seeded.
  • I did not read the governing issue or the #429/#434 context beyond what the PR text states.
  • I did not independently check the inferrsi.apps.api.prepare and archive integration beyond what the tests assert.

Optional follow-ups, not blockers:

  • expected_candidate is validated only after the resources are read. Moving that check earlier would give a faster rejection.
  • The merge ruleset still needs a second qualifying approval. This Approve is one vote.

Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

@Dong1017 Dong1017 added the kind:task Bounded delivery task with a delegated slice label Nov 20, 2026
…urces

Agent-assisted-by: Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>

Copy link
Copy Markdown
Contributor Author

已处理候选 metadata 校验时机建议:当前 head 3a19e5ae9dd718d3d2f4538a150bebd4242256c2 在任何 resource opener 调用前拒绝非法 expected_candidate,增加缺字段及格式不符时零读取的回归;相关 Benchmark/Apps prepare 51 passed,全仓 Ruff 和 whitespace 通过。新 head 的 CI 已触发,原 229 项和独立安装结果保留实际版本,未将旧 CI 作为当前结果。

审批角色说明:本 PR 的 限定交付责任人是 Dong1017。依照 独立检视规则,责任 parent 的技术审批不计独立票。因此 Dong 的 PASS 是技术检视结论,本 PR 当前有效独立审批仍为 0/2,需要 david6666666 和 amy-why-3459(或其他符合独立条件的 write+ 评审)分别提交正式结论。此角色限制针对 #430;Dong 在 #528 是独立评审,原有有效票不受影响。

@Dong1017 请复核当前增量并继续确认配置式 P1 prepare 的接口;@david6666666 @amy-why-3459 请独立检视。首个非空行单独写 LGTM 的正式 Approve 才计入有效审批,满足两份和必需 CI 后再集成。

Dong1017
Dong1017 previously approved these changes Nov 20, 2026

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 4934165 and head 3a19e5a (seeded diff complete, 4 files).

Role note: this is the accountable-parent technical/interface verdict for #430, posted by Dong-bot (@Dong1017's authorized delegate). Per docs/agent-collaboration.md a parent's review — including one submitted through this account — is not an eligible independent approval, so the PR's eligible count remains 0/2; david6666666 and amy-why-3459 still need to approve this head. This review does not substitute for either, and I am not asserting merge readiness.

Verified by reading src/inferrsi/benchmark/preparation.py, tests/test_benchmark_preparation.py, docs/contracts/production-prepare.md and docs/operations.md:

  • Current increment. _prepare now validates expected_candidate via _expectation(...) before the closure loop that calls self._resources(...), so a malformed or wrong-contract expectation is refused with zero opener calls; test_request_rejections_are_wire_errors_before_any_archive asserts opened == [] and an empty archive dir for all seven cases, including the missing-field and wrong-format_schema expectation. The seeded patch is base..head, so I read current file state rather than an isolated 8980ff6..3a19e5a hunk; the ordering above is what I checked in code.
  • Request boundary. Closed shape (_REQUEST), schema_version/task/operation/scope checked against the frozen protocol, comparison_conditions must be {}, task_parameters re-validated by _parameters and then copied into the frozen protocol before _protocol is re-applied. A pin for an existing direct role must match on projected fields (IdentityMismatch); a pin for a role that exists only as baseline/reference/quality-reference is refused because only direct may be extended.
  • Frozen rules and worker binding. _recipe pins recipe/entrypoint revisions, args, input roles, capabilities and the single required grader output (UnsupportedRuleVersion); _requirements and the claim registry are checked strictly; PreparationService.__init__ requires worker_role to be in the closure and its artifact.sha256 to equal worker_recipe.code_sha256, then freezes canonical_encode(protocol) so later caller mutation of the dict cannot leak in (covered by test_real_prepare_freezes_exact_supplied_bytes_and_isolates_caller_mutations).
  • Identity, both layers. prepared_id drops prepared_id/created_at/display_notes, the outer protocol.location and projected resources[].artifact.location; protocol_digest drops display_notes and locations only on declared ArtifactRef positions. test_two_layer_locations_and_ref_shaped_parameters covers outer relocation of protocol and resources[].artifact (id unchanged, encode bytes still differ), an internal location change (protocol.sha256 and prepared_id change, protocol_digest equal), and a non-location field (label) changing all three.
  • Opener lifecycle. _measure wraps the stream in contextlib.closing and consumes it in 1 MiB chunks, so the reader owns closing on both the normal and the mid-read failure path; the T36 test asserts all(stream.closed for stream in opened) after prepare. The returned read_resource closure refuses roles outside that preparation's closure with KeyError.
  • Failure boundary. PreparationError.to_json() emits the portable PublicError wire with stage: "prepare" and empty retained_refs/latest_record_ref; Benchmark writes nothing, so a prepare failure cannot publish a closure. Missing/None resources and OSError map to ArtifactUnavailable; byte mismatch maps to ArtifactIntegrityError. test_t36_resource_loss_at_each_boundary covers loss at both prepare and prepared_archive with prepared_ref is None and an empty archive, and test_source_change_after_preparation_is_rejected_by_independent_apps_verification covers post-prepare byte change being rejected by Apps' verifier.
  • Apps cross-check is the canonical one. The test imports inferrsi.apps.handoff.bridge.verify_protocol (not the apps.bridge shim) and calls it on real prepare output, plus a -I subprocess readback that re-validates archived prepared/protocol/resource bytes against the ref digests.
  • #430 pending item is now closed. The wiring I asked for in the 2026-11-20 scope-acceptance comment (D023 / t108-v4 must be explicitly named) is present: docs/contracts/production-prepare.md opens on "the accepted D023 / t108-v4 task family", and the tests read docs/contracts/fixtures/t108-v4. The limit is still stated correctly — the historical golden digest is a compatibility vector, not an available executable.

Non-blocking observations (no correction required for this head):

  1. _measure's _require(isinstance(chunk, bytes), ...) defaults to code InvalidRequest, so a host opener returning a text stream is reported as a request rejection rather than surfacing as a host capability bug, which sits slightly off the stated rule that host programming errors propagate. It still fails closed; test_programming_error_in_a_host_capability_is_not_a_port_rejection covers raising capabilities but not a non-bytes return. A distinct code or propagation plus one regression line would close the gap.
  2. Pins are matched to frozen direct roles on projected fields, so location is request-settable for a frozen role by design. If Apps keys archived artifacts by logical location, one pin could point at another role's path; a one-line uniqueness assertion (or an explicit note that Apps de-duplicates) would remove the ambiguity.

Not verified here: I ran no tests — the 229/51 numbers and the Ruff/whitespace results are the author's, and required CI on 3a19e5ae is still in flight per the author's note, so nothing here should be read as green CI. I did not re-read Apps' archive internals beyond the call sites and cross-test. Merge still needs two eligible independent approvals on this head, required CI, and host-compatibility treated as advisory per #508. I'll re-read on the next @Dong1017 mention or a further DO NOT PASS cycle.


Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

Agent-assisted-by: Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>

Copy link
Copy Markdown
Contributor Author

已处理本轮两项边界建议,当前 head 73d1e446e3a877d801c7ff1377a91c3e508b75a4:

  1. 宿主资源 capability 返回 text stream 时,现抛出 TypeError 并关闭 stream,不再标成请求的 InvalidRequest;增加回归,保留编程错误传播边界。
  2. 源位置允许别名,实际字节权限按 role 读取。Apps storage.archive.archive_prepared 为排序后的每个 role 分配独立 resource-{index} 最终路径,并逐项验证字节,不按临时 source location 查找或去重。新回归让两个不同内容的 role 具有同一协议内 source location,实际 Apps 归档后得到两个不同固定路径,重读字节分别正确;文档已明确这项语义,因此没有引入额外的 source-location 唯一性限制。

当前相关 Benchmark/Apps prepare 53 passed,全仓 Ruff / whitespace 通过。原 229/330 和独立安装结果保留初始版本,当前新 CI 等待完成。@Dong1017 请复核增量;仍按责任 parent 技术意见记录,独立审批当前 0/2。

@david6666666 @amy-why-3459 请先完成可直接推进合入的 #528 第二份有效审批,再继续独立检视本 PR。

Dong1017
Dong1017 previously approved these changes Nov 20, 2026

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS

Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head 73d1e446e3a877d801c7ff1377a91c3e508b75a4 (seeded diff complete, 4 files). Increment since the last head I reviewed: 3a19e5ae9dd718d3d2f4538a150bebd4242256c2..73d1e446. The seeded patch is base..head, so I read current file state (src/inferrsi/benchmark/preparation.py, tests/test_benchmark_preparation.py, docs/contracts/production-prepare.md, docs/operations.md) rather than an isolated hunk.

Role note: this is the accountable-parent technical/interface verdict for #430, posted by Dong-bot (@Dong1017's authorized delegate). Per docs/agent-collaboration.md a parent's review — including one submitted through this account — is not an eligible independent approval, so #530's eligible count remains 0/2; david6666666 and amy-why-3459 still need to approve this head. This review does not substitute for either, and I am not asserting merge readiness: required CI plus two eligible approvals still gate integration.

The two items from the last round are resolved in code, not only in the tests

  1. Host text-stream capability is no longer a port rejection. _measure raises TypeError("A host resource capability must return binary bytes.") on a non-bytes chunk, and the surrounding with closing(source) closes the handle as the exception unwinds. _prepare catches only (KeyError, OSError) around the read loop and prepare catches only PreparationError, so that TypeError propagates as a host programming error instead of being reshaped into InvalidRequest or ArtifactUnavailable. test_text_stream_is_a_closed_host_capability_error asserts the exception type/message and stream.closed.
  2. Role, not provisional location, is the byte authority. _resource/_closure never require distinct locations; uniqueness is enforced on role only (closure plus roles == sorted(set(roles)) in _prepared). A pin for an existing role is accepted when _project_resource(pin) == _project_resource(direct[role]), which compares artifact.{sha256,size_bytes,media_type}, format_schema and required_for but not location. Bytes are opened and measured per role (self._resources(resource["role"])) against that role's own metadata, and the returned read_resource is keyed by role. test_provisional_locations_do_not_replace_role_authority_or_archive_paths repairs the case at the real Apps boundary: archive_prepared gives every sorted role its own resource-{index} path, both readback byte values match the per-role sources, and the archived protocol still shows a single aliased location — the alias survives in the frozen protocol without affecting byte authority or archive paths.

Earlier findings: resolved, no regressions observed

Candidate expectation is still validated before any opener call (_expectation precedes the closure read loop), so the zero-read rejection cases keep holding; the request stays closed-shape; frozen rule versions are refused at PreparationService.__init__ (test_frozen_task_rules_are_not_request_parameters); identity projections, pure codecs and the IdentityMismatch self-check are unchanged in substance.

#430 acceptance points re-checked in this diff

  • Two-layer relocation, including outer resources[].artifact (Amy, 6856558760): test_two_layer_locations_and_ref_shaped_parameters relocates the outer protocol ref and every outer resource artifact and asserts prepared_id unchanged; a location change inside the protocol keeps protocol_digest, changes protocol.sha256/size_bytes and therefore prepared_id; ref-shaped Parameters stay semantic.
  • Canonical cross-test: the test imports inferrsi.apps.handoff.bridge.verify_protocol (not the apps.bridge shim) and runs it on real prepared output.
  • Both unavailability stages: ArtifactUnavailable at prepare and at prepared_archive, prepared_ref is None and an empty archive directory in both.
  • Opener lifecycle: the fixture records every opened stream; T36 asserts all are closed after prepare, and closing covers the error path.
  • #430's pending D023 hook is closed at this head: docs/contracts/production-prepare.md names the accepted D023 / t108-v4 family (p1-fixture@1, p1-rules@1, evaluate, p1), and the T36 fixture derives its protocol from docs/contracts/fixtures/t108-v4/protocol.json while binding actual code bytes, with the historical golden digest retained only as an encoding-compatibility reference. That matches the accepted scope and the body's own "不以历史 golden 摘要表示可执行文件可用" disclosure.

Boundaries re-checked: preparation.py imports stdlib plus contracts.json_v1 only — no Apps, no Harness; the request cannot supply task rules, worker or opener; input_pins cannot replace a frozen role's hash, format or required use, and a pin for a role that exists only as baseline/reference/quality-reference is refused (role in direct fails); comparison_conditions must be {}; unknown protocol/task/rule versions are refused. Only two docs files and the two new files change — no existing consumer, CI lane or frozen fixture is modified.

Not verified here

I did not run pytest, Ruff or the sdist → wheel isolation check, and I cannot see CI status for 73d1e446. The 229 passed / 330 subtests and isolated-install results in the body are explicitly version-pinned to 8980ff6e; the increment's stated coverage is 53 relevant Benchmark/Apps prepare tests plus repo-wide Ruff/whitespace. Independent approvals are 0/2 on this head.

Non-blocking follow-ups

  1. PreparationService's docstring says the caller owns closing each opener stream, but _measure closes the handle on the non-binary path. Closing is the right behaviour (no leak) — please align the docstring/docs sentence with it rather than change the code.
  2. test_closed_prepared_shape_rejected_even_with_recomputed_identity assigns prepared_id inside pytest.raises, so a rejection raised by prepared_id() itself would also satisfy the test; asserting the specific rejection for at least the encode call would keep the closed-shape intent explicit.
  3. The new doc paragraph covers location aliasing, but not that an input_pins entry may rewrite a frozen role's declared location (every other projected field must match). Since the archived protocol keeps that location, one sentence would make the request-visible surface explicit.

I'll re-read on the next push or on an explicit mention; a later DO NOT PASS cycle also brings me back.


Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

Agent-assisted-by: Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>

Copy link
Copy Markdown
Contributor Author

当前 head a1d441ae8b758aa603b684b23b3222e23c64c750 已处理最新三项非阻断建议:说明 prepare 验证阶段由服务关闭 stream、交付闭包后由消费者关闭;明确 input_pins 可改临时 source location 但不能改其他固定字段;将身份重算移出 raises,使闭合形状必须由 encode 本身拒绝,并为非法位置单列 encode 断言。相关端口 53 passed,全仓 Ruff / whitespace 通过。@Dong1017 请复核当前说明/测试增量;责任 parent 技术意见仍不计独立票。必需 CI 重新运行,等待 david6666666 / amy-why-3459 的独立检视。

Dong1017
Dong1017 previously approved these changes Nov 20, 2026

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS

Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head a1d441ae8b758aa603b684b23b3222e23c64c750 (seeded diff complete, 4 files: src/inferrsi/benchmark/preparation.py, tests/test_benchmark_preparation.py, docs/contracts/production-prepare.md, docs/operations.md). Increment since the last head I reviewed: 73d1e446e3a877d801c7ff1377a91c3e508b75a4..a1d441ae. The patch is base..head, so I checked current file state rather than an isolated hunk. No formal APPROVED review exists on this head and there is no outstanding CHANGES_REQUESTED; my earlier PASS reviews sit on 8980ff6e / 3a19e5ae / 73d1e446 and this one supersedes them for the current head.

Role note: this is the accountable-parent technical/interface verdict for #430, posted by Dong-bot (@Dong1017's authorized delegate). Per docs/agent-collaboration.md and the #430 assignment record (6889156052), a parent's review — including one submitted through this account — is not an eligible independent approval. #530's eligible count remains 0/2; david6666666 and amy-why-3459 still need to approve this head. This is not an assertion of merge readiness.

The three items from the last round are resolved, and in code rather than only in prose

  1. Stream ownership is now stated where each side acts. _measure keeps with closing(source) around the read loop, so verification streams close as it returns or unwinds; the returned read_resource hands the host capability's stream straight to the caller and never closes it. docs/contracts/production-prepare.md and the PreparationService docstring both split the two phases explicitly ("Preparation closes each stream it opens… after prepare, the closure's consumer (normally Apps) owns closing each stream it opens"). test_real_prepare_freezes_exact_supplied_bytes_and_isolates_caller_mutations asserts all(stream.closed for stream in opened) and the text-stream test asserts stream.closed after the TypeError.
  2. input_pins may move only a provisional source location. In _prepare, a pin whose role is already in the closure must satisfy role in direct and _project_resource(pin) == _project_resource(direct[role]) — _project_resource drops location and keeps sha256, size_bytes, media_type, format_schema and required_for, so the pin can relocate but cannot re-hash, re-type or drop a use. Pins for roles that exist only as baseline/reference/quality-reference still fail the role in direct conjunct with IdentityMismatch. test_provisional_locations_do_not_replace_role_authority_or_archive_paths shows the intended asymmetry end to end: the frozen protocol keeps one aliased location while the archived closure gets two distinct per-role paths with bytes re-read per role.
  3. Closed shape is rejected by encode itself. encode → _prepared now ends with the prepared_id recompute-and-compare (IdentityMismatch), and _prepared runs the structural checks (_object on _PREPARED, _candidate, _recipe, _requirements, timestamp parse, sorted-unique roles) before that comparison, so a recomputed identity cannot launder a closed-shape violation. test_closed_prepared_shape_rejected_even_with_recomputed_identity recomputes prepared_id first and still gets ValueError for all eight shapes, and test_unsafe_location_is_rejected_by_the_encoder_itself pins the illegal-location case to _artifact's own message instead of relying on the identity mismatch.

Also re-checked on this head

  • Ordering preserved. _expectation(request["expected_candidate"], …) plus _parameters, comparison_conditions == {}, pin validation and _protocol(protocol) all run before the _closure(protocol) read loop, so every rejection is zero-read; test_request_rejections_are_wire_errors_before_any_archive asserts opened == [] and an empty archive dir for all seven cases.
  • Host programming errors stay host errors. Only (KeyError, OSError) are reshaped around _measure and only PreparationError in prepare, so the text-stream TypeError and the injected ValueError / CanonicalError propagate as themselves; None from the capability is FileNotFoundError (an OSError) → ArtifactUnavailable at prepare, matching the docs.
  • Frozen configuration is decoupled from the caller's dict. self._frozen = canonical_encode(protocol) at construction and a fresh canonical_decode per call, so the fixture's post-construction protocol["task_parameters"]["label"] mutation cannot leak into the prepared bytes.
  • Validation hardening. _requirements / _protocol check key in expected before pop, so a duplicated check or claim id yields UnsupportedRuleVersion instead of a bare KeyError; recipe["outputs"] == [{…}] short-circuits before the type(…["required"]) is bool probe, so the output check cannot index into an empty or wrong-typed list.
  • D023 / t108-v4 hook (the #430 condition). The T36 fixture decodes docs/contracts/fixtures/t108-v4/protocol.json and binds real code bytes through the resource capability, and docs/contracts/production-prepare.md now names "the accepted D023 / t108-v4 task family: p1-fixture@1, p1-rules@1" as the port's target. That is the explicit wiring #430 asked for, and test_frozen_documentary_ids_and_pure_codec_remain_compatible keeps the historical vectors byte-identical.
  • Cross-test target. verify_protocol is imported from the canonical inferrsi.apps.handoff.bridge, not the apps.bridge shim.

Not verified here / non-blocking

  • I did not execute the suite, Ruff or CI. The 53-relevant-test and 229-passed/330-subtests figures are the author's; the required CI on this head is re-running and gates integration.
  • Optional follow-up, not a blocker: a host capability that itself raises benchmark.PreparationError would be absorbed by prepare's handler and returned as a declared wire rejection with the host's code string, since PreparationError is a ValueError and is not in the (KeyError, OSError) tuple. Only reachable if a host imports this module's error type to signal its own bug; the documented host-error contract is otherwise intact.
  • Eligible independent approvals remain 0/2 on this head; the follow-on #429 execution and #434 assess ports are out of scope for this slice.

I will re-read after the next push if anything changes.


Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

Copy link
Copy Markdown
Contributor Author

当前 a1d441a 的技术 PASS 及三项建议已清除记录收到。对最后的可选观察补充类型语义:PreparationError 是 B 显式的端口拒绝类型;capability 主动使用它,就表示向 prepare 声明该拒绝语义。一般编程异常(包括 ValueError、TypeError、CanonicalError)继续原样传播,现有回归已覆盖。调用方误用 B 的拒绝类型不通过猜测调用意图区分;Apps 的公开错误校验仍会拒绝非法 wire code。保持当前实现与已记录接口契约。

本 head 的独立有效审批仍为 0/2,等待非责任 parent 的 write+ 评审;CI 在运行。

Agent-assisted-by: Codex (author)
Signed-off-by: zengchuang <zengchuang3@huawei.com>

Copy link
Copy Markdown
Contributor Author

已修复当前完整 CI 的调用方登记失败,head 0a82e7b5ad7115f1b2e81818bcf227129208e4ec。新测试导入 canonical apps.handoff.bridge / apps.storage.archive,但我漏将它登记到 docs/apps-module-migrations.json 的对应两项 consumers。已只补两处登记,保留原顺序;未修改或跳过迁移检查。

本地连同 test_apps_module_migrations.py、新 Benchmark prepare 和既有 Apps prepare 78 passed,全仓 Ruff/whitespace 通过。旧 a1d441a 的完整 Python 3.10 CI 实际为 1 failed、5553 passed、31 skipped、672 subtests,唯一失败为此次登记遗漏,保留原记录;新 head CI 已重新触发。该提交不改运行代码或接口,仍待两份独立有效审批,董的技术意见不计独立票。@Dong1017 请复核该登记增量。

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS

Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head 0a82e7b5ad7115f1b2e81818bcf227129208e4ec (seeded diff complete, 5 files). Increment since the last head I reviewed: a1d441ae8b758aa603b684b23b3222e23c64c750..0a82e7b5. The patch is base..head, so I re-read current file state rather than an isolated hunk. No formal APPROVED review exists on this head and there is no outstanding CHANGES_REQUESTED; my earlier PASS reviews sit on 8980ff6e / 3a19e5ae / 73d1e446 / a1d441ae and this one supersedes them for the current head.

Role note: this is the accountable-parent technical/interface verdict for #430, posted by Dong-bot (@Dong1017's authorized delegate). Per docs/agent-collaboration.md and the #430 assignment record (6889156052), a parent's review — including one submitted through this account — is not an eligible independent approval. #530's eligible count remains 0/2; david6666666 and amy-why-3459 still need to approve this head. This is not an assertion of merge readiness: required CI plus two eligible approvals still gate integration.

The increment is the registration fix, and nothing else moved

  • docs/apps-module-migrations.json gains exactly one consumer entry in each of two arrays: tests/test_benchmark_preparation.py inserted after tests/test_apps_replay.py (before tests/test_result_delivery.py) in the first, and appended after tests/test_apps_module_migrations.py (with the trailing comma corrected) in the second. Both positions preserve the existing alphabetical ordering of the surrounding entries, which matches the author's "保留原顺序" claim.
  • The registrations are motivated by real imports in the new test: inferrsi.apps.handoff.bridge.verify_protocol and inferrsi.apps.storage.archive.read_archive_artifact / read_prepared (plus inferrsi.apps.api.prepare, already registered by the earlier test_apps_prepare coverage in the first array).
  • No check was weakened to get green: no change to tests/test_apps_module_migrations.py, no new skip/xfail, no CI lane or gate edit in the diff. src/inferrsi/benchmark/preparation.py and docs/contracts/production-prepare.md are byte-identical in substance to what I reviewed at a1d441ae, so the four previously-cleared findings (stream ownership, pin-limited-to-provisional-location, encode itself rejecting closed-shape/unsafe paths, host TypeError propagation) stay resolved and are untouched by this head.
  • This also closes the one recorded gap from a1d441ae — the author's reported full CI failure (Python 3.10: 1 failed, 5553 passed, 31 skipped, 672 subtests) was the missing registration, not a code defect, and the fix targets that failure directly.

Not verified here

  • The seeded hunks carry three lines of context, so the array keys for the two edited blocks are not visible in pr/diff.patch. I confirmed insertion position, ordering and shape, but the mapping of those two arrays to the apps.handoff.bridge / apps.storage.archive module entries rests on the author's local test_apps_module_migrations.py run rather than my own inspection of docs/apps-module-migrations.json.
  • I did not run any tests. The claimed 78 passed (migration check + new Benchmark prepare + existing Apps prepare) and the Ruff/whitespace pass are the author's results; required CI on 0a82e7b5 was retriggered and I have not seen it green. Green CI plus two eligible independent approvals are still the merge gate.

Optional, non-blocking

  • When CI on this head completes, a one-line confirmation on this thread is enough for me; no further review round is needed unless the head moves.

I will re-read if the author pushes a new head.


Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

Copy link
Copy Markdown
Contributor Author

当前 head 0a82e7b5ad7115f1b2e81818bcf227129208e4ec 的 Ready run 44659195558 已全部成功:full-check 3.10/3.11、installed-package 3.10/3.11、dashboard-browser、pr-references;两宿主 advisory 也成功,knowledge-model-policy 为 success。此前缺 consumers 的 CI 失败已清除,未改运行代码或放宽检查。

@Dong1017 按本轮技术检视的约定补这一行 CI 完成确认;head 保持不变,无待处理代码阻断。技术 PASS 保留,当前独立有效票 0/2。@david6666666 @amy-why-3459 请先完成 #528 所缺的第二份有效审批,再独立检视本 PR;满足各自两份有效 write+ 批准后按顺序集成。

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head 0a82e7b5ad7115f1b2e81818bcf227129208e4ec.

Independent third-party second verify after Dong1017 tip Approve (always-on) on the same head. Scope: #430 frozen P1 Benchmark preparation port; tip increment registers tests/test_benchmark_preparation.py as a handoff.bridge / storage.archive consumer in docs/apps-module-migrations.json (fixes prior Ready full-check failure; no production code change vs a1d441ae…).

Verified:

  • Tip diff is registration-only; preparation contract/code unchanged from prior reviewed substance.
  • Local: tests/test_benchmark_preparation.py + tests/test_apps_module_migrations.py → passed.
  • Ready CI green on tip (full-check 3.10/3.11, declarations, installed-package, dashboard-browser, pr-references).

Eligibility: cursor[bot] collaborator permission is none. This Approve is technical/third-party verify only — not a write+ eligible Approver under docs/agent-collaboration.md, and not a claim that dual tip / merge-ready is satisfied. Merge still needs write+ independent Approves and steward integration (Dong tip note: parent technical verdict is not an eligible independent approval).

@Dong1017

Copy link
Copy Markdown
Collaborator

Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

CI 完成确认记在这一条上,技术 PASS 保留,head 0a82e7b5 未变。

本 head 的增量我按 diff 复核过(base 4934165,5 文件):相对 a1d441ae 只动了 docs/apps-module-migrations.json 两处 consumers 登记 —— tests/test_benchmark_preparation.py 插入第一个数组 tests/test_apps_replay.py 之后、tests/test_result_delivery.py 之前,第二个数组追加在 tests/test_apps_module_migrations.py 之后并修好尾逗号。src/inferrsi/benchmark/preparation.py、tests/test_benchmark_preparation.py、docs/contracts/production-prepare.md、docs/operations.md 与上一轮我读到的内容一致;运行代码、检查口径、CI lanes 都没有改动或放宽。

这正好对应上一轮的失败原因:新测试导入 canonical apps.handoff.bridge / apps.storage.archive,但漏登记 consumers,a1d441ae 的完整 Python 3.10 因此 1 failed / 5553 passed / 31 skipped / 672 subtests。补登记后 Ready run 44659195558 全绿(full-check 3.10/3.11、installed-package 3.10/3.11、dashboard-browser、pr-references;两项宿主 advisory 成功,knowledge-model-policy success),与"未改运行代码或放宽检查"的说法相符。

形式化评审状态不需要再发一次:本 head 0a82e7b5 上已有我的 APPROVED(review 5986539318),prior-reviews.md 里没有未清除的 CHANGES_REQUESTED;head 未变,重复提交同 head 评审没有意义,所以这一轮是确认记录而非新结论。

独立票仍是 0/2,不受这一行影响:#430 是本切片的负责 parent,Dong-bot(授权代理)经本账号提交的技术/接口结论按 docs/agent-collaboration.md 不计独立审批。合入前仍需 @david6666666 与 @amy-why-3459(或其他符合独立条件的 write+ 账号)在 0a82e7b5 上各提交首行单独写 LGTM 的正式 Approve。

未在本环境执行的部分:我没有重跑 CI 或 pytest,CI 结果按你补报的运行记录采信;代码与 diff 是我逐段读的。你们先 #528 再本 PR 的检视顺序由集成方安排,我不对 #528 的当前票数作判断。

@Dong1017 Dong1017 added the area:benchmark Benchmark line: preparation ports, metric contracts, validators, and isolated workers label Nov 20, 2026

@amy-why-3459 amy-why-3459 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head 0a82e7b5ad7115f1b2e81818bcf227129208e4ec.

#430 first P1 Benchmark preparation port. Verified in benchmark/preparation.py and tests/test_benchmark_preparation.py:

  • Requests are closed-shape; pins cannot override frozen resource metadata, and pins for baseline/reference/quality-reference roles are refused.
  • expected_candidate is checked before resource reads; declared rejections return a PublicError wire; host TypeError and other programming errors propagate.
  • prepared_id / protocol_digest drop outer locations, clock and display notes; encode/decode/validate_prepared are pure and do not open artifacts.
  • Benchmark imports only contracts.json_v1 (no Apps/Harness). The current-head increment only registers the new test as a consumer in docs/apps-module-migrations.json.

Accuracy and performance comparisons are not applicable: this port freezes and verifies configured artifacts and does not install, register or execute the worker. No outstanding CHANGES_REQUESTED on this head.

@zengchuang-hw
zengchuang-hw merged commit f0e92e8 into main Nov 20, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:benchmark Benchmark line: preparation ports, metric contracts, validators, and isolated workers kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants