Adopt one verified benchmark acquisition for release reports and artifacts #323

Summary

Target v0.5.3 (existing consumer milestone preserved). Integrate the public candidate-reuse capability from research-repo-tools#179, the committed-output extension in #189, and subsequent README/resume fixes so one Step 1 acquisition supplies report, README and release archive, and Step 2's just release-evidence validates it before tagging without another acquisition.

Implementation prerequisites: research-repo-tools#179/#189 and the follow-up fixes in the reviewed development pin. Final supplying publication and normal resolution belong to #333 (target v0.2.7). #169/#176 and #170 supply the narrower v0.2.5 API/history; that exact-tag-only contract does not satisfy the required pre-tag Step 1 -> final merged Step 2 ordering. Do not treat a version pin, CI-only reuse, post-tag measurement or the existing two-acquisition fallback as completion.

Implementation handoff — 2026-11-07

The maintainer has moved the published-tooling completion criteria to #333. The implementation and bounded local validation are complete in PR #335, commit a2debb424426b75a97e8a56a061b70985f53d950, using the explicitly authorized research-repo-tools development pin 055353c630604b1ae3438c1eaf534ba85e067a7e. This issue remains open until PR #335 merges after required review and CI. The PR's Closes reference owns automatic closure; published-package adoption and release verification remain separate in #333.

Release-only acceptance is retained, unchecked, in #333: verify the exact supplying PyPI release (target v0.2.7), replace every development pin through normal manifest/lock/isolated-invocation resolution, rerun the affected consumer and native platform/feature checks for that adoption revision, and record the final release evidence. No unpublished pin is accepted for an actual release.

Validation and contract evidence: workload review, candidate review, and #335. Full macOS just ci passed (367 Python tests, 1,143 Rust tests, default/exact doctests and the owning static/native checks); bounded rehearsals used synthetic timing samples, not a real campaign. Required PR CI is still separate from that local evidence.

Historical context

The current local comparison measures both revisions and 16 peers, then the tagged hosted release workflow measures all 1,403 current la-stack cases again. These fulfill different evidence/identity contracts. Existing #304/#316 integrations and the v0.5.2 release workflow remain valid until a replacement is verified.

Proposed Changes

• At a bounded Astra architecture checkpoint, choose acquisition ownership and resolve release ordering: local reports/README currently precede the final release commit/tag, while the hosted artifact requires tagged source identity. Do not assume equal source subtrees justify relabeling evidence.
• Map each required guarantee to the owning validator and retained artifact: protected source validation, exact measured source/harness, compatible baseline/current/peer comparisons, complete raw samples, release artifact identity, signatures and approval-time checks.
• Integrate the public installed contract using the maintainer-authorized exact development pin for rehearsal. release: plan, publish and verify la-stack v0.5.3 #333 owns the final published dependency/lock resolution, with no sibling/editable/Git/path override in the release. Do not copy the shared evidence engine into the consumer.
• Eliminate duplicate acquisition only where the accepted contract proves equivalent eligible evidence. Preserve original measurement host/session and raw bytes; no local/hosted ratio mixing or conversion of local results into falsely attributed CI evidence.
• Keep performance-release coherent and performance-readme separately retryable. Report rendering, packaging or upload retries must not invoke measurement again.
• Update native declarations, workflows and release/benchmark/tooling guides consistently. Keep current safe behavior until the new ownership and tagged-release handoff pass focused integration.

Benefits

Removes justified duplicate current-version measurement without removing scientific coverage or weakening publication safeguards.

Implementation Notes and Acceptance

• Merge feat: resume and reuse verified release benchmark acquisitions #335 after required review and CI; its closing reference then closes this issue automatically.

• Link upstream Astra's public-contract decision and record Astra's consumer release-ordering/trust checkpoint before wiring changes.

• Transferred to release: plan, publish and verify la-stack v0.5.3 #333: record the exact normally resolvable supplying PyPI version and package evidence.

• Installed-package consumer fixtures prove eligible evidence feeds all required consumers without acquisition; stale/missing/incompatible/incorrectly asserted evidence remains blocking.

• Reconcile source-validation reuse with actual release artifact identity and native packaging checks. Preserve immutable historical evidence, signed CI eligibility, draft-release and protected-environment rules.

• Run required local consumer workflow/native/scientific checks; feat: resume and reuse verified release benchmark acquisitions #335 retains required PR CI and release: plan, publish and verify la-stack v0.5.3 #333 owns final published-pin revalidation. Generic format and evidence regressions remain upstream.

• Compare aggregate measurement work and critical-path elapsed time separately; savings remain estimates until comparable authorized runs exist.

• If the identity/order checkpoint is unresolved, keep this adoption open or explicitly retarget it with maintainer approval. Retaining the old two-acquisition behavior is safe fallback, not completion.

Related: #320 (availability), #322 (workload review), #324 (progress) and #325 (resume). Consumer adoption or publication must not block upstream publication. This does not change or interrupt v0.5.2.

Workstream tracker and PR/dependency plan: #326.

Explicit Step 1 / Step 2 acceptance — 2026-11-03

The required operator contract refers specifically to the two main sections of docs/RELEASING.md:

1. Step 1 — prepare the release PR: acquire the complete declared baseline/current evidence once, retain raw samples and original provenance, and generate the release report/README outputs from it.
2. Step 2 — publish after merge: just release-evidence validates the selected Step 1 acquisition against the final reviewed commit/version and required CI/package evidence, before creating or pushing the tag. Later packaging, attachment and publication reuse that accepted acquisition without measurement.

Moving acquisition after tagging, leaving a second hosted acquisition, or only reusing correctness CI does not satisfy this issue.

Scope and acceptance

• Transferred to release: plan, publish and verify la-stack v0.5.3 #333: verify the exact published upstream capability and update every consumer pin/lock/isolated invocation through normal registry resolution, with no editable/sibling/Git override. The implementation already uses the public API without copying the generic validation engine.
• Integrate the supported acquisition format and independently asserted candidate-to-final-release binding. Keep effective/original measured source distinct from final release acceptance; preserve host/session/time, commands, dependencies, harness/workload policy and every original raw byte.
• Retain all repository-specific scientific gates and benchmark coverage. Changed performance-relevant inputs, incomplete/tampered/stale evidence, and unsupported provenance must block reuse with an actionable reason.
• Extend the canonical just release-evidence composition to validate both final-main CI/package evidence and the selected Step 1 benchmark evidence. It must inspect only: no benchmarks, preflight sampling, rebuilding, tagging or publication.
• Replace release-time acquisition with validated archive derivation and attachment. Freeze the accepted run/identity/digests and revalidate the same selection at publication and approval.
• Preserve existing security checks, package guarantees, protected environments, immutable artifact/draft/tag checks, archive safety, and credential separation.
• Keep report/README generation and package/upload retries independently repeatable without acquiring data. Do not modify published releases or frozen historical evidence.
• Update docs/RELEASING.md, canonical recipes, Actions workflows and related command documentation together so the Step 1/Step 2 boundary is unambiguous.
• Consumer tests prove all outputs derive from one acquisition; use failure sentinels to prove Step 2 and later stages never invoke acquisition. Cover changed relevant inputs, allowed output-only changes, bad final-commit bindings, replaced artifacts and retries.
• Record local implementation checks and bounded installed-package consumer evidence in feat: resume and reuse verified release benchmark acquisitions #335; required native PR CI remains enforced there. release: plan, publish and verify la-stack v0.5.3 #333 owns native Linux/macOS/Windows and default/exact checks for the final published-package adoption revision, plus actual release evidence. Upstream tests or a dependency bump alone do not complete release adoption.

Delivery boundary

The implementation and bounded validation are complete in #335; this issue closes only when that PR passes required review/CI and merges. Published-package adoption and release-revision verification are retained in #333. Common command and automatic-draft alignment remain owned by #330. This handoff does not claim a real benchmark campaign or release has occurred, and consumer adoption does not block upstream publication.