Adopt shared benchmark budgets, progress, and remaining-time reporting #324

Summary

Target v0.5.3. Adopt acgetchell/research-repo-tools#167 so maintainers can see the full expected cost, active phase, completed/remaining cases and an honest ETA from the canonical performance workflow.

Implementation prerequisite: research-repo-tools#167 (completed). Historical supplier release: research-repo-tools#170. Final supplying publication and normal-package adoption for v0.5.3 belong to #333.

Implementation handoff — 2026-11-07

The maintainer has moved the published-tooling completion criteria to #333. The implementation and bounded local validation are complete in PR #335, commit a2debb424426b75a97e8a56a061b70985f53d950, using the explicitly authorized research-repo-tools development pin 055353c630604b1ae3438c1eaf534ba85e067a7e. This issue remains open until PR #335 merges after required review and CI. The PR's Closes reference owns automatic closure; published-package adoption and release verification remain separate in #333.

Release-only acceptance is retained, unchecked, in #333: verify the exact supplying PyPI release (target v0.2.7), replace every development pin through normal manifest/lock/isolated-invocation resolution, rerun the affected consumer and native platform/feature checks for that adoption revision, and record the final release evidence. No unpublished pin is accepted for an actual release.

Validation and contract evidence: workload review, candidate review, and #335. Full macOS just ci passed (367 Python tests, 1,143 Rust tests, default/exact doctests and the owning static/native checks); bounded rehearsals used synthetic timing samples, not a real campaign. Required PR CI is still separate from that local evidence.

Historical context

The local driver prints the case count and nominal budget, and the outer recipe records final elapsed time. The v0.5.2 run still required manual inspection to distinguish completed baseline work from current sampling and estimate remaining time. The full plan has 2,673 measurements; nominal timing and observed elapsed time are different quantities.

Proposed Changes

• Integrate the supported public progress contract using the authorized exact development pin for rehearsal; do not add local log scraping or a second progress engine. release: plan, publish and verify la-stack v0.5.3 #333 owns the final published pin and normal package/lock resolution.
• Display a pre-acquisition budget and active preparation/discovery/gate/measurement/report state, with per-revision/suite counts, elapsed time and remaining estimates where supported.
• Retain the final runtime record; distinguish observed timings, nominal budgets, estimates, unknowns and timeouts in terminal output and documentation.
• Keep output useful in local TTYs and captured CI logs without noisy unbounded updates. Progress must not affect scientific sampling or replace final evidence validation.
• Update the release, benchmark and tooling guides to explain long-run expectations and recovery links. README publication remains separate and does not measure.

Benefits

Maintainers can tell whether a run is working, which phase remains and what budget to expect without external monitoring or inspecting temporary files.

Implementation Notes and Acceptance

• Merge feat: resume and reuse verified release benchmark acquisitions #335 after required review and CI; its closing reference then closes this issue automatically.

• Transferred to release: plan, publish and verify la-stack v0.5.3 #333: verify exact-version PyPI availability, the released public interface and final normal-package consumer adoption.

• Focused consumer fixtures cover normal output, unknown ETA, early failure, non-TTY logs, and separate README/report retries without real timing campaigns.

• Verify declarations and shared totals agree for default/exact, baseline/current, peers and current-only API cases; preserve native inventory completeness.

• Run the applicable consumer Python, recipe, shell and documentation checks. Generic event/state/clock regressions remain upstream.

• Astra checkpoint: no routine Astra review required. Escalate only if integration changes evidence acceptance, scientific comparability or the settled shared contract.

This adoption is independent of implementing resume or choosing new release acquisition ownership. It does not add an automation, monitor the current run, or alter v0.5.2.

Workstream tracker and PR/dependency plan: #326.