Adopt shared benchmark workload availability across API revisions #320

Summary

Target v0.5.3. Integrate the shared benchmark workload-availability contract from acgetchell/research-repo-tools#166. Published-version adoption for v0.5.3 is tracked in #333. Replace release-specific baseline exclusions while preserving complete current coverage and scientifically valid shared comparisons.

Implementation prerequisite: acgetchell/research-repo-tools#166 (completed). Verified supplying publication and final normal-package adoption are release gates in #333. Related: #301 and #311. This follow-up is not a blocker for v0.5.2.

Implementation handoff — 2026-11-07

The maintainer has moved the published-tooling completion criteria to #333. The implementation and bounded local validation are complete in PR #335, commit a2debb424426b75a97e8a56a061b70985f53d950, using the explicitly authorized research-repo-tools development pin 055353c630604b1ae3438c1eaf534ba85e067a7e. This issue remains open until PR #335 merges after required review and CI. The PR's Closes reference owns automatic closure; published-package adoption and release verification remain separate in #333.

Release-only acceptance is retained, unchecked, in #333: verify the exact supplying PyPI release (target v0.2.7), replace every development pin through normal manifest/lock/isolated-invocation resolution, rerun the affected consumer and native platform/feature checks for that adoption revision, and record the final release evidence. No unpublished pin is accepted for an actual release.

Validation and contract evidence: workload review, candidate review, and #335. Full macOS just ci passed (367 Python tests, 1,143 Rust tests, default/exact doctests and the owning static/native checks); bounded rehearsals used synthetic timing samples, not a real campaign. Required PR CI is still separate from that local evidence.

Historical context

Preparing v0.5.2 against v0.5.1 failed when the common harness compiled benches/column_basis.rs against a library with no column_basis export. The same incompatibility affects interval division/solve, rational dot products, and polynomial determinants.

The local v0.5.2 mitigation keeps all 1,403 current cases, 1,254 shared baseline cases, and 16 baseline-phase peer cases. The 149 new-API cases retain current timings only. It separates native shared/new targets and correctness gates and declares the current-only families in the independent catalog. This boundary must not become a permanent omission when the baseline eventually supports those APIs.

Proposed Changes

• Integrate the public installed availability contract using the maintainer-authorized exact development pin for rehearsal. release: plan, publish and verify la-stack v0.5.3 #333 owns the transition to an exact published release and normal lock resolution, with no sibling, editable, Git/path or private-API override in the release.
• Replace the version-specific [baseline].current-only policy and redundant phase orchestration with the supported availability declarations. Keep scientific IDs, inputs, native target/feature boundaries, and correctness oracles consumer-owned.
• Resolve availability before compilation. Current-only cases remain required on the current revision; unsupported historical APIs produce unavailable cells and no fabricated ratios. Unexpected compile errors or missing supported cases still fail.
• Once the baseline includes these APIs, automatically require their baseline workloads and expose comparisons only under the declared compatibility evidence. Preserve validated common-API ratios when unrelated workloads are one-sided.
• Keep just performance-release as the preparation/acquisition/report entry point and just performance-readme as a separate retryable publication step. Update the release, benchmark, and tooling guides together.

Benefits

New public APIs can gain benchmark coverage without another hard-coded release repair. Historical absence and genuine benchmark regressions remain distinguishable, with complete evidence retained.

Implementation Notes and Acceptance

• Merge feat: resume and reuse verified release benchmark acquisitions #335 after required review and CI; its closing reference then closes this issue automatically.

• Transferred to release: plan, publish and verify la-stack v0.5.3 #333: record and verify the exact supplying PyPI version, normal pin/lock adoption and release-revision revalidation.

• Focused installed-package integration covers v0.5.1 → v0.5.2, a baseline that supports the new APIs, and a deliberately broken supported target. Test declaration/freshness invalidation and that README retries do not remeasure.

• Cover the availability boundary using the resolved baseline release: v0.5.0/v0.5.1 retain the current-only exclusions, while v0.5.2 and later include those matching cases. Keep baseline_ids() expectations, native baseline recipes, correctness gates, independent discovery requirements, and budget reporting consistent. Test older, exact-boundary, and later baselines; do not infer the historical version from a current-harness manifest overlaid onto the baseline checkout. This review follow-up is deferred from v0.5.2 to preserve its captured harness and retained evidence.

• Preserve the full 1,403-case current catalog unless an independently reviewed scientific policy change supersedes it; retain the 149 new-API cases, current-only correctness gates, and baseline-origin peer attribution.

• Run native discovery and correctness gates on both real source revisions before authorizing long measurements. Keep historical reports/raw evidence immutable; no backfilled baseline timings or comparability.

• Local consumer checks and native default/exact integration pass for the implementation revision; required cross-platform PR CI remains enforced by feat: resume and reuse verified release benchmark acquisitions #335. release: plan, publish and verify la-stack v0.5.3 #333 owns repeating the necessary checks for the final published-package adoption revision. Generic availability, parsing, provenance, failure and publication regressions remain upstream.

• Link the upstream Astra contract decision; request Astra again only if the consumer reveals a substantive scientific/comparability contract gap.

This implementation issue closes when #335 merges with the declarations, documentation and bounded integration evidence. #333 owns the separate supplying-publication, final normal-pin and release-verification criteria. Neither this implementation nor downstream release verification blocks upstream publication.