Repository navigation
Prepare reproducible public retrospective release - #38
Merged
Merged
Conversation
Adds the completion plan derived from a seven-lens review of the release candidate at 6ba2c0f: status against the original stages, remaining gaps, owner decisions, and ordered tasks through merge, visibility and tag. It records the replay-bound-bytes constraint: the postmortem manifests hash current bytes of producer sources, replay tooling and inputs, including pyproject.toml and uv.lock.
The postmortem replay manifests bind the current bytes of their producer sources, replay tooling and inputs, but CI never runs the replay. The only test loading a real manifest caught producer edits by accident of check order and missed input edits such as pyproject.toml or uv.lock. test_committed_replay_manifests_match_the_tree verifies both committed manifests against the tree. Mutation evidence: - Appending a blank line to pyproject.toml failed both new cases with "input sha256 mismatch: pyproject.toml"; the existing test still passed. - Running ruff format on src/ti26/teams.py failed them with "producer source sha256 mismatch: src/ti26/teams.py". Both mutations were restored and the tests pass again. CONTRIBUTING.md now exempts replay-bound files from "format changed code".
- Checklist: hosted CI observed on 6ba2c0f, replay guard, stage D criterion recorded as not exercised by owner decision, merge-commit and final-verification steps, hosted settings before and after the flip, Zenodo ordering, and a post-publication incident section. - Data sources: owner publication decision with its reply deadline, attribution and takedown route; the analyses stay on hold pending owner review. - SECURITY.md: second reporting channel at publication; rights-holder removal requests. - reproduce.md: bundle enumeration command, verify-run success signal, replay duration and replay-bound failure mode. - README glosses TI and the compendium card. The docs index notes that the verbatim external audit's local-path links do not resolve. - The forecast design spec's links to the private gpt-pro-output.md become plain text.
- reproduce.md: the replay prints one ingest line before its receipt, and it fails before ingesting (manifest verification runs first). - data-sources.md: drop pre-screen findings that have no committed receipt. Full-history publication leaves no per-file hold, so the owner must approve both analyses or switch routes before the deadline. - release-notes.md: no longer contradicts the checklist on hosted CI. - SECURITY.md: private vulnerability reporting can only be enabled after the flip; email always works. - Completion plan: CI evidence cites the hosted run rather than a receipt field recording that CI had not yet run; drop compression percentages whose source files were deleted; D3 must be settled before the D1 deadline.
The previous scorecard reviewed 56b653e, before the public-release work, and its top fix and CI/release rows had gone stale. The new evidence file re-grades every rubric dimension at b23a418 with cited evidence and computes the aggregate with a recorded command. The prior evidence file is unchanged and marked superseded. The Metrics proposal from the completion plan was rejected: the measurement code and reports are byte-identical across the reviewed span. An adversarial verifier reproduced the aggregate and the checks. The corrections after review (pre-screen claims withdrawn in 47b8a79, CI run on an ancestor commit, offline-test evidence) are recorded in post_review_corrections. No grade changed.
A fresh worktree detached to 0186447 ran: - uv sync --locked - unfiltered pytest and Ruff - verify-run --at-source-revision for every tracked bundle - the postmortem replay (group byte-equal; playoff equal after the one store-path normalization) - the optional full forecast oracle ("frozen output replay: match") A second lane ran: - a full-history trufflehog scan of every public ref (no findings) - a pip-audit query of the locked requirements (no advisories) - a link check of all tracked markdown - a capture of the hosted CI record An adversarial cross-check reproduced the results. The receipt drops the lane-B ref count it could not reproduce and redacts home-directory paths. This commit touches only the receipt and the checklist.
The reproducibility reason said the review independently confirmed both guard mutations, but its evidence records only the pyproject.toml mutation. The teams.py mutation is now attributed to implementation commit b07023f, and post_review_corrections records the change. No grade changed. This touches a file outside the receipt/checklist rule for 0186447. The next commit records a receipt addendum that verifies this commit.
3b1cb7c corrected an audit record after 0186447 was verified, so the receipt's rule (only the receipt and checklist may change) no longer held. The addendum binds 3b1cb7c and its tree, lists the diff from the candidate, and records the checks re-run on that commit: - unfiltered pytest and Ruff - all four bundles at their source revisions (logs byte-identical to the candidate run) - local links The postmortem replay and forecast oracle were not re-run: no replay-bound, code, data or report file changed, and the guard test passed. The checklist now says local links (external URLs not checked) and moves the change rule to 3b1cb7c.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prepares this repository for public release as a reproducible research retrospective. Merging this PR does not make the repository public: the visibility change stays with the owner and is gated on the rights decision below.
What changes
docs/reproduce.mdcovers bundle integrity, store reconstruction, the postmortem replay and the optional full forecast replay.reports/postmortems/*.manifest.jsonplusti26.cli_replay_postmortems.uv.lockis now bound into newly produced run descriptors. Historical manifests are unchanged..github/workflows/offline-ci.ymlwith pinned action SHAs, read-only token and network-isolated verification. Tests also deny Python socket access locally.pyproject.tomlanduv.lock. Do not run a repo-wideruff format.docs/superpowers/plans/2026-09-27-public-release-completion.md, the remaining steps through publication.No frozen evidence changed:
git diff 56b653e -- reports/runs data predictions-from-llmsis empty.Verification
docs/audits/2026-09-27-release-candidate-verification.jsonverifies 0186447. Its addendum verifies 3b1cb7c, an audit-record correction to the design re-review (unfiltered tests, Ruff, all bundles, local links). The PR head differs from 3b1cb7c only in the receipt and the checklist. The receipt records:Merge instructions
Use "Create a merge commit". Squash or rebase would leave the commits cited by the receipts reachable from no branch once this one is deleted.
Still open before making the repository public
See
docs/release-checklist.md.docs/data-sources.md#publication-decision).predictions-from-llms/: the owner must approve them as they are before the deadline, or switch to a fresh-history repository. Full-history publication leaves no per-file hold.main.v0.1.0, optional Zenodo DOI, and deleting the merged branches.