Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions .github/workflows/cbz-replay-delivery.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
name: cbz-replay-delivery
on:
pull_request:
paths:
- 'cbz/evidence/kernel/additional/**'
- 'cbz/evidence/kernel/determinant/**'
- 'tests/test_cbz_replay_harness_adverse.py'
- 'cbz/replay-qualification/**'
- '.github/workflows/cbz-replay-delivery.yml'
workflow_dispatch:
permissions:
contents: read
defaults:
run:
shell: bash
jobs:
historical-replay:
runs-on: ubuntu-24.04
timeout-minutes: 35
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
repository: d6g8k5htny-coder/Math-
ref: 8151f592918a845bf19eb8eafaf8444540a32734
path: task-math
persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: '3.11.16'
- name: Prepare output and isolate bytecode caches
run: |
set -euo pipefail
mkdir -p replay-current-results
git rev-parse HEAD | tee replay-current-results/commit.txt
cache=$(mktemp -d "$RUNNER_TEMP/replay-bytecode-XXXXXX")
printf 'PYTHONPYCACHEPREFIX=%s\n' "$cache" >> "$GITHUB_ENV"
- name: Check all four original manifests and corresponding source copies
run: |
set -euo pipefail
for package in additional determinant; do
for suffix in '' '/replay-tool'; do
folder="cbz/evidence/kernel/$package$suffix"
label="$(printf '%s%s' "$package" "$suffix" | tr / -)"
(cd "$folder" && sha256sum -c SHA256SUMS) 2>&1 | tee "replay-current-results/checksum-$label.log"
done
done
for file in replay_tool.py test_replay_tool.py run_mutations.py run_lean_selftest.py; do
cmp "cbz/evidence/kernel/additional/replay-tool/$file" "cbz/evidence/kernel/determinant/replay-tool/$file"
done
- name: Historical unit tests and validator-source mutations
run: |
set -euo pipefail
for package in additional determinant; do
cd "$GITHUB_WORKSPACE/cbz/evidence/kernel/$package/replay-tool"
python -B -S -m unittest -v test_replay_tool.py 2>&1 | tee "$GITHUB_WORKSPACE/replay-current-results/$package-unit-normal.log"
python -B -O -S -m unittest -v test_replay_tool.py 2>&1 | tee "$GITHUB_WORKSPACE/replay-current-results/$package-unit-optimized.log"
python -B -S run_mutations.py --output "$GITHUB_WORKSPACE/replay-current-results/$package-mutations" 2>&1 | tee "$GITHUB_WORKSPACE/replay-current-results/$package-mutations.log"
done
- name: Install pinned Lean runtime
run: |
set -euo pipefail
curl -sSfL https://github.com/leanprover/elan/releases/download/v4.2.4/elan-x86_64-unknown-linux-gnu.tar.gz -o "$RUNNER_TEMP/elan.tar.gz"
echo "42b94d4244e8353142c456ec0e4ca6528fd898a6c604d4059f494e706e431f63 $RUNNER_TEMP/elan.tar.gz" | sha256sum -c -
install_dir=$(mktemp -d "$RUNNER_TEMP/replay-elan-XXXXXX")
tar -xzf "$RUNNER_TEMP/elan.tar.gz" -C "$install_dir"
"$install_dir/elan-init" -y --default-toolchain none
"$HOME/.elan/bin/elan" toolchain install "$(cat task-math/formal/lean-toolchain)"
echo "$HOME/.elan/bin" >> "$GITHUB_PATH"
- name: Historical actual-Lean controls in both Python modes
run: |
set -euo pipefail
cd "$GITHUB_WORKSPACE/task-math/formal"
lean=$(elan which lean)
for package in additional determinant; do
tool="$GITHUB_WORKSPACE/cbz/evidence/kernel/$package/replay-tool/run_lean_selftest.py"
for mode in normal optimized; do
flags=(-B -S)
if [ "$mode" = optimized ]; then flags=(-B -O -S); fi
python "${flags[@]}" "$tool" --lean "$lean" \
--collector "$GITHUB_WORKSPACE/task-math/formal/gate_helper.py" \
--collector-sha256 ff8a5374597772b2d37054fd722c2243c4b552a63802c69f7c605519832be6c4 \
--output "$GITHUB_WORKSPACE/replay-current-results/$package-lean-$mode" 2>&1 |
tee "$GITHUB_WORKSPACE/replay-current-results/$package-lean-$mode.log"
done
done
- name: Preserve observed output
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: cbz-replay-historical-${{ github.run_id }}-${{ github.run_attempt }}
path: replay-current-results
if-no-files-found: warn
historical-counterexamples:
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: '3.11.16'
- name: Exercise historical harness defects, including poisoned bytecode
run: |
set -euo pipefail
mkdir -p replay-adverse-results
git rev-parse HEAD | tee replay-adverse-results/commit.txt
env -u PYTHONPYCACHEPREFIX python -B -S -m unittest -v tests.test_cbz_replay_harness_adverse 2>&1 | tee replay-adverse-results/normal.log
env -u PYTHONPYCACHEPREFIX python -B -O -S -m unittest -v tests.test_cbz_replay_harness_adverse 2>&1 | tee replay-adverse-results/optimized.log
echo 'Counterexample test PASS means historical defects were reproduced; it is not auditor certification.' >> "$GITHUB_STEP_SUMMARY"
- name: Preserve counterexample outputs
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: cbz-replay-counterexamples-${{ github.run_id }}-${{ github.run_attempt }}
path: replay-adverse-results
if-no-files-found: warn
strict-qualification:
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: '3.11.16'
- name: Check new qualification manifest and run adverse tests and strict mutations
run: |
set -euo pipefail
mkdir -p replay-strict-results
git rev-parse HEAD | tee replay-strict-results/commit.txt
(cd cbz/replay-qualification && sha256sum -c SHA256SUMS) 2>&1 | tee replay-strict-results/checksum.log
for package in additional determinant; do
dir="$GITHUB_WORKSPACE/cbz/evidence/kernel/$package/replay-tool"
export CBZ_REPLAY_DIR="$dir"
for mode in normal optimized; do
flags=(-B -S)
if [ "$mode" = optimized ]; then flags=(-B -O -S); fi
env -u PYTHONPYCACHEPREFIX python "${flags[@]}" cbz/replay-qualification/test_replay_mutations_strict.py -v 2>&1 |
tee "replay-strict-results/$package-adverse-$mode.log"
done
python -I -B -S cbz/replay-qualification/replay_mutations_strict.py \
--directory "$dir" \
--source-sha256 b97bee53a590651b83d68056c090ba70e8a055157e65d2bd3fcc5cd0219840b3 \
--test-sha256 a07eaab8b6fd40a911d313197fae688927ebb3e1d74644628e4888a7e6505fc0 \
--output "$GITHUB_WORKSPACE/replay-strict-results/$package-mutations" 2>&1 |
tee "replay-strict-results/$package-mutations.log"
done
- name: Preserve strict qualification outputs
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: cbz-replay-strict-${{ github.run_id }}-${{ github.run_attempt }}
path: replay-strict-results
if-no-files-found: warn
4 changes: 4 additions & 0 deletions DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,7 @@ Append-only. Historical source outcomes remain in their original records.
| 2026-10-29 16:56 | 1 | C | Began installing the repository-pinned Lean toolchain for fresh build/kernel execution. Historical hosted CI will remain historical evidence. | Yes | Math `formal/lean-toolchain`, `lake-manifest.json` |
| 2026-10-29 16:54 | 1 | C | Timestamp correction: the entries labelled 16:55 and 16:56 above used projected times by mistake; both actions had actually completed by the observed clock reading 16:54:09 UTC. The actions and evidence are unchanged. | Yes | UTC clock observation 2026-10-29 16:54:09; append-only correction |
| 2026-10-29 16:54 | 1 | C | Direct fast-forward push to main was rejected by repository rule GH013 (PR required and three current status checks required). Use a PR with actual automated checks and autonomous integration; no owner input is needed. Do not bypass the rejection. | Yes | Git push exit 1, remote rule readback |
| 2026-10-29 22:00 | 2 | C | Restore the eight missing replay source/test blobs from composite commit 82d9e480a51b946b3e74525ccf32139129d37d28, preserving prior split metadata and manifests; add scoped hosted rechecks and explicit historical-harness counterexamples. Historical results do not certify current head or G0. | Yes | PR #705; TOOLING.md; .github/workflows/cbz-replay-delivery.yml |
| 2026-10-29 22:03 | 2 | C | Add separately checksummed source-only mutation-grader candidate and adverse tests to address the identified historical false kills without changing historical files. Record trusted-runtime and Lean-self-test limits; verification pending. | Yes | cbz/replay-qualification; PR #705 |
| 2026-10-29 22:03 | 2 | C | Place current-status notices beside both preserved evidence packages and their replay tools to qualify historical verified labels without rewriting historical metadata. | Yes | cbz/evidence/kernel/{additional,determinant}/CURRENT_STATUS.md |
| 2026-10-29 22:04 | 3 | B | Bind hosted replay workflow checkout to the pull-request head SHA (or dispatched SHA) so recorded commit identity names the tested candidate. | Yes | .github/workflows/cbz-replay-delivery.yml |
37 changes: 37 additions & 0 deletions GOVERNANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,3 +107,40 @@ are being inventoried against their actual tests and execution coverage.
Mission tools must be finished, documented with exact usage and verification,
and registered here and in `TOOLING.md` before final admission. Tools are a
means to closure, never a reason to open a general infrastructure project.

### Historical bounded replay helper and delivery workflow

The two evidence packages contain byte-identical historical copies of
`replay_tool.py`, `test_replay_tool.py`, `run_mutations.py`, and
`run_lean_selftest.py`. They generate and check a bounded Lean replay and
exercise finite controls; they are not a federation-wide Layer A auditor.
The original manifests, historical reports and metadata are immutable evidence.
The delivered source is restored from composite source commit
`82d9e480a51b946b3e74525ccf32139129d37d28`. Historical Python and Lean v3
reports are distinct from a fresh current-head run. The v3 reports identify
the delivered Lean self-test source; the separate final-normal report identifies
a different self-test source and must not be attributed to these bytes.

The historical mutation grader accepts a named failure plus an unrelated
assertion or another error, and a zero-test baseline; its Python loader may
read pre-existing bytecode even under `-B`. The Lean self-test's textual
diagnostics have similarly limited binding. These are source-review findings
with executable counterexample tests provided, not a claim that those tests
have passed at the current head. The helper is not admitted as a sound G0
certifier. See `TOOLING.md` for commands and scope. The hosted workflow uses a
fresh private bytecode prefix for historical runs and preserves raw outputs;
this reduces cache exposure on a trusted runner but does not fix the tool.

### Proposed source-only mutation qualification

`cbz/replay-qualification/replay_mutations_strict.py` is a separately
checksummed candidate repair for the historical Python mutation grader. It
pins the historical source and test hashes, compiles their bytes directly,
records the adapted test-loader hash and structured per-test results, requires
31 distinct matching test identities, and binds each sole failure to its
guard-specific assertion. Its accompanying tests exercise classifier mutations,
wrong assertions, foreign errors and poisoned target/test bytecode. It does not
repair the separate historical actual-Lean self-test. Both candidate and tests
are pending current-head execution; they are not yet admitted as a certifier.
Trust remains in Python, the standard library, process/OS, and filesystem; a
transient source replacement between reads is not ruled out.
24 changes: 24 additions & 0 deletions STATUS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
# Current replay delivery notice

This notice is source-bound to the proposed #705 delivery and does not supersede
the scientific snapshot below. The prior #704 split recorded four replay files
as deferred. Those historical split records are retained unchanged. The #705
tree now restores the four exact historical scripts in each of the additional
and determinant evidence packages. Their original four SHA256SUMS files and
historical execution receipts remain untouched.

The historical replay helper is bounded to the named Lean module closure and
does not establish federation-wide Layer A compliance or authenticate supplied
transcripts. Source review found false-acceptance weaknesses in its mutation
grader and bytecode loading, plus limited diagnostic binding in its actual-Lean
self-test. A separate source-only mutation harness candidate and its adverse suite are
present under `cbz/replay-qualification`; current execution is pending and it
does not repair the Lean self-test. The counterexample tests exercise the original harness;
their passing would demonstrate defects, not certify the helper. No fresh
current-head verification result is asserted here. The workflow
`.github/workflows/cbz-replay-delivery.yml` checks the four original manifests,
runs the original suites in both Python modes, and records counterexamples;
hosted results must be inspected for the exact current head. See
`GOVERNANCE.md`, `TOOLING.md`, and `DECISIONS.md`. This delivery does not
make or change a mathematical closure claim, and CBZ-0 remains unverified.

# Scientific status snapshot

**Snapshot date:** 2026-09-29
Expand Down
71 changes: 71 additions & 0 deletions TOOLING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# CBZ replay delivery tooling

The historical additional and determinant packages each contain the same
bounded replay helper: `cbz/evidence/kernel/{additional,determinant}/replay-tool`.
It generates a Lean closure-replay program and validates its transcript against
a caller-supplied census; it does not authenticate a caller-supplied execution.
It is not a global auditor. Maintenance owner: none, bounded evidence artifact.
It serves criteria 3 and 6 only within the explicitly supplied closure, and
criteria 9 and 11 for its documentation and reproducibility.

With Python 3.11.16, from either replay-tool directory, run
`python -B -S -m unittest -v test_replay_tool.py` and
`python -B -O -S -m unittest -v test_replay_tool.py`.
Run `python -B -S run_mutations.py --output /new/output-directory` to
exercise the three registered source mutations in both modes. Actual-Lean
controls require the Lean 4.34.1 runtime and collector described in the package
README; use its documented command once in normal mode and once with `-O`,
with separate new output directories. Verify `sha256sum -c SHA256SUMS` in each
package directory and each replay-tool directory. The hosted
`.github/workflows/cbz-replay-delivery.yml` executes these checks and retains
logs and reports. Current-head execution is pending until observed.

The historical `verification-v2` and `lean-verification-v3-*` reports are
historical evidence, not fresh verification. The Lean `final-normal` report
names a different self-test digest, so does not authenticate the delivered
self-test bytes. Historical Python mutation grading accepts unrelated assertion
failures, extra errors and a zero-test baseline, and its module loader can read
poisoned bytecode despite `-B`. The Lean self-test checks substrings, which
does not bind the entire transcript to the asserted reason. Source review
identified these issues; hosted counterexample execution is pending. Therefore
the helper and its historical self-tests are not admitted as an A8/G0
certifier. The historical source and reports remain unchanged.

The new `tests/test_cbz_replay_harness_adverse.py` exercises the actual
historical mutation `main()` with controlled subprocess results and tests a
timestamp/size-valid poisoned cache against the actual historical module loader.
Run `python -B -S -m unittest -v tests.test_cbz_replay_harness_adverse` and
the same command with `-O`. These are counterexample demonstrations: a pass
means the historical weakness was observed, not that it is fixed. The workflow
runs them and preserves logs in both modes. Owner: none. Their own execution
and the workflow are pending observation at the current head; they do not
certify Layer A. Fresh private bytecode prefix isolation in the historical
workflow assumes a trusted hosted runner and mitigates pre-existing caches only.

## Proposed strict mutation qualification

`cbz/replay-qualification/replay_mutations_strict.py` and
`test_replay_mutations_strict.py` have a **new separate** SHA256SUMS. They
do not replace or modify the four historical manifests. This candidate pins the
original source/test hashes, adapts only their loader in memory, executes the
verified source bytes, retains the adapted test hash and structured results,
and requires 31 unique tests, no extra failures/errors/skips, and the exact
guard-specific assertion in each Python mode. Its tests plant classifier
mutations and exercise unrelated assertions, foreign errors and poisoned
timestamp-valid source/test caches. Current-head verification is pending.
The candidate has not been admitted as a certifying tool.

With `dir="$PWD/cbz/evidence/kernel/additional/replay-tool"`, verify the new
manifest using `(cd cbz/replay-qualification && sha256sum -c SHA256SUMS)`.
Run its tests using
`CBZ_REPLAY_DIR="$dir" python -B -S cbz/replay-qualification/test_replay_mutations_strict.py -v`
and again with `-O`. Run the candidate itself with
`python -I -B -S cbz/replay-qualification/replay_mutations_strict.py --directory "$dir" --source-sha256 b97bee53a590651b83d68056c090ba70e8a055157e65d2bd3fcc5cd0219840b3 --test-sha256 a07eaab8b6fd40a911d313197fae688927ebb3e1d74644628e4888a7e6505fc0 --output /new/strict-output`.
Repeat for the determinant directory with a distinct output. The candidate
runs both Python modes internally. The workflow performs all of these.

Trust assumptions include the Python runtime, standard library, subprocess
results and filesystem; read/re-read hashing cannot rule out transient
replacement. This candidate does not fix the separate Lean self-test's
substring grader or normal-import behavior. Owner: none, bounded
self-contained candidate, serving criteria 6 and 9 if verified.
Loading
Loading