Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 35 additions & 5 deletions .github/workflows/formal-verification.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
persist-credentials: false
- name: Parse formal workflow with the hosted YAML parser
run: |
ruby -rpsych -e 'Psych.safe_load(File.read(ARGV.fetch(0)), aliases: false); puts "FORMAL_WORKFLOW_YAML_PARSED"' .github/workflows/formal-verification.yml
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: '3.11'
Expand All @@ -59,8 +62,14 @@ jobs:
echo 'PIPELINE_FAILURE_PROPAGATION_PASS'
- name: Shell regression controls
run: python -B -S -m unittest discover -s tests -p test_formal_workflow_shell.py -v
- name: Phase custody regression controls in both interpreter modes
run: |
python -B -S -m unittest discover -s tests -p test_formal_evidence_retention.py -v
python -B -O -S -m unittest discover -s tests -p test_formal_evidence_retention.py -v
- name: Source-only identity gate (no Lean yet; fails closed before any download)
run: python -B -S tools/formal_gate_check.py
- name: Separate preexisting evidence from this execution
run: python -B -S tools/formal_evidence_history.py prepare
- name: Install elan (pinned release script, digest checked) and the pinned toolchain
run: |
set -euo pipefail
Expand All @@ -70,24 +79,44 @@ jobs:
echo "$HOME/.elan/bin" >> "$GITHUB_PATH"
"$HOME/.elan/bin/elan" toolchain install "$(cat formal/lean-toolchain)"
- name: Kernel lane — build, leanchecker, axiom audit, negative controls, receipt
id: initial
env:
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: python -B -S tools/formal_gate_check.py --run-lean | tee /tmp/run-lean.log
- name: Gate controls in both interpreter modes
run: |
python -B -S -m unittest tests.test_formal_gate -v 2>&1 | tee /tmp/tests-normal.log
python -B -O -S -m unittest tests.test_formal_gate -v 2>&1 | tee /tmp/tests-optimized.log
- name: Retain initial evidence before the normal execution
if: always()
env:
PHASE_OUTCOME: ${{ steps.initial.outcome }}
run: python -B -S tools/formal_evidence_history.py capture initial "$PHASE_OUTCOME"
- name: Gate controls in normal interpreter mode
id: normal
run: python -B -S -m unittest tests.test_formal_gate -v 2>&1 | tee /tmp/tests-normal.log
- name: Retain normal evidence before the optimized execution
if: always()
env:
PHASE_OUTCOME: ${{ steps.normal.outcome }}
run: python -B -S tools/formal_evidence_history.py capture normal "$PHASE_OUTCOME"
- name: Gate controls in optimized interpreter mode
id: optimized
run: python -B -O -S -m unittest tests.test_formal_gate -v 2>&1 | tee /tmp/tests-optimized.log
- name: Bind current-run receipt to required check
id: required-binding
run: python3 -B -S tools/required_formal_check.py receipt
- name: Retain optimized evidence and the actual binding outcome
if: always()
env:
PHASE_OUTCOME: ${{ steps.optimized.outcome }}
BINDING_OUTCOME: ${{ steps.required-binding.outcome }}
run: python -B -S tools/formal_evidence_history.py capture optimized "$PHASE_OUTCOME" --binding-outcome "$BINDING_OUTCOME"
- name: Collect evidence
if: always()
run: |
mkdir -p formal-evidence
git rev-parse HEAD > formal-evidence/commit.txt
cp /tmp/run-lean.log /tmp/tests-normal.log /tmp/tests-optimized.log formal-evidence/ 2>/dev/null || true
if [ -d formal/.lake/formal-evidence ]; then cp -r formal/.lake/formal-evidence formal-evidence/receipt; fi
python -B -S tools/formal_evidence_history.py verify
if [ -d formal-evidence/phases/optimized/raw ]; then cp -r formal-evidence/phases/optimized/raw formal-evidence/receipt; fi
ls -R formal-evidence
- name: Describe coverage
if: always()
Expand All @@ -103,5 +132,6 @@ jobs:
with:
name: formal-verification-${{ github.run_id }}
path: formal-evidence
include-hidden-files: true
if-no-files-found: error
retention-days: 30
42 changes: 42 additions & 0 deletions docs/FORMAL_REQUIRED_CHECKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,48 @@ PR-head evidence with base/head integration checks. Changing a base changes the
tested merge; the existing strict rulesets handle revalidation. Manual diagnostic
runs do not substitute for PR checks. This is not merge-queue configuration.

## Per-phase execution custody

The main formal workflow still executes the real gate three times: the initial
CLI, the normal test suite's real-package control, and the optimized test suite's
real-package control. Their raw outputs are retained separately under
`formal-evidence/phases/{initial,normal,optimized}/raw`. The initial and normal
directories are moved out of the shared producer path before the next execution;
the optimized directory remains at `formal/.lake/formal-evidence` for the
unchanged required-check binder. Its snapshot is copied after the binding step,
including any binding file actually produced. The artifact's existing `receipt/`
view comes from that verified optimized snapshot, never an earlier fallback.

Each phase's `observation.json` records its native step outcome, producing
commit/repository/run/attempt, source presence and complete file-size/SHA-256
inventory. Optimized observations also record the actual binding-step outcome.
These are custody records, not new gate receipts or status authorities. Receipt
presence does not establish phase success: a suite may fail after its real gate
has written a valid receipt. Skipped phases adopt no leftover canonical files.
Partial failed output and a failed phase with no output are recorded as such.
Any evidence already present before the first execution is preserved separately
as `preexisting`; it receives no fresh execution credit.

The standard-library helper refuses duplicate destinations, linked ancestors,
symlinks, hard-linked files and special files, then verifies snapshot membership
and bytes. Capture or verification failure fails the formal job. Always-run
collection/upload preserves available evidence; job cancellation, timeout or
upload failure may prevent full retention and must not be reported as complete.
The gate writes a subprocess log only after that subprocess returns, so this
change cannot supply a log that the gate never wrote. This is sequential custody
inside an owned job workspace, not protection against a concurrent hostile writer.
Outside this workflow, the gate's existing fixed-directory behavior is unchanged.

The historical C225 initial log bytes were overwritten and remain unavailable.
New runs produce new evidence; this change does not recover those bytes. Earlier
phase records do not replace the final required receipt, independent statement
alignment, source review, or scientific acceptance.

Regression controls are `tests/test_formal_evidence_retention.py`, run normally
and with `-O` before toolchain installation. Fixtures test custody and actual
receipt binding without compiling Lean; hosted real gate/suite execution remains
a separate required check. The existing shell failure controls remain in place.

## Current and future agents

Coordinate in the current source-bound PR. The earlier discussion is in
Expand Down
Loading
Loading