Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ jobs:
python3 scripts/test-release-control-attempt-v090.py
python3 scripts/test-run-release-control-v090.py
python3 scripts/test-execute-release-controls-v090.py
python3 scripts/test-modeling-probe-arguments-v090.py
python3 scripts/test-release-python-parser-v090.py
python3 scripts/test-supervise-release-command-v090.py
python3 scripts/test-swift-common-v2.py
- run: cargo fmt --check
- run: cargo test
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,7 @@ __pycache__/
/reports/raw/swift-endpoints-v1/

/reports/raw/swift-topology-v1/

# Durable task-owned runtime assets and raw execution evidence; never disposable scratch.
/execution-state/
/recovery/2026-09-30-execution-loss/reconstruction-source-events.json
21 changes: 21 additions & 0 deletions recovery/2026-09-30-execution-loss/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Execution recovery, 30 September 2026

The previous serial supervisor stopped on 29 September at 16:26:04 UTC after seven controls. Its eighth launcher, `probe-java-modeling-load-bearing`, returned 1. No matrix group started. Terminal session events survive, but the original raw outputs, logs, registration checkout and local-only commits do not. Seven transcript completions therefore contribute **zero currently qualified controls**. All 33 controls and 84 matrix groups still require preserved, verifiable evidence.

The missing original contract was identified as SHA-256 `29c1a14d422506cd585a9ab2bc7fe7035b87ed5ca3426a46c51399b2c129db39`, registered in local commits `18322a95c` and `761c693ca`. These names are historical references, not recovered objects. Current main `74c0df38e` changes only README and docs relative to merged harness `94ff0ad5ac1c8e54b9bda73903d461702a89cd9b`.

The command-level replay exits before scratch creation or analyzers: the Java probe requires `--codeql-packs`, absent from its registered argv. The JavaScript probe has the same omission. These deterministic reproductions support an argument defect; the original failure stderr remains unavailable. No native analysis was repeated.

The new candidate is `reports/releases/v0.9.0/execution-v1/recovery-20260930-01/contract.json`. It is intentionally disabled and is not a reconstruction of the missing original bytes. It fixes the two prospective argument arrays, designates durable execution storage inside this workspace, preserves immutable parent-plan bindings, and requires new runtime/source reservations. Historical runner receipts and Swift plans remain evidence of their original installations, not proof that those paths still exist.

Before launch:

1. Restore and verify exact missing CodeQL 2.27.1, Swift 6.3.3, repaired extractor/packs, OpenTaint full bundle, and runner in durable storage. Six other runtime-tree inventories currently verify.
2. Explicitly register relocated runtime plans and new build provenance. Recalculate acquisition/extraction scratch while preserving the 136 GiB launch floor and 40 GiB reserve.
3. Review attempt accounting across the lost run. A new directory does not reset the two-attempt limit; no outcome-selecting retry is allowed.
4. Validate the corrected eighth control under a retained diagnostic attempt before starting a full serial recovery. Wait for the active Bifrost build/coverage slot to clear.
5. Validate all 117 operations in designated roots; commit and push the final sealed packet before heavy execution. Persist supervisor stdout, stderr, started and terminal status in durable storage, stopping on failure with no automatic retry.

`reconstruction-source-events.json` is a local recovery aid extracted from this session, not a release artifact. It is intentionally not part of the committed evidence package. No private session narrative is required to reproduce the argument failures.

The `supervise-release-command-v090.py` wrapper requires an explicit deadline and fresh durable log directory, preserves stdout/stderr and start/terminal receipts, and propagates nonzero status. It never retries. It cannot persist a terminal event after SIGKILL, sudden power loss, or filesystem failure; a started-only receipt is an interrupted/unknown attempt requiring review. Descendants that create their own process sessions remain the inner launcher's containment responsibility. All 117 exact parser/mode checks remain a launch blocker: the two defective shell parsers are covered now, but missing runtimes prevent full command qualification.
16 changes: 16 additions & 0 deletions recovery/2026-09-30-execution-loss/codeql-restoration-drift.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"tree_differences": [
{
"path": "codeql",
"expected": {
"sha256": "5010324098d11f9e2d34e0fcf53bb44e40e8e3ffc8aecefe695269b54edaa413",
"mode": 365
},
"actual": {
"sha256": "5010324098d11f9e2d34e0fcf53bb44e40e8e3ffc8aecefe695269b54edaa413",
"mode": 493
}
}
],
"all_file_hashes": "matched"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"captured_utc": "2026-09-30T09:07:26.697261+00:00",
"root": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/tools/full108/codeql-runtime/codeql",
"full_inventory": "matched historical CLI files, hashes, modes, symlinks",
"mode_restoration": "codeql launcher0755 from installer restored to recorded0555; bytes unchanged",
"native_execution": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"scope": "argument-validation-only; exits before scratch creation or any analyzer",
"argv": [
"bash",
"scripts/probe-java-modeling-load-bearing.sh",
"--bifrost",
"/Users/dave/.cache/dataflowbench-tools/bifrost-v0.11.4/bifrost-v0.11.4-universal-apple-darwin/bifrost",
"--codeql",
"/private/tmp/dfb-full108-assets-20260928/codeql-runtime/codeql/codeql",
"--joern",
"/Users/dave/.cache/dataflowbench-tools/joern-v4.0.628/joern-cli/joern",
"--semgrep",
"/Users/dave/.cache/dataflowbench-tools/semgrep-1.177.0-venv/bin/semgrep"
],
"exit_code": 2,
"stdout": "",
"stderr": "--codeql-packs is required so the probe uses the release-pinned pack tree\n",
"inference": "Registered argv lacks required --codeql-packs. Deterministic current replay confirms missing argument; original stderr remains missing."
}
20 changes: 20 additions & 0 deletions recovery/2026-09-30-execution-loss/extractor-rebuild-plan.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"schema": "extractor-recovery-build-plan/v1",
"registered_utc": "2026-09-30T09:14:26.819570+00:00",
"source_repository": "https://github.com/github/codeql.git",
"source_revision": "6e9f9e38390175c41b99070a423c875f450759ca",
"patch_sha256": "e7da7352a8b3a8e7679760e49e8f8abff05d7fa3e16007c7ed2987f49074f26a",
"historical_binary_sha256": "593eb7afe23d63c57d04461bd046e45672555bb0586353edcf27a0313249841f",
"historical_binary_state": "unavailable; equality must be measured after rebuild",
"root": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/extractor-build",
"jobs": 2,
"cpu_scheduler_hint": 2,
"memory_scheduler_hint_mb": 4096,
"jvm_heap_mb": 2048,
"deadline_seconds": 900,
"attempt_limit": 1,
"additional_acquisition_build_allowance_gib": 40,
"free_bytes": 288768937984,
"minimum_launch_floor_gib": 136,
"action": "Acquire exact source/dependencies first. Fresh process and owner checks before build; record argv/stdout/stderr/terminal. No benchmark. If output differs, new prospective identity and qualification required."
}
81 changes: 81 additions & 0 deletions recovery/2026-09-30-execution-loss/extractor-rebuild-result.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
{
"exit_code": 0,
"stop_reason": null,
"elapsed_seconds": 140.076607458,
"minimum_free_bytes": 284155932672,
"observed_group_peak_rss_kib": 728704,
"tracked_pids": [
41718,
42179,
42530,
42539,
42573,
42686,
42698,
42721,
42749,
42750,
42751,
42779,
42785,
42805,
42811,
42832,
42836,
42883,
42887,
42904,
42908,
42913,
42921,
42944,
42952,
42977,
42986,
43007,
43030,
43052,
43068,
43081,
43087,
43092,
43109,
43125,
43135,
43199,
43202,
43220,
43224,
43237,
43241,
43250,
43256,
43265,
43269,
43280,
43284,
43294,
43301,
43306,
43318,
43328,
43337,
43345,
43369,
43390,
43401,
43408,
43421,
43425,
43432,
44072
],
"remaining_process_group": [],
"cleanup_scope": "Batch mode process group; escaped descendants are not proven absent. No retries.",
"status": "build-success",
"binary_path": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/extractor-build/bazel-cache/a13f666caa8d1d6b13733618918be8f4/execroot/_main/bazel-out/darwin_arm64-opt/bin/swift/extractor/extractor.real",
"binary_sha256": "593eb7afe23d63c57d04461bd046e45672555bb0586353edcf27a0313249841f",
"historical_binary_sha256": "593eb7afe23d63c57d04461bd046e45672555bb0586353edcf27a0313249841f",
"matches_historical_binary": true,
"benchmark_execution": false
}
17 changes: 17 additions & 0 deletions recovery/2026-09-30-execution-loss/javascript-argument-replay.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"argv": [
"bash",
"scripts/probe-javascript-modeling-load-bearing.sh",
"--bifrost",
"/Users/dave/.cache/dataflowbench-tools/bifrost-v0.11.4/bifrost-v0.11.4-universal-apple-darwin/bifrost",
"--codeql",
"/private/tmp/dfb-full108-assets-20260928/codeql-runtime/codeql/codeql",
"--joern",
"/Users/dave/.cache/dataflowbench-tools/joern-v4.0.628/joern-cli/joern",
"--semgrep",
"/Users/dave/.cache/dataflowbench-tools/semgrep-1.177.0-venv/bin/semgrep"
],
"exit_code": 2,
"stderr": "--codeql-packs is required so the probe uses the release-pinned pack tree\n",
"scope": "argument validation only; no native execution"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"captured_utc": "2026-09-30T09:08:26.515262+00:00",
"root": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/tools/opentaint-full",
"status": "matched recorded entries, hashes, modes and links; root relocated",
"native_execution": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"extractor_root": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/extractor-build/candidate-extractor",
"packs_root": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/extractor-build/repaired-packs",
"extractor_sha256": "593eb7afe23d63c57d04461bd046e45672555bb0586353edcf27a0313249841f",
"extractor_tree": "exact historical match",
"packs_files": 3402,
"packs_tree": "exact historical match including membership,mode,symlinks",
"native_execution": false
}
29 changes: 29 additions & 0 deletions recovery/2026-09-30-execution-loss/restoration-inputs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"provenance": "Installer identities recovered from prior session tool output; no downloads in this recovery turn",
"installers": [
{
"name": "CodeQL2.27.1",
"url": "https://github.com/github/codeql-cli-binaries/releases/download/v2.27.1/codeql-osx64.zip",
"bytes": 981846365,
"sha256": "412c600764a7835f9548af120d0bdadea1040c6f68b8f6bf04ec72a664891f63"
},
{
"name": "Swift6.3.3",
"url": "https://download.swift.org/swift-6.3.3-release/xcode/swift-6.3.3-RELEASE/swift-6.3.3-RELEASE-osx.pkg",
"bytes": 1503529808,
"sha256": "ee82e57774d6650f94aa06302435d6f44a055b9411698db8ecb85d9a3bcc91d0"
},
{
"name": "OpenTaint full0.4.6",
"url": "https://github.com/seqra/opentaint/releases/download/v0.4.6/opentaint-full_darwin_arm64.tar.gz",
"bytes": 111028216,
"sha256": "95c14073cb94b3a942488531c74d7812653bc7f684d1efa53f422d5ba22d3d92"
}
],
"verification_required": "Installer hashes plus exact runtime membership, file hashes, executable modes and symlinks. Repaired Swift extractor/packs must be reconstructed and verified against committed manifests, never assumed equal.",
"custom_extractor_gap": {
"sha256": "593eb7afe23d63c57d04461bd046e45672555bb0586353edcf27a0313249841f",
"original_build": "Session records show a locally compiled extractor.real copied from source/bazel-bin/swift/extractor/extractor.real, plus compiler-prebuilt dylibs, into stock CodeQL Swift extractor. Stock installer alone cannot restore this identity.",
"required": "Recover original binary/build inputs or rebuild and verify exact hash. Any different binary requires new prospective identity and qualification, never claim original restoration."
}
}
34 changes: 34 additions & 0 deletions recovery/2026-09-30-execution-loss/runtime-readback.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"captured_utc": "2026-09-30T08:51:52.062010+00:00",
"checks": [
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/bifrost.json",
"status": "verified"
},
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/flowdroid.json",
"status": "verified"
},
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/infer.json",
"status": "verified"
},
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/opentaint.json",
"status": "verified"
},
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/python.json",
"status": "verified"
},
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/semgrep.json",
"status": "verified"
},
{
"path": "reports/releases/v0.9.0/execution-v1/runtime-trees/opentaint-full.json",
"status": "unavailable-or-changed",
"error": "[Errno 2] No such file or directory: '/private/tmp/dfb-v090-opentaint-full'"
}
]
}
20 changes: 20 additions & 0 deletions recovery/2026-09-30-execution-loss/status.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"captured_utc": "2026-09-30T08:47:58.670496+00:00",
"state": "stopped-with-temporary-artifacts-missing",
"completed_controls_from_supervisor_log": 7,
"failed_launcher": "probe-java-modeling-load-bearing",
"failed_at_utc": "2026-09-29T16:26:04.873Z",
"matrix_groups_started": 0,
"registration_commits_missing": [
"18322a95c",
"761c693ca"
],
"sealed_contract_sha256_from_prior_record": "29c1a14d422506cd585a9ab2bc7fe7035b87ed5ca3426a46c51399b2c129db39",
"merged_harness_commit": "94ff0ad5ac1c8e54b9bda73903d461702a89cd9b",
"raw_evidence_reverified": false,
"cause_of_failure": "unknown: temporary launcher log missing",
"new_execution_started": false,
"currently_qualified_controls": 0,
"remaining_controls_requiring_qualification": 33,
"argument_replay": "Current unchanged script exits2 before analyzers: missing required --codeql-packs. Original stderr unavailable."
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"captured_utc": "2026-09-30T09:08:12.379005+00:00",
"root": "/Users/dave/.codex/worktrees/1ba0/dataflowbench/execution-state/v090-recovery-20260930-01/tools/full108/swift-expanded/swift-6.3.3-RELEASE-osx-package.pkg/Payload/usr",
"differences": [],
"status": "verified",
"native_execution": false
}
Loading
Loading