diff --git a/docs/BUILD_LOG.md b/docs/BUILD_LOG.md index 725fd8a..2110c6c 100644 --- a/docs/BUILD_LOG.md +++ b/docs/BUILD_LOG.md @@ -2,6 +2,17 @@ This is a concise chronological record of material implementation work and runtime evidence. It is not a substitute for Git history; it captures intent, verification, and blockers that a commit alone may not explain. +## 2026-08-01 — R7 corpus frozen + +- Implemented the outcome-free Git scanner, isolated Docker provisioner, immutable resume evidence, deterministic candidate merge, and exact 30-patch freezer. RED tests preceded the scanner batching, evidence merge, global-slot cap, and registered-script precheck changes. +- Registered replacements before their eligibility histories were inspected. The collector evaluated 457 records across 10 registrations and froze 30 eligible patches from six repositories: 12 development and 18 held-out. +- Frozen manifest SHA-256: `a4ef6cbfa48c66cb9d384bcc2834ecbfae8ff08810abfd1863b395b8aa47d149`. The held-out set includes `ufo`, which is absent from development. +- Collection left no `pureflow-r7-corpus-*` Docker container or volume. No compiler, probe, model, expert, or participant outcome was inspected during selection. +- The complete extension suite passed 93/93, the explicitly enabled Docker backend suite passed 2/2, TypeScript/build/VSIX packaging passed, and a clean re-freeze reproduced the manifest byte-for-byte. +- R7 remains open: automated compiler validity, replay/parity/adversarial evidence, and two independent human ratings are not yet measured. R5/R6 remain gated. + +Evidence: `docs/v0.3/r7/`, `docs/v0.3/results/R7_CORPUS_COLLECTION.md`, corpus tests, local Docker provisioning on 2026-08-01. + ## 2026-08-01 — R7 Docker SandboxRunner local candidate - Wrote `SANDBOX_RUNNER_SPEC.md` first and passed the strict spec validator at 100/100 before implementation. The contract fixes command authority, trust/consent, opaque handles, image identity, isolation flags, mountpoints, oracle integrity, evidence bounds, single-use IDs, cancellation, cleanup, and fail-closed behavior. @@ -318,3 +329,24 @@ Append an entry after each material milestone with: - Added `openDocsSideBySide` configuration using `vscode.ViewColumn.Beside`. - Created safety gate requiring developers to manually record recall context before loading external documentation during an active Focus Rep. + +## 2026-08-01 — R7 development compiler audit + +- Implemented the deterministic `r7-rewind-v1` plan compiler, Git-backed preparation, digest-pinned Docker executor, typed outcome classifier, immutable per-patch receipts, and cohort summarizer. +- Froze and executed all 12 development identities from corpus `a4ef6cbfa48c66cb9d384bcc2834ecbfae8ff08810abfd1863b395b8aa47d149`: 10 compiled, 2 abstained because a complete base source blob did not exist, 9 were executable-valid, and 1 correctly rejected because the rewind did not fail the registered observation. +- Recorded 75% end-to-end recovery/probe yield and 90% validity among compiled cases. Summary hash: `d4457d21be7939eee4e205d0b559e7e7ef01973df5f0a072f8d2ae4adfcc070b`. +- Verified unique plan/report hashes, unchanged external repository snapshots, source blob integrity around every test, and zero remaining audit containers or volumes. +- Added 13 focused compiler/preparation/classification/summary tests and passed TypeScript checking. Disclosed the archive-hash placement deviation before the held-out freeze. +- Frozen 16 compiler/config inputs in `docs/v0.3/r7/compiler-freeze-v1.json`; raw freeze-file SHA-256: `605b645e9cc27653515665ac002a4b1d9c1e6f36c6ddf3bffbf2be4d2081a81b`. + +Evidence: `docs/v0.3/results/R7_DEVELOPMENT_AUDIT.md` and immutable JSON receipts. R7 remains blocked on the frozen 18-patch held-out run and two independent blind expert ratings; no retention or product-effectiveness claim is made. + +## 2026-08-01 — R7 held-out automatic audit + +- Committed compiler freeze `ab1ea56`, verified all 16 freeze-manifest file hashes, and opened the 18 held-out identities only after that boundary. +- Compiled 17/18 held-out patches, emitted one structural abstention, and executed every compiled patch with the frozen one-control/three-mutation/three-repair protocol. +- Recorded 16/18 end-to-end valid recovery episodes and probes (88.9%, Wilson 95% CI 67.2–96.9%); one compiled mutation correctly emitted `mutation-did-not-fail`. +- Passed the preregistered automatic thresholds without modifying compiler code/config. Summary hash: `fe561009447e187e8c085ac499d62496816f2714fc4dcabccdb064ca97708942`. +- Generated all 18 blind rating packets before outcomes, with index `54d78382b3ddbe15cba1f8153275e8149d32ddaa5192163f99ca5f43d903e8fe`, and verified zero remaining audit containers/volumes. + +Evidence: `docs/v0.3/results/R7_HELD_OUT_AUDIT.md`, immutable execution receipts, and blind packet bundle. Full R7 remains blocked on two independent experienced TypeScript ratings and adjudication; R5/R6 have not started. diff --git a/docs/PROJECT_STATE.md b/docs/PROJECT_STATE.md index 0367a49..3a34cb1 100644 --- a/docs/PROJECT_STATE.md +++ b/docs/PROJECT_STATE.md @@ -2,7 +2,7 @@ Last updated: 2026-08-01 -## Current branch milestone — R7 sandbox accepted; corpus protocol frozen +## Current branch milestone — R7 corpus frozen; compiler audit pending Branch `codex/shadow-cockpit-rnd` resets the product R&D thesis around **Dual-Control Development**. @@ -33,7 +33,8 @@ Branch `codex/shadow-cockpit-rnd` resets the product R&D thesis around **Dual-Co - Sixteen deterministic sandbox contract tests pass locally on Windows. A separate explicit Docker integration suite passed 2/2 against the selected digest on Docker Desktop: the active capability probe and a real frozen command in a path with spaces. The complete default extension suite passes 79/79 with the two provisioned-backend tests skipped; TypeScript, production build, and VSIX packaging pass. - Protected PR #17 run `30674334938` passed `extension`, `extension-windows`, `contract`, `web`, and `jules-rnd-policy`. The Linux extension job explicitly provisioned the exact digest and passed the real Docker backend suite; Windows independently passed the deterministic contract suite. The R7 sandbox implementation gate is complete. - `CONCEPT_LAB_CONTROLLABILITY.md` records a post-R7 category extension: compile an executable `observe → actuate → recover` human control surface, select takeover cut sets, and let a context-starved agent continue writing code from human-selected evidence and directives. Dissent cases and control dividends remain hypotheses with explicit falsifiers, not implemented features. -- The 30-patch corpus is not assembled. The runner candidate and local probe are not permission to begin corpus or participant execution before protected checks and R7 eligibility infrastructure pass. +- The preregistered R7 collector froze 30 eligible patches from six repositories after evaluating 457 bounded eligibility records. Manifest `a4ef6cbfa48c66cb9d384bcc2834ecbfae8ff08810abfd1863b395b8aa47d149` contains 12 development and 18 held-out patches; `docs/v0.3/results/R7_CORPUS_COLLECTION.md` reports repository and first-match exclusion counts. No compiler or human outcome influenced selection. +- R7 has not passed. The recovery/probe compiler audit, three-run evidence, protected parity/adversarial runs, and two independent human ratings remain pending; R5/R6 stay gated. - The readiness ledger and v0.3 cockpit do not exist yet. R0–R4.5 remain a closed reviewed-fixture mechanism and do not execute arbitrary participant or workspace code. - No skill-retention or speed metric has been measured. Values in the PRD are predeclared R&D targets. - A new implementation audit found five R0 ambiguities: candidate-diff identity, pre-store fixture blobs, runtime identity, check IDs, and Git object format. The normative contract closes them with structured diffs, catalog-owned blobs, standalone Node `v22.17.0`, declared test IDs, and SHA-1 Git initialization; R0a/R0b now implement and verify that complete substrate. @@ -132,7 +133,7 @@ The repository contains no verified evidence that the owner submitted the final | Input | Impact | Resolution | | --- | --- | --- | | The first live adapter is selected but no accessible Codex CLI is configured for this checkout | ADR-006 selects Codex App Server over local stdio, but the Microsoft Store packaged executable discovered here returns `Access denied` when launched from the repository shell | Keep replay R&D independent; the live spike must preflight a separately accessible, exact-version user-installed Codex CLI and fail closed when unavailable | -| Technical patch corpus is not assembled | Automatic episode-generation rate cannot be measured | Collect at least 30 consented or open-source test-backed TypeScript patches for R7 | +| R7 expert audit is not complete | Frozen held-out automatic audit passed at 16/18 end-to-end, but independent causal-relevance ratings are not yet measured | Give `docs/v0.3/results/held-out-rater-packets/` to two experienced TypeScript raters using `docs/v0.3/R7_EXPERT_RATING.md`, then adjudicate and report agreement | | Human participants are not recruited | Takeover and delayed-transfer claims cannot be tested | Complete the technical gate, then recruit for the preregistered pilot | | Default-branch Jules scheduler awaits explicit merge approval | Scheduled/manual continuation is not installed on `main`; draft PR #8 remains isolated and the enable variable stays off | Owner explicitly says `merge #8`; then merge through protected `main`, remove the temporary infrastructure branch, and run one guarded canary through the protected R&D branch | @@ -151,8 +152,8 @@ No external input blocks the repository-owned fixture R0–R4.5 mechanism in `do ## Next ordered actions -1. Collect and freeze the preregistered 30 eligible patches, then run the recovery-plus-probe audit. -2. Add the local readiness ledger and minimal cockpit only after the R7 corpus gate passes. +1. Obtain and adjudicate two independent blind expert ratings for packet index `54d78382b3ddbe15cba1f8153275e8149d32ddaa5192163f99ca5f43d903e8fe`. +2. Add the local readiness ledger and minimal cockpit only after the complete R7 expert gate passes. 3. Run the preregistered delayed-transfer pilot before making any skill-retention claim. ## Recent milestone commits diff --git a/docs/v0.3/CORPUS_FREEZE_SPEC.md b/docs/v0.3/CORPUS_FREEZE_SPEC.md new file mode 100644 index 0000000..b1d1ede --- /dev/null +++ b/docs/v0.3/CORPUS_FREEZE_SPEC.md @@ -0,0 +1,48 @@ +# R7 corpus freeze contract + +**Status:** frozen before any R7 compiler outcome is inspected +**Protocol:** `r7-typescript-node-v1` + +## Purpose + +The corpus freezer converts repository registrations and independently collected eligibility facts into one canonical 30-patch manifest. It does not compile recovery episodes, expose held-out compiler outcomes, or decide whether a compiler result is successful. + +## Required behavior + +- Repository registration happens before candidate inspection and records URL, OSI license, pinned tip, runtime, package manager, immutable install/test argv, and the exact first-parent history command. +- A candidate is a full target commit plus its single adjacent first parent and its ordinal among the first 60 coarse candidates that changed TypeScript source and a test path. +- Eligibility facts are outcome-free. They contain bounded Git/test/provisioning evidence only and cannot contain episode, probe, judge, rater, or compiler fields. +- Every ineligible candidate receives exactly one exclusion code: the first failing rule in the frozen order below. +- At most the first 60 coarse candidates per repository are accepted. At most the first 10 eligible candidates per repository enter the corpus. +- Freezing fails unless exactly 30 eligible patches from at least three repositories exist. +- The split key is exactly `SHA-256("pureflow/r7-typescript-node-v1\n" + repository URL + "\n" + target commit)`. +- Patches are sorted by the raw UTF-8 bytes of that lowercase key. Positions 1–12 are `development`; positions 13–30 are `held-out`. +- Canonical JSON and its SHA-256 are emitted so the manifest can be reviewed and frozen before compiler execution. + +## First-match exclusion order + +1. `not-adjacent-first-parent` +2. `license-not-approved` +3. `missing-lockfile` +4. `dependency-or-lockfile-change` +5. `unsupported-artifact` +6. `diff-too-large` +7. `snapshot-too-large` +8. `unsupported-typescript-boundary` +9. `missing-attributed-test` +10. `requires-production-capability` +11. `network-required-at-execution` +12. `base-provision-or-test-failed` +13. `target-provision-or-test-failed` + +The order is deliberately structural-first: cheap immutable failures are classified before provisioning failures. Changing the order requires a new protocol version, never an edit after outcomes are known. + +Three-run replay belongs to compiler evaluation after the corpus is frozen; it is not an eligibility field and cannot influence corpus membership. + +## Non-goals + +- downloading or copying third-party source into this repository; +- treating a static heuristic as proof that a test is causally relevant; +- rating compiler output; +- silently replacing missing evidence with defaults; +- claiming that a frozen corpus has already passed R7. diff --git a/docs/v0.3/R7_COMPILER_SPEC.md b/docs/v0.3/R7_COMPILER_SPEC.md new file mode 100644 index 0000000..5710dc8 --- /dev/null +++ b/docs/v0.3/R7_COMPILER_SPEC.md @@ -0,0 +1,95 @@ +# R7 rewind compiler specification + +**Status:** development iteration 1, frozen before compiler execution +**Compiler protocol:** `r7-rewind-v1` +**Corpus:** `a4ef6cbfa48c66cb9d384bcc2834ecbfae8ff08810abfd1863b395b8aa47d149` + +## Purpose + +Test whether an ordinary historical TypeScript patch can be converted into executable human-control work without a model authoring code, tests, commands, mutations, or answers. + +Iteration 1 uses one repository-independent operator: materialize the passing target revision, replace every changed TypeScript source file with the complete adjacent-base version, retain target tests, and execute the preregistered test command. The target versions of those source files are the protected repair. + +## Frozen inputs + +- only the 12 manifest patches labeled `development` may be executed or inspected while iteration 1 is evaluated; +- repository URL, pinned runtime/package manager, install/test argv, base/target commits, changed source/test paths, and evidence hashes come from the frozen corpus artifacts; +- the external clone is resolved by registered repository ID and verified at the pinned tip; +- no prompt, path, command, test, repair, exception, or mutation may vary by patch. + +## Support decision + +A patch is supported only when every `changedSourcePath`: + +1. is a safe workspace-relative `.ts` or `.tsx` path already accepted by corpus preflight; +2. resolves to an ordinary Git blob in both base and target; +3. is not a declaration file, test, dependency, generated, vendored, build, or coverage path. + +If any source path was added, deleted, or is not an ordinary blob, emit `unsupported-source-rewind`. Do not substitute a different operator during iteration 1. + +## Compile plan + +For a supported patch, emit an internal immutable plan containing: + +- corpus/compiler identity and patch evidence hash; +- target snapshot archive hash; +- base rewind archive hash; +- protected target repair archive hash; +- sorted writable source paths and attributed test paths; +- one deterministic semantic-unit selection from the existing R2 extractor; +- exact registered install/test argv and sandbox toolchain handle; +- participant projection hash and internal plan hash. + +The participant projection contains the mutated snapshot identity, writable paths, attributed tests, selected seam label, attention budget, and registered observation label. It excludes repository URL, source commits, target archive, repair archive, oracle/controller paths, command argv, host paths, and compiler outcome. + +## Execution + +All repository code runs in the digest-pinned R7 Docker boundary. + +1. Seed a native Docker volume from the target Git archive. +2. Install dependencies with the registered install argv and bridge networking. +3. Run the registered target command once with network none; it must pass. +4. Apply the base rewind archive and verify the candidate tree hash. +5. Run the same command three clean times with network none. +6. Apply the protected target repair archive and verify the repaired tree hash. +7. Run the same command three clean times with network none. +8. Remove exact controller-owned containers, volumes, archives, and temporary roots. + +No production checkout, home directory, credentials, extension storage, Git objects, model output, participant text, host executable, caller environment, or unregistered mount is available to the execution container. + +## Validity + +A recovery episode is valid only when: + +- target setup and control pass; +- all three rewound runs fail cleanly with a nonzero exit code; +- all three repaired runs pass; +- immutable plan/tree/command identities remain stable; +- no cleanup or production invariant fails. + +The executable probe is valid under the same evidence when it binds the selected semantic seam to the precommitted prediction “the registered observation fails in the rewound state.” Timeout, cancellation, launch error, output overflow, or integrity drift is an execution error, never a predicted failure. + +Typed outcomes are: + +- `valid`; +- `unsupported-source-rewind`; +- `unsupported-semantic-boundary`; +- `target-control-failed`; +- `mutation-did-not-fail`; +- `mutation-not-reproducible`; +- `repair-failed`; +- `integrity-failed`; +- `execution-error`; +- `cleanup-failed`. + +## Development rule + +Iteration 1 is evaluated on all 12 development patches. If recovery validity is below 60% or probe validity below 50%, one iteration-2 change may be made using aggregate failure categories only. No patch-specific rule is allowed. If iteration 1 clears those floors, freeze compiler source/config immediately rather than optimizing against development identities. + +Held-out execution is forbidden until the freeze commit and config hash are recorded. Human rating packets must be generated for all patches without compiler status or outcomes. + +## Post-run conformance record + +Development iteration 1 completed on 2026-08-01 without patch-specific rules. Nine of twelve patches produced valid episodes and probes, two emitted `unsupported-source-rewind`, and one emitted `mutation-did-not-fail`. The immutable summary hash is `d4457d21be7939eee4e205d0b559e7e7ef01973df5f0a072f8d2ae4adfcc070b`. Both development floors passed, so iteration 2 is forbidden and the implementation proceeds directly to a source/config freeze. + +One implementation deviation was found before the held-out freeze: target, rewind, and repair archive SHA-256 values are recorded in the hash-bound execution report instead of the pre-execution compile plan. The compile plan still binds the full target/base commits, every source blob OID, exact argv, corpus evidence, participant projection, and compiler protocol; the executor creates archives only from those bound Git objects, records each archive SHA-256 and byte length, verifies source blob identities before and after every test, and hashes the complete report. This placement does not change support, mutation, probe, repair, or outcome rules. It is disclosed rather than retroactively changing iteration 1 or rerunning an outcome-tuned compiler. diff --git a/docs/v0.3/R7_EXPERT_RATING.md b/docs/v0.3/R7_EXPERT_RATING.md new file mode 100644 index 0000000..048185c --- /dev/null +++ b/docs/v0.3/R7_EXPERT_RATING.md @@ -0,0 +1,35 @@ +# R7 blind expert rating + +Two experienced TypeScript engineers rate every packet independently before seeing compiler status or execution outcome. Do not open the held-out result directory, internal plans, Git commits, agent transcripts, or protected repair artifacts while rating. + +Each rater returns one bundle containing exactly one record for every `packet-*.json`: + +```json +{ + "schemaVersion": 1, + "protocol": "r7-blind-expert-v1", + "raterId": "stable-pseudonymous-id", + "ratings": [ + { + "packetId": "copied from packet", + "causalRelevance": "yes | no | uncertain", + "targetExpected": "pass | fail | uncertain", + "rewindExpected": "pass | fail | uncertain", + "leakage": "none | repair | compiler-status | observed-outcome | other", + "confidence": 1, + "reason": "one or two sentences grounded in the visible diff" + } + ] +} +``` + +`confidence` is an integer from 1 to 5. A packet is expert-valid only when the adjudicated answer is `causalRelevance=yes`, `targetExpected=pass`, `rewindExpected=fail`, and `leakage=none`. Raters must not execute code or search commit history; this stage measures causal review of the disclosed participant surface, while automated sandbox receipts independently measure behavior. + +After both files are returned, compare exact categorical agreement per field and adjudicate disagreements without exposing compiler outcomes. Save the full final 18-record decision as a third bundle with a distinct panel/adjudication ID. Then run: + +```powershell +cd extension +node scripts/r7-rating.mjs ..\docs\v0.3\results\held-out-rater-packets\index.json +``` + +The join tool verifies the hash-bound packet index, exact coverage, independent rater IDs, categorical values, confidence bounds, and reasons before reporting raw agreement, Cohen's kappa where defined, disagreement count, and adjudicated expert-valid rate. Only then may ratings be joined to the automated held-out summary. diff --git a/docs/v0.3/r7/README.md b/docs/v0.3/r7/README.md new file mode 100644 index 0000000..687e085 --- /dev/null +++ b/docs/v0.3/r7/README.md @@ -0,0 +1,42 @@ +# R7 corpus artifacts + +This directory contains metadata and hashes only. Third-party source, dependencies, logs with absolute paths, and compiler outcomes do not belong here. + +- `repositories.json` is the registration frozen before eligibility inspection. +- `preflight-.json` is emitted by the read-only Git scanner. +- provisioning evidence is collected separately from clean base/target environments. +- `corpus-manifest.json` may be created only after exactly 30 candidates have complete evidence. + +From `extension/`: + +```powershell +npm run r7:corpus -- scan +npm run r7:corpus -- provision [prior-evidence.json] [eligible-limit] +npm run r7:corpus -- complete +npm run r7:corpus -- merge [...] +npm run r7:corpus -- freeze +``` + +Every output command refuses to overwrite an existing artifact. The scanner never checks out a revision or runs repository code. Provisioning and execution remain separate so a static scan cannot manufacture a passing test result. + +Provisioning uses at most three isolated native Docker-volume workspaces concurrently. Host bind-mounted dependency trees are forbidden because their Windows filesystem cost distorted setup time; the host provides only a read-only Git archive during seeding. This bound is a throughput setting, not a sampling rule; each container retains its own workspace, exact registered argv, resource limits, and network mode. + +For pnpm repositories, the controller installs the registered pnpm version into the candidate's Corepack cache, disables project-version substitution, creates `/work/.pureflow-bin` with an exact shell-free `mkdir` invocation, and enables a disposable shim there. This bootstrap is hashed separately from the unchanged registered install/test argv. Provision artifacts expose typed install/test booleans for diagnosis while retaining command output only as bounded hashes. + +Resume evidence is immutable input, never overwritten. The provisioner skips recorded targets, advances in ordinal order, and sizes each batch to the remaining eligible slots so it never executes a candidate after the tenth passing base/target pair. + +The optional `eligible-limit` lowers that ceiling when earlier registrations leave fewer than ten global corpus slots. It must be between 1 and 10 and follows an explicit prior-evidence input, which may be `empty-evidence.json`; this keeps the stop reason auditable. + +For a registered `npm run