docs(research): record the fixture provenance audit and the boundary fixture convention

[doublewhy]

Summary

Part of #1344. The PR adds the audit record and the convention the issue asks for. It does not complete #1344: most divergences it registers need production changes, and several of those involve release-admission or semantics choices.

• docs/research/fixture-provenance/inventory.md lists each boundary class the audit examined where a test stands in for something this repository does not control: GitHub, libvirt, the guest appliance, network services and supply chain, command-line tools, the SQLite store and Ground Control client, MCP and filesystem and time, backend manifests, runtime snapshots, and release and experiment evidence. For each boundary it records the consumer, the fixture, whether the shape was captured or inferred, what the real producer returned, a verdict and a blast-radius rating. The inventory is not exhaustive: the record's limits name stand-ins the audit did not examine, such as the pinned z3 solver binding.
• docs/research/fixture-provenance/captures.md keeps the observed responses: GitHub REST and gh, libvirt 12.7.0 (including the TechVault readback and missing-object codes behind LV-2 and LV-3), guest serial output from a booted appliance, macOS and Ubuntu curl, public OCI registries, conftest 0.68.0, git, SQLite 3.50.4 and the MCP stdio server. Every observation was made on 2026-10-09, except the NS-5 registry manifest, which was read on 2026-10-10. The store, time, MCP and runtime-snapshot entries are marked as summaries, not raw responses.
• docs/research/fixture-provenance/index.md has the method and its limits, results per class and a divergence register. Entries give the fixture location where one exists, the asserted shape, the real observation, the production code that relies on it, a recommended fix and a status. The record also lists what remains, including the unverified boundaries and the boundaries not examined, and the findings that apply to sibling repositories.
• docs/explain/reference/fixture-provenance.md is the convention for new boundary fixtures: capture before encoding, keep the capture where the tests use it, prove the fake reaches the real branch, and keep captures current. docs/explain/reference/README.md indexes it.

The PR stands alone: no file in it points at a file that exists only in an open pull request. #1453 proposes the libvirt capture as test data, and captures.md records the same readback and error codes itself.

Clear-cut divergences are fixed in separate pull requests, which the register cites: #1453 (libvirt TechVault fakes, test only), #1455 (conftest error output read as no failures, for #1454), #1459 (runtime-owner lease fork test, test only) and #1464 (workflow compensation order, for #1462). #1415 already covers the gh release upload divergence (#1414).

Requirement UIDs

• None. Research record and testing convention with no requirement scope.

Related Issues

Closes #1460

ADR Impact

• None.

Changes

• docs/research/fixture-provenance/index.md (new): method, limits, results by class, divergence register, remaining work, findings for sibling repositories.
• docs/research/fixture-provenance/inventory.md (new): per-boundary inventory for the ten classes.
• docs/research/fixture-provenance/captures.md (new): observed responses, trimmed. Signed URL query strings are removed.
• docs/explain/reference/fixture-provenance.md (new): the boundary fixture convention.
• docs/explain/reference/README.md: index entry for the convention.

The second commit, e4ef92d0, makes the record stand alone:

• captures.md records the LV-2 and LV-3 readback (<memory unit='KiB'>131072</memory> with currentMemory) and the missing-object codes 42 and 43. The convention names the existing implementations/python/tests/data/ directory and describes the pattern without citing files from test(libvirt): take TechVault fake shapes from a captured libvirt readback #1453.
• Labels follow the record's own definitions. Two libvirt rows that said captured are inferred. nwfilterDefineXML is unverified, with the libvirt function that rejects collisions cited. LV-4 no longer lists UUIDString(), which the inventory rates a match in effect. The evidence-class count names the three boundaries its table leaves out.
• The register adds fixture locations for ST-1 to ST-4, TM-2 and MF-3, uses present tense for CL-1 and TM-1, and says "the divergences found" instead of "every divergence found".
• captures.md states the observation date, the probe environment and the versions the audit recorded. For NS-3 and the GITHUB_SHA row it names the source artifact (ID 11210897140), which expired at 05:29Z on 2026-10-09.

The third commit, 0a528fcf, corrects one rating and qualifies the record's coverage:

• ASR-502: Backend Conformance Suite And Fixture Corpus #6 is withdrawn. implementations/python/tests/test_issue_1226_release_evidence_cli.py:56-70 gives GITHUB_SHA and GITHUB_WORKFLOW_SHA distinct values on purpose: its source_sha != workflow_sha assertion shows that release_identity does not copy one variable into both fields, and docs/decisions/package-artifacts/issue-1226-preflight.md:84-85 keeps the two revisions as separate identities. The inventory rates that row match, so the GitHub class has 6 divergences, not 7.
• The opening paragraph claims only the boundaries the audit examined. A new limits item says the inventory is not exhaustive and names examples it did not examine: the pinned z3 binding (z3.Solver.check and reason_unknown are patched in implementations/python/tests/test_scenario_satisfiability.py), the solver's time.monotonic_ns fakes, the TechVault host probe stub and podman. "What remains" lists them apart from the unverified boundaries.
• NS-6 is rated n/a, since nothing reads its fields, so the network class has 7 divergences, not 8. The inventory defines match in effect and the qualified labels (real, protocol-limited, real, ASCII-only, captured once). The two LilRAE rows that said "captured from the producer source" are labelled by how their fixtures were made: the hand-built exports are inferred, and the snapshot projection has no fixture.
• Add Sphinx documentation site with Furo theme #1 shows the nested REST shape ("base": {"ref": "dev"}), and Add MCP server with 14 tools for SDL understanding, authoring, and inspection #2 marks closes_lines as derived from body. NS-5 names its manifest: the linux/amd64 manifest of the Docker Official Image tomcat:latest (sha256:594377d0...) lists one 32-byte layer three times. The sibling-repository findings mark libvirt code 62 as taken from libvirt source, and the lease close() range is 217-231.

All three commits were replayed onto 35122105 on 2026-10-10, so that the first commit's message, which the squash carries to dev, says "the test boundaries the audit examined" instead of "every test boundary". The first two commits keep the trees of a44c84a4 and 62899a8b.

Test Plan

• Unit tests pass
• Integration tests pass if applicable
• Full completion suite required in CI before merge
• No coverage regression

Evidence for the boxes above: CI run 38071594250 on head 0a528fcf ran 21 jobs, and all 21 passed, including canonical / test-shard (0) to (3), canonical / integration, integration-docker, canonical / coverage-reduce, canonical / gate and verify. sonar passes on 0a528fcf (run 38071594250, re-run after SONAR_TOKEN was replaced on 2026-10-11), and SonarCloud's analysis of that head passed the quality gate. Docs run 38071593919 built, and CodeQL (run 38071590503), GitGuardian, the title guard and the body guard pass. No code or tests change, so no test module was run locally.

Local checks, all on 0a528fcf:

• nox -s verify-fast-feedback -- --base-rev origin/dev passed: every hygiene stage with files to check, including gitleaks, and every policy stage it runs, including policy / identity cutover. It skips requirement governance by design, and lint and targeted pytest because no Python file changes.
• make policy passed 15 stages. The session skips requirement governance by design.
• nox -s lint passed its 4 ruff stages.
• nox -s docs-local does not cover these files. Its Vale stage reads only the public entrypoints and docs/public, and Sphinx builds only docs/public (tools/nox_support/test_lanes.py:280-301; "Vale Enforcement" in docs/explain/reference/documentation-style-guide.md). So the repository-pinned Vale 3.15.2, installed through tools/vale_tool.py, was run directly. vale --config=.vale.ini on the five files reports 4 errors. Two are already on dev in docs/explain/reference/README.md: the reading level (16.69 on dev, 15.85 here) and the heading "Reference Notes". The other two are heading-case false positives on the proper nouns SQLite and Ground Control, in two headings this PR adds (inventory.md:175, captures.md:237); the rule's exception list does not contain those names.
• A script resolved each of the 285 path:line references in the four record files, including :N continuations, against 35122105. Each names a file that exists there and stays within its length.

The observations were made on 2026-10-09. GH-1, GH-2 and GH-5 (GitHub REST), LV-1 and LV-4 (libvirt), NS-1 (curl), NS-2 (registries) and CL-1 (conftest) were captured again while this record was written. The SQLite store (ST-1 to ST-4), time (TM-1, TM-2) and runtime-snapshot (RS-1 to RS-3) probes were rerun against the same commit. The libvirt source citations were read at libvirt's v12.7.0 tag: virNWFilterObjListAssignDef in src/conf/virnwfilterobj.c reports VIR_ERR_OPERATION_FAILED (9) on a name or UUID mismatch, and nwfilterLookupByName in src/nwfilter/nwfilter_driver.c reports VIR_ERR_NO_NWFILTER (62).

Checked on 2026-10-10 for this revision: gh api repos/OpenRAE/rae/pulls/25 --jq '{base: {ref: .base.ref}, body, number}' returns {"base":{"ref":"dev"},"body":null,"number":25}, and gh pr view 1413 --json baseRefName,closingIssuesReferences,body with the jq expression in captures.md returns the GH-2 object. The NS-5 manifest was read anonymously from public.ecr.aws/docker/library/tomcat, and HEAD requests to registry-1.docker.io returned the same index and linux/amd64 manifest digests. The duplicate-digest check at tools/tooling_artifact_policy_oci.py:102-108 fails that layer list.

Ground Control Checks

• Repository policy command passes: make policy on 0a528fcf, as above.
• Pre-push review: the captures carry no credentials, tokens or signed URL query strings (the gitleaks and private-key stages pass on the five files), and every path:line reference resolves at 35122105.

Traceability

• IMPLEMENTS: (none)
• TESTS: (none)
• Audit issue: Audit the test suite for fixtures that assert shapes the real system never produces #1344. Fix pull requests: test(libvirt): take TechVault fake shapes from a captured libvirt readback #1453, ci(policy): fail the repository policy gate when conftest cannot evaluate its policy #1455 (fix(policy): fail the repository policy gate when conftest cannot evaluate its policy #1454), test(runtime): prove a lease closed in a forked child keeps the parent's locks #1459, fix(runtime): order workflow compensation by completion instant, not timestamp text #1464 (fix(runtime): order workflow compensation by completion instant, not timestamp text #1462); related: fix(release): name the repository when publish-github uploads release assets #1415 (fix(release): name the repository when publish-github uploads release assets #1414).

Checklist

• Code follows the project coding standards
• FM: not applicable, no semantic change (documentation only); no executable or runtime adoption is claimed.
• No production module, test, published schema, research capture under docs/research/specification-coverage/ or docs/research/formal-semantic-validation/, or other evidence-bound file changes.
• PR title is a Conventional Commit (release-please derives the version and CHANGELOG.md from it)

Documentation

Updated: docs/explain/reference/README.md. Added: docs/research/fixture-provenance/ and docs/explain/reference/fixture-provenance.md. Both sit outside docs/public/, so the hosted Sphinx build does not include them.