Declare cited-source on the verification types (sw/unit/sys-verification) + a native 'named-test-exists' check — make requirement→test evidence drift-checked

[avrabe]

From applying the methodology on relay (test-level traceability work, pulseengine.eu#89). Two affordances would make the requirement → verification → actual test chain mechanically drift-proof, instead of prose-backed.

1. Declare cited-source on the verification types

cited-source (kind: file, sha256-stamped, drift-checked by rivet validate / rivet check sources) is a great mechanism — but it's only declared on requirement-ish types. On a verification artifact it's rejected:

INFO: [FV-RELAY-HAL-005] field 'cited-source' is not defined in schema for type 'sw-verification'
  ... declared fields [method, preconditions, priority, steps]

A verification artifact is exactly where a hash-stamped pointer to the test source file belongs: "this requirement is verified by the tests in crates/X/src/lib.rs, hash f5f98a…", and rivet validate flags drift the moment that file changes without a re-stamp — i.e., the evidence can't silently rot.

Ask: declare cited-source under fields: for sw-verification, unit-verification, sys-verification, sw-integration-verification, sys-integration-verification in the aspice/common preset. (It already round-trips cleanly — rivet check sources even computes the hash — it's just not declared there.)

2. A native check: a named-test step must actually run a test

A common verification shape is fields.steps[].run: "cargo test -p X some_test_name" to tie a requirement clause to a specific test. But cargo test -p X typo exits 0 with "0 passed" when the filter matches nothing — so a renamed/removed/typo'd test name silently keeps the requirement 'verified'. I worked around it in relay's gate script (require ≥1 test result: ok. N passed with N≥1 for a named-filter step), but it belongs in rivet so every project gets it for free.

Ask: a rivet check oracle (e.g. verification-evidence) that, for automated-test steps whose command names a specific test filter, asserts the named test exists / ran — or at least a documented convention + helper. Language-specific (cargo/pytest/…) test-name extraction is the hard part; even a "step names a filter but command-runner reports 0 ran → error" heuristic would catch the silent-drift case.

Why it matters

On relay, rivet validate is PASS with 113 warnings, many of the form "requirement should be verified by at least one verification measure" — requirements with no verifies edge at all — and the verifications that exist map to crate-level commands, not specific tests. Closing #1+#2 turns the trace from "prose says test T covers req R" into "rivet proves test T exists, ran, and its file hasn't drifted." That's the difference between a traceability matrix an assessor trusts and one they have to re-audit by hand.

Context: pulseengine.eu#89 (agent methodology retrospective), relay PR #221 (the test-level demonstration + gate stopgap).

[avrabe]

Triage from the issue-fixing routine: clean, two-part report — both asks are independently scoped, both have an obvious done-shape, neither has an explicit ## Acceptance Criteria block the routine can verify before opening a PR.

Premise confirmed against main:

• cited-source lives in schemas/common.yaml:381 (mapping with uri / kind / sha256 / last-checked, drift-checked by rivet check sources per REQ-066), declared only on the external-anchor type. In schemas/aspice.yaml the sw-verification type (line 242) declares exactly method / preconditions / priority / steps — no cited-source — and the same is true for unit-verification, sys-verification, sw-integration-verification, sys-integration-verification. So the INFO in the body (field 'cited-source' is not defined in schema for type 'sw-verification') is the schema's actual behaviour, and the field is declared once, just on the wrong types.
• The "filter matches zero → exit 0" gap on cargo test is real and not rivet-specific (also bites pytest -k, vitest -t, go test -run); the existing verification-execution type (the aspice link-types section confirms it's already a first-class target via result-of / part-of-execution) is where a "named-test step actually ran" oracle would naturally hang, since the verdict envelope already carries per-step output.

Coordination call before locking AC: this issue's part 1 lives in the same neighbourhood as #554's refinement and #555 (the verification-coverage diagnostic). Those are about whether a requirement has an inbound verifies from a test artifact; this is about whether that test artifact's cited-source can be hash-pinned to the actual test file. They're complementary, not overlapping — different layers of the same drift-checking story — so this one can ship independently, but worth a Refs: cross-link in the commit so the trace graph shows both layers landing in the same v0.x bump.

The two asks are also independently shippable; recommend two separate PRs, two separate AC blocks per "one PR per issue, keep PRs focused." Sketches matching #532's kill-criterion shape:

## Acceptance Criteria — part 1 (declare `cited-source` on verification types)
- [ ] `schemas/aspice.yaml` adds `cited-source` (same `type: mapping`, same shape as the `common.yaml:381` declaration on `external-anchor` — `uri` / `kind` / `sha256` / `last-checked`, `required: false`) under `fields:` for `sw-verification`, `unit-verification`, `sys-verification`, `sw-integration-verification`, `sys-integration-verification`. Same field definition referenced (or duplicated — pick one and document the convention in `docs/schemas.md`); the round-trip through `rivet validate` is already clean per the body.
- [ ] On a fixture matching the relay layout (verification artifacts with `cited-source: { uri: …, kind: file, sha256: f5f98a…, last-checked: … }`), the body's INFO (`field 'cited-source' is not defined in schema for type 'sw-verification'`) no longer fires. Golden-file test on `rivet validate --format json` on both the field-present and field-absent fixture.
- [ ] `rivet check sources` (REQ-066 — the existing drift-checker) walks the new `cited-source` fields on verification artifacts the same way it walks them on `external-anchor` today. Verified by a fixture whose `sha256` deliberately mismatches the named file — drift is reported with the same diagnostic shape as the existing case.
- [ ] `rivet validate` on the rivet repo unchanged (rivet's own verification artifacts don't yet use `cited-source` — confirm before merging; if any do they'll be re-validated against the new declared field, which should be the no-op case).
- [ ] `cargo test -p rivet-core -p rivet-cli` green; `cargo fmt --check` / `clippy --all-targets -- -D warnings` clean.
- [ ] Commit trailers: `Implements: REQ-066` (or a new REQ if you'd rather track "cited-source on verification artifacts" separately) + `Refs: #556, pulseengine.eu#89, pulseengine/relay#221`.
- [ ] **Explicitly out of scope (file separately):** widening `cited-source` to *all* artifact types (richer scope-decision call — keep this one targeted at the verification-evidence use case the body cites); per-step `cited-source` on individual `steps[]` entries (also useful but a deeper schema change); part 2 below.

## Acceptance Criteria — part 2 (`verification-evidence` oracle: named-test-filter actually ran)
- [ ] New `rivet check verification-evidence` (or extend an existing oracle — the body says "even a heuristic"; the existing `verification-execution`/`verification-verdict` types already capture the data shape, so an oracle that reads them is the cleanest seat). For each `*-verification` artifact whose `steps[].method` is `automated-test` and whose `steps[].run` names a specific test filter (the language-specific parsers are the hard part — start with the cargo `cargo test -p X test_name` shape; pytest `-k` / vitest `-t` / go `-run` flagged as TODO with an explicit "unknown runner" pass).
- [ ] Heuristic the body recommends ships first: when a step has a named filter (per the parser above) and the linked `verification-execution`'s recorded stdout shows `0 passed` / `0 ran` / equivalent runner-specific "matched nothing" pattern, emit an ERROR (not a WARN — the silent-drift case is exactly what should block release-gating). When no execution is linked, emit INFO (don't crash the validation pass on absent evidence).
- [ ] Per-runner test-name extraction is in scope only for `cargo test` in this PR (the body's repro shape). Other runners flagged as "unknown — passes through unchecked, file a follow-up to add a parser". Documented in `docs/oracles/verification-evidence.md`.
- [ ] On a fixture mirroring the relay gate-script case (a `sw-verification` step whose `cargo test -p X typo` runs but matches 0), `rivet check verification-evidence` exits with the diagnostic count = 1 on the typo'd name and exits clean on the correctly-named filter. Golden-file on both directions.
- [ ] `rivet validate` posture: oracle is opt-in via the `rivet check` command (matches the existing `rivet check sources` pattern from REQ-066); not added to `validate`'s default pass in this PR. Promotion-to-default is a follow-up policy call (matches the WARN→ERROR ramp pattern from #555 / #550).
- [ ] `cargo test` green; lint clean; `rivet validate` on the rivet repo unchanged.
- [ ] Commit trailers: `Implements: REQ-NNN` (file a new REQ — e.g. "verification-evidence oracle: named-test-filter must actually run") + `Refs: #556, pulseengine.eu#89, pulseengine/relay#221`.
- [ ] **Explicitly out of scope (file separately):** parsers for non-cargo runners (pytest / vitest / go test — separate small PRs each, one parser per PR); MC/DC coverage assertion (`rivet check verification-evidence` only checks *existence + ran*, not coverage — that's #554's ask 2 territory); auto-discovery of test artifacts from `cargo test --list -- --format json` (also #554's ask 2 — heavier slice).

The split keeps each PR reviewable and lets part 1 ship even if part 2's per-runner parser-list discussion stalls.

Process side-rail (carried from every recent triage thread): https://pulseengine.eu/blog/ is still HTTP 503 (re-verified at the start of this run via curl -I — HTTP/2 503 — same long-standing expired-TLS / site-infra symptom flagged across the last ~25 triage threads: #420, #468, #456, #436, #431, #422, #479, #482, #488, #490, #500, #508, #509, #516, #522, #523, #530, #532, #538, #543, #546, #547, #548, #549, #550, #552, #554, #555). The binding workflow guidance can't be consulted, so no PR will land from this triage regardless. Comment is the AC ask + #554/#555 coordination note only.

Not implemented this run.

---

Generated by Claude Code — issue-triage agent run 2026-06-21.

---

Generated by Claude Code

[avrabe]

Partially delivered in v0.18.0: REQ-225 / #565 declared cited-source as a common base-field, so it's now usable and drift-checked on the verification types (sw/unit/sys-verification) — the primary ask. Remaining scope for this issue: the native named-test-exists check (verify a referenced test name actually exists). Narrowing this issue to that follow-up; relabel/retitle at will.