Skip to content

docs(readme): catch both READMEs up to fix-record v2, and publish its evidence in the Action comment - #220

Merged
arthurpanhku merged 2 commits into
mainfrom
claude/readme-update-review-z0lcw5
Sep 4, 2026
Merged

docs(readme): catch both READMEs up to fix-record v2, and publish its evidence in the Action comment#220
arthurpanhku merged 2 commits into
mainfrom
claude/readme-update-review-z0lcw5

Conversation

@arthurpanhku

@arthurpanhku arthurpanhku commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Summary

A review of what landed since the READMEs were last touched (#216, #217, #218) turned up three stale front-page claims and one real gap in the Action.

Fix-record samples showed v1 output. Since #218 both producers issue v2, so renderFixRecord prints introduced with the gate it was judged under and the outcome. A record printed today matched neither the old sample nor a v1 one, which carries its own "issued under v1 rules" note instead. Samples now show what the command prints, and the paragraph beside the hero block states the claim v2 actually added: a repair that removes an eval and introduces an SQL injection is recorded as regressed and does not verify — and a record whose issuer never looked does not verify either.

The Action published the proof without the regression evidence. action.yml built its pull-request comment from remainingTargets, coverage and checks, and never read after.introduced, gate or outcome. So the surface that puts the record next to the diff showed a regressed repair the same way it shows a clean one — targets: 1 → 0 under a NOT VERIFIED header, with the reason buried in verdict prose. It now names what the re-scan saw that the first scan did not, with the gate and the outcome; introduced findings are listed with the severity the gate itself read (securitySeverity first, then the SARIF level), because a finding must not look milder in the list than in the rule that blocked on it. A v1 record gets the note rather than the fields.

The severity gate needed no change: a regressed v2 record already re-derives to verified: false and fails the job. What was missing was the evidence for it.

The headless verification envelope from #216 was undocumented. The README already argues that "no findings" without coverage is not an answer; the surface where that costs most is the one with no human reading it, and it now carries coverage and any fix record's path in json, stream-json and the MCP dvalin_run_task result. Documented with the jq gate a CI job would actually write.

Test counts were stale and disagreed with each other — a 442 / 442 badge over a "518 core tests · 70 files" section.

The contributors table was missing two people. git log on main carries commits from Samran Asif (@webdevsamran, #211) and @dchaudhari7177 (#213, #212) that the table did not list. Names and handles taken from the commits and pull requests themselves.

Every change is mirrored in README.zh-CN.md, except the MCP record description, which that file condenses and does not carry.

Testing

  • npx vitest run — 584 passing across 74 files, none skipped (the counts the READMEs now quote).
  • npx tsc -p tsconfig.json --noEmit — clean.
  • New tests/actionSummaryRender.test.ts (6 tests) extracts the node -e script out of action.yml and runs it against records built by this repo's own buildFixRecord + verifyFixRecord, asserting the comment a reviewer would read for v2-clean, v2-regressed, v2-undetermined, v1, and a record edited after issue. It also pins the invariant that the script stays free of the single quote that would break its shell wrapper — the comment is JavaScript inside YAML inside a shell script, and nothing else typechecks it.
  • Mutation-checked: pointing the schema comparison at a version that never matches fails the three v2 cases.

Security and AI Governance

  • This change does not expand file, shell, network, model, or approval permissions.
  • If it changes agent behavior, prompts, policy, providers, audit logging, or release/build security, I updated the relevant governance evidence in docs/. — none of those change here; the Action edit affects only how an already-computed verdict is displayed, and docs/spec/FIX-VERIFICATION.md already gained FV-10a/FV-10b/FV-12a in feat(fix-record): add the v2 shape and version-aware re-derivation (1/2) #217.
  • If it introduces a new model/provider/tool or new data flow, I completed docs/governance/AI-CHANGE-IMPACT-ASSESSMENT.md. — no new model, provider, tool, or data flow.

Notes

Two things left alone on purpose:

  • The workflow example still pins arthurpanhku/dvalincode@v0.18.0. The v2 records and this rendering ship together in the next release, so the pin is consistent as it stands; bumping it is a release-cadence decision.
  • The closing blockquote on the Action comment ("attests that these findings were gone and these checks were observed to pass") is unchanged — still accurate, and for a v2 record the introduced line above it already carries the stronger claim.

The contributors table also still lists @shivasb42 and @adity982, who have no commits in git log; the section credits issues and ideas as well as code, so they were left in place rather than pruned.

One correction worth flagging for anyone reading the branch history: the first commit put 572 passing · 6 skipped · 73 files in the READMEs. That measurement came from a run against a still-installing node_modules; the skips were an artifact of it. The second commit corrects all six spots to 584 / 74.

🤖 Generated with Claude Code

https://claude.ai/code/session_015f83ci81a7cKYf253ckwTb

…arness envelope

Three claims on the front page went stale as #216, #217 and #218 landed.

The fix-record sample showed v1 output. Since #218 both producers issue v2, so
`renderFixRecord` prints `introduced` with the gate it was judged under and the
`outcome` — a record printed today matches neither the old sample nor a v1 one,
which carries its own "issued under v1 rules" note instead. The sample now shows
what the command actually prints, and the paragraph beside it states the claim
v2 added: a repair is a change, so a fix that removes an `eval` and introduces
an SQL injection is recorded as `regressed` and does not verify, and a record
whose issuer never looked does not verify either. That last part is the point of
the three-state field, and it was the strongest thing on the page left unsaid.

The headless section did not mention the `verification` envelope from #216. The
README already argues that "no findings" without coverage is not an answer; the
surface where that costs most is the one with no human reading it, and it now
carries coverage and any fix record's path in `json`, `stream-json` and the MCP
`dvalin_run_task` result. Documented with the jq gate a CI job would actually
write.

Test counts were stale and disagreed with each other — a 442/442 badge over a
"518 core tests · 70 files" section. `npm test` reports 572 passing, 6 skipped,
73 files; both numbers now say that, in both languages.

The Action's fix-record comment block is left as it is: action.yml renders the
record without reading `introduced`, `gate` or `outcome`, so the sample is
accurate to what the runner posts today. The gap is in the Action, not the
README, and closing it is a code change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015f83ci81a7cKYf253ckwTb
The pull-request comment was built from `remainingTargets`, `coverage` and
`checks` and never read `after.introduced`, `gate` or `outcome`. Since #218 both
producers issue v2, so the surface that publishes the proof next to the diff was
showing a regressed repair the same way it shows a clean one -- "targets: 1 → 0"
above a NOT VERIFIED header, with the reason buried in the verdict prose.

The comment now names what the re-scan saw that the first scan did not, with the
gate the verdict was reached under, and the outcome. Introduced findings are
listed with the severity the gate itself read (`securitySeverity` first, then the
SARIF level) rather than the raw level, because a finding must not look milder in
the list than in the rule that blocked on it.

A v1 record gets the note instead of the fields. It was judged under rules that
never asked about regressions, and rendering it as though it asked and found
nothing is the one reading the record format spends its credibility to prevent.

The severity gate needs no change: a regressed v2 record already re-derives to
`verified: false` and fails the job. What was missing was the evidence for it.

Covered by tests that run the script out of action.yml against records this
repo's builder produced, since the comment is JavaScript inside YAML inside a
shell script and nothing else typechecks it -- including the invariant that the
script stays free of the single quote that would break its shell wrapper.

README samples in both languages updated to the output the step now produces,
with a regressed comment shown beside the verified one, and the test counts
refreshed against a full local run (584 across 74 files).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015f83ci81a7cKYf253ckwTb
@arthurpanhku
arthurpanhku merged commit f603426 into main Sep 4, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants