Skip to content

docs: acs-verdict-evidence-v1 predicate (draft) - #297

Open
zhaoxinghua09-cell wants to merge 1 commit into
GenAI-Security-Project:integrationfrom
zhaoxinghua09-cell:docs/acs-verdict-evidence-v1
Open

zhaoxinghua09-cell wants to merge 1 commit into
GenAI-Security-Project:integrationfrom
zhaoxinghua09-cell:docs/acs-verdict-evidence-v1

Conversation

@zhaoxinghua09-cell

Copy link
Copy Markdown

What

Draft predicate docs/predicates/acs-verdict-evidence-v1.md (v1.0.1-draft), the evidence format for the acs-conformance verdict record discussed in #33.

This addresses the two points raised in #33 (comment 6243997020):

  1. No self-asserted lift. Removes runner-written lift_state/counter_signature; scope is derived by the verifier from a peer-authored signed statement (provenance.peer_artifact). Absent that artifact the record reads SELF (co-signature is the default, non-repudiation only).
  2. Quarantine record gets its own envelope. Adds integrity.payload_digest over its own declared fields and a defined expiry_disposition for review_by.

Sample digests are reproducible on a clean checkout via python run_demo.py --selftest (kit: zhaoxinghua09-cell/agent-evidence-kit). The quarantine row points at the MODIFY-composition gap opened as #296.

The predicateType URI in the sample resolves on merge (doc is this PR's payload).

@astrogilda

Copy link
Copy Markdown

@narko4u, thank you for catching both holes in #33: the reused peer statement and expiry_disposition outside the quarantine digest. Vectors commit 617cf55 now puts the peer's verdict inside its signed statement, and #298 refuses each case.

@zhaoxinghua09-cell, two more things turned up in those tests. The kit at e2f85f6 writes no signature, so its samples cannot pass your verification steps yet. The predicateType URI names the main branch, while this PR targets integration, so it stays 404 after merge.

@rocklambros

Copy link
Copy Markdown
Collaborator

Hey! Quick note on the PR format. You've got "discussed in #33" in your description, but CONTRIBUTING.md needs one of these formal keywords before the issue number:

Just swap "discussed in" for "Part of" and you're good. Everything else looks solid.

Thanks for the work on the verdict evidence predicate!

@rocklambros

Copy link
Copy Markdown
Collaborator

Hi Steven, OWASP requires a sign-off on every commit, and ef2745c on this PR doesn't have one. The dco check we're adding in PR #407 will fail it.

Please run these commands:

git checkout docs/acs-verdict-evidence-v1
git fetch https://github.com/GenAI-Security-Project/agent-control-standard.git integration
git rebase --signoff FETCH_HEAD
git push --force-with-lease

They add a Signed-off-by: line to your commit and move your branch onto the latest integration. On a copy of your branch they ran with no conflicts. Thanks!

@rocklambros rocklambros mentioned this pull request Nov 1, 2026
11 of 15 tasks
@rocklambros

Copy link
Copy Markdown
Collaborator

Hi Steven, this comment replaces my earlier one. The dco check is now required on integration, so this pull request can't merge until the check passes.

Nothing is wrong with your work. The check looks for a sign-off, and OWASP requires one on every commit.

A sign-off is a Signed-off-by: line in the commit message. It states that you have the right to submit the work. It is not a GPG or SSH signature, so you don't need any keys.

The one commit on this pull request, ef2745c, doesn't have the line. Please run these steps on your computer.

  1. Set your name and email. You only need to do this once. Git puts them in each commit, and they are public.
git config --global user.name "Your Name"
git config --global user.email "you@example.com"
git config user.name

The last command prints your name, so you can check it.

  1. Go to your branch and connect it to the project. If Git says upstream already exists, that's fine. Continue with the next command.
git checkout docs/acs-verdict-evidence-v1
git remote add upstream https://github.com/GenAI-Security-Project/agent-control-standard.git
git fetch upstream
  1. Add the sign-off to your commit and push.
git rebase --signoff upstream/integration
git push --force-with-lease

If Git says your branch has no upstream branch, run git push --force-with-lease origin HEAD.

These steps ran on a copy of your branch with no conflicts.

If the rebase does stop on a conflict, open each file Git lists and fix those lines. Then run git add <file> and git rebase --continue. To stop and go back to where you started, run git rebase --abort.

  1. Check that it worked.
git log -1 --format=%B

The output should end with Signed-off-by: and your name and email. The dco check runs again by itself after you push.

The full guide is here: How to sign your commits.

Thanks!

@drk-ram drk-ram left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at ef2745c288f64f96e727e67643687e204463ffcf. I merged origin/integration (bba1b846fbe7511b0b6f876c61b1fe91b27b07da) into a scratch worktree (clean merge, nothing committed) and ran the suite on Windows, Python 3.11.9, because the checks shown on this PR are from 18 October and predate the nav and link guards.

Blocking

1. The merged tree fails two tests that this file causes. python -m pytest tests gives 1182 passed, 13 failed, 20 errors. Eleven of the failures and all 20 errors are in areas this PR does not touch (symlink handling, patch-context re-derivation, subprocess environment, intake-workflow tests) and I did not chase them. Two are about this page:

  • tests/test_doc_links.py::test_no_doc_links_to_a_github_repository_outside_the_allowlist reports docs\predicates\acs-verdict-evidence-v1.md:7: links to probityai/agent-evidence-vocabulary, not on the allowlist.
  • tests/test_site_nav.py::test_every_pane_docs_url_is_built_and_every_built_page_is_in_the_pane_or_declared reports that predicates/acs-verdict-evidence-v1/index.html is built but not in the pane, and asks for a titled nav entry in mkdocs.yml or an extra.acs_nav.out_of_nav_prefixes entry for the directory.

The fix for each is small (a nav entry or declared prefix; an allowlist entry or a link to a document in this repository), but the PR cannot merge without it.

2. Process items already raised in this thread are still open: the dco sign-off on ef2745c and the discussed in #33 wording (Part of #33). The branch has to be rebased onto integration for any of that to take effect.

3. The text does not yet contain the two fixes the #33 thread says it contains. In the thread @astrogilda credited @narko4u with two holes: a reusable peer statement, and expiry_disposition outside the quarantine digest. Reading the file at this SHA, both are still there:

  • Quarantine digest. The digest covers vector_id, status, reason, quarantine_ref, quarantine_gap, owner, review_by, excluded_from. expiry_disposition is not in the list, so it can be edited without breaking quarantine_digest_mismatch, which is the property the sentence below the list claims. status and reason are listed but defined nowhere in the quarantine table.
  • Peer statement. The statement is described as naming the vector, the requirement and the evidence artifact. It does not carry the verdict being witnessed. A statement that a peer checked requirement R on vector V against deployment D can therefore be attached to any verdict record for V, including one whose observed or verdict the peer never saw. The thread says the vectors commit now puts the peer's verdict inside its signed statement. The predicate has to say so, and step 9 has to compare it.

Spec defects to fix in the same pass

  1. witness_scope contradicts itself. The field table says every field is required in the record and witness_scope is one of them. The "Deriving witness_scope" section says it is "computed by the consumer, never asserted by the runner", and the Fields section says a record "must not carry any field by which the runner asserts its own scope". A runner-written witness_scope is exactly that field, even though step 9 later checks it. Either remove it from the record and make it a verifier output, or rename it to a claimed value that step 9 compares and say that plainly.
  2. The derivation table is incomplete. Rows cover equal/null, equal/present (peer, external), unequal/null and any/invalid. There is no row for hosting_operator_id != deployment_operator_id with a verifying peer_artifact, and the unequal/null row says the record is "not a valid SELF record" without saying what it is. Both operator IDs are runner-written strings in the same signed payload, so "comparable identifier" needs a definition (issuer-qualified, from a pinned registry) or equality is self-asserted. The row also says "rejected at verification step 9", but rejected is a verdict value and step 9 emits contradicted. That is an unfortunate collision.
  3. vector_id is never tied to the subject. vector_id is the first 16 hex characters (64 bits) of a SHA-256 of "the vector input's canonical form". Step 7 compares the subject digest to the canonical vector input, and no step checks vector_id == subject.digest.sha256[:16]. Add that check, or drop the truncated field and use the full digest.
  4. "Canonical" is undefined. payload_digest, vector_id and provenance.peer_artifact all say "canonical bytes"; the one definition given is "keys sorted, no whitespace, UTF-8". That is not RFC 8785 and leaves number serialization, escape forms and key ordering (code point vs UTF-16 code units) open, so two conforming verifiers can disagree on a digest. Name RFC 8785 JCS. draft-durga-agenttrust-trace-verification-00 specifies JCS (RFC 8785) canonicalization together with fail-closed verification, which is the combination this predicate needs. Separately, peer_artifact should be the SHA-256 of the DSSE payload bytes exactly as signed (or the PAE), not a re-canonicalization of a signed statement. A signature covers specific bytes.
  5. Step 3 mislabels an unpinned key. untrusted_key is mapped to contradicted. An unknown signer is not_established; contradicted should mean the evidence disproves the claim. The intro's three-valued outcome only works if those are kept apart.
  6. anchor has no version binding. "Repo-relative, resolvable anchor" does not say relative to which spec release or commit. A heading rename silently changes what a verdict was about. Carry the spec version (or tree digest) in the record and in payload_digest_fields.

Smaller

  • The predicate URI is blob/main/.... @astrogilda already noted it is 404 after a merge to integration; it also stays 404 until the next promotion reaches main (#464 makes that explicit), and a mutable branch URL is a weak identifier for something signed. Use the published site URL or another persistent URI and keep v2 on a new path as the Versioning section already does.
  • v1.0.1-draft is described as breaking relative to v1.0.0-draft and becomes v1.0.0 on acceptance. A reader will take that as a lower version number than the draft whose breaking changes it contains. Pick one scheme.
  • The PR body says the sample digests reproduce with run_demo.py --selftest; the thread says the kit at e2f85f6 writes no signature, so its samples cannot yet pass the verification steps. That should be stated in the document's status line.

The design direction (scope derived by the verifier, never asserted; a rejection weighted like a pass; quarantine with an expiry disposition) is sound and worth landing. It needs a pass for the points above first.

Raghuvar Karthik Durga (@drk-ram)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

4 participants