diff --git a/CHANGELOG.md b/CHANGELOG.md index f2db67d..d8bc671 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,9 +12,10 @@ Why: keep static guard releases auditable while the package is still alpha. sanitized configuration errors; no raw pattern or context text is emitted. - Content-bound optional `agent-policy` audit-event references with a canonical-JSON, profile-bound, public-safe digest. Producers require a - caller-designated repo-local JSON event and explicit profile; maintainer review - and event-schema validation remain external. Consumers require the separately - supplied event and reject missing, malformed, or replaced content. Audit-event + caller-designated repo-local JSON event and the recognized + `agent-policy.audit_event.v1.1` profile. Producers and consumers validate that + profile's published event shape and reject unsupported profiles, malformed + events, or replaced content. Maintainer review remains external. Audit-event binding uses report and manifest v2; the released v1 schemas remain unchanged and their path-and-role references remain readable as explicitly unbound legacy metadata. The event body remains outside the diff --git a/README.md b/README.md index 444c042..d0b824d 100644 --- a/README.md +++ b/README.md @@ -351,15 +351,15 @@ command only after that policy is reviewed and committed. **Optional reviewed audit event.** To record a companion `agent-policy` audit event, add the same `--agent-policy-audit-event ` and -`--agent-policy-audit-event-profile ` options to both the -`report` command and the standalone `evidence-pack manifest` command. Generate -both artifacts again after the producer has written the repo-local JSON event -and a maintainer has reviewed it. The manifest records only a sanitized -repository-relative path and a profile-bound canonical-content digest, not the -event body. Audit events select evidence v2; event-free reports stay v1. If the standalone manifest is present, the public bundle consumer -requires it to match the manifest embedded in the report. It also requires the -event path and profile again to verify the binding. Keep the event outside -`.agent-guard/evidence`: the seven-file public bundle allow-list rejects it. +`--agent-policy-audit-event-profile agent-policy.audit_event.v1.1` options to +both producer commands. Regenerate both artifacts after a maintainer reviews +the repo-local JSON event. The manifest records a sanitized relative path and +profile-bound digest, never the body. Producer and consumer validate the +recognized event shape; this checks semantics, not who approved the event. +Events select evidence v2; event-free reports stay v1. A standalone manifest +must match the embedded one, and consumers require the event and profile again. +Keep the event outside `.agent-guard/evidence`; the bundle allow-list rejects +it. The following optional PR review command is available in `0.3.0`: @@ -604,11 +604,11 @@ detect MCP tool-poisoning behavior, or act as an MCP runtime security validator. MCP 2026-07-28 protocol/runtime/OAuth changes do not justify runtime execution or live OAuth validation. No changelog item directly invalidates the current static committed-config labels, so this update does not change their taxonomy or code. With `--evidence-pack-manifest`, -it embeds a public-safe artifact handoff manifest for pull request review. Add -`--agent-policy-audit-event ` and -`--agent-policy-audit-event-profile ` to bind a reviewed repo-local -companion `agent-policy` audit event without embedding its body. Consumers must -receive that event separately and verify it with the same expected profile. +it embeds a public-safe artifact handoff manifest for pull request review. Use +`--agent-policy-audit-event ` with profile `agent-policy.audit_event.v1.1` +to bind a reviewed event without its body. Consumers require that event again; +arbitrary JSON objects and unsupported profile labels fail closed before +binding verification. Read `recommended` as the reviewed static evidence baseline, not as the full pin-integrity profile. The recommended preset can emit digest and context-lock diff --git a/docs/evidence-contracts.md b/docs/evidence-contracts.md index c0be359..9fd2a31 100644 --- a/docs/evidence-contracts.md +++ b/docs/evidence-contracts.md @@ -169,9 +169,16 @@ profile to both producers. Events select v2; event-free reports stay v1. The manifest records a sanitized repository-relative path and a profile-bound digest. `agent-guard` reads and canonicalizes the bounded event JSON locally to compute that binding, but never embeds the event -body. The consumer requires the event separately and fails closed when the -event is missing, malformed, supplied under a different expected profile, or -changed. The event itself is not part of the fixed seven-file public bundle. +body. The only recognized profile is `agent-policy.audit_event.v1.1`; producer +and consumer validate its required fields, exact top-level and decision fields, +decision enums, and bounded optional strings against the published +[agent-policy v0.1.11 schema](https://github.com/yui-stingray/agent-policy/blob/v0.1.11/src/agent_policy/schemas/agent-policy.audit_event.v1.1.schema.json). +Canonicalization also rejects strings that cannot be encoded as valid UTF-8, +including escaped lone surrogates, before computing a digest. +The consumer requires the event separately and fails closed when the event is +missing, malformed, outside that profile schema, supplied under a different +expected profile, or changed. The event itself is not part of the fixed +seven-file public bundle. The binding does not protect an attacker who can replace both the evidence manifest and the event; use a signature, attestation, or immutable trusted storage for that threat model. diff --git a/docs/quickstart-existing-repo.md b/docs/quickstart-existing-repo.md index 56f0cc7..51ba150 100644 --- a/docs/quickstart-existing-repo.md +++ b/docs/quickstart-existing-repo.md @@ -279,15 +279,19 @@ to both the `report` and standalone `evidence-pack manifest` commands, then generate both artifacts again. The event must be a repo-local regular JSON file. If the standalone manifest is present, the public bundle consumer requires it to match the manifest embedded in the report. It also requires the -event and expected profile again to verify its canonical-content binding. Keep -the event outside `.agent-guard/evidence`; it is not one of the seven allowed -public bundle files, and its body is never copied into public evidence. +event and expected profile again to verify its canonical-content binding. The +recognized profile is exactly `agent-policy.audit_event.v1.1`; both producer +and consumer reject JSON that does not match that profile's published event +shape. Keep the event outside `.agent-guard/evidence`; it is not one of the +seven allowed public bundle files, and its body is never copied into public +evidence. ## 5. Consume Evidence Safely -Downstream wrappers should read the sanitized report JSON and validate it -against the packaged `agent-guard.report_evidence.v1` schema before making -decisions. Fail closed on schema drift, inconsistent counts, missing +Downstream wrappers should use the packaged consumer, which selects only the +packaged v1 or v2 schema from the controlled `report.schema_version`. Do not +force a bound-event v2 report through the v1 schema. Fail closed on schema +drift, inconsistent counts, missing `surface_inventory`, missing `evidence_coverage`, unexplained top-level `status` values, non-sanitized reports, unexpected conformance profiles, or forbidden public-evidence fragments such as raw snippets, hash values, diff --git a/scripts/check_wheel_contract.py b/scripts/check_wheel_contract.py index 726bc1e..af89ffd 100644 --- a/scripts/check_wheel_contract.py +++ b/scripts/check_wheel_contract.py @@ -1915,9 +1915,14 @@ def main() -> int: audit_event_marker = "reviewed-wheel-contract-event" audit_event_path = report_output.parent / "policy-admission-event.json" audit_event_payload = { - "schema_version": audit_event_profile, - "status": "reviewed", - "marker": audit_event_marker, + "repo": "example/repo", + "capability": "read", + "context": {"marker": audit_event_marker}, + "decision": { + "mode": "auto_allow", + "reason": "repo_policy", + "matched_repo": "example/repo", + }, } audit_event_path.write_text( json.dumps(audit_event_payload, sort_keys=True), diff --git a/src/agent_guard/cli/evidence_pack.py b/src/agent_guard/cli/evidence_pack.py index 677e593..8c6bc1a 100644 --- a/src/agent_guard/cli/evidence_pack.py +++ b/src/agent_guard/cli/evidence_pack.py @@ -36,7 +36,7 @@ def add_evidence_pack_parser(top) -> None: evidence_pack_manifest.add_argument( "--agent-policy-audit-event-profile", default="", - help="validated public profile identifier for every attached agent-policy audit event", + help="recognized profile agent-policy.audit_event.v1.1 for every attached audit event", ) evidence_pack_manifest.add_argument("--json", action="store_true", help="emit JSON") diff --git a/src/agent_guard/cli/report.py b/src/agent_guard/cli/report.py index 60b1588..d629bed 100644 --- a/src/agent_guard/cli/report.py +++ b/src/agent_guard/cli/report.py @@ -119,7 +119,7 @@ def add_report_parser(top) -> None: report.add_argument( "--agent-policy-audit-event-profile", default="", - help="validated public profile identifier for every attached agent-policy audit event", + help="recognized profile agent-policy.audit_event.v1.1 for every attached audit event", ) report.add_argument( "--format", diff --git a/src/agent_guard/consumer/_cli.py b/src/agent_guard/consumer/_cli.py index a558e7f..d7b06f6 100644 --- a/src/agent_guard/consumer/_cli.py +++ b/src/agent_guard/consumer/_cli.py @@ -42,7 +42,7 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace: parser.add_argument( "--agent-policy-audit-event-profile", default="", - help="Expected public profile identifier for every supplied audit event", + help="Expected recognized profile agent-policy.audit_event.v1.1 for every supplied event", ) args = parser.parse_args(argv) if args.emit_annotations and args.evidence_dir is None: diff --git a/src/agent_guard/evidence_pack.py b/src/agent_guard/evidence_pack.py index 53d96d6..24740e8 100644 --- a/src/agent_guard/evidence_pack.py +++ b/src/agent_guard/evidence_pack.py @@ -35,8 +35,25 @@ ERROR_AUDIT_EVENT_BINDING_REQUIRED = ( "report evidence v2 requires bound agent-policy audit events" ) +AGENT_POLICY_AUDIT_EVENT_PROFILE_V1_1 = "agent-policy.audit_event.v1.1" +SUPPORTED_AGENT_POLICY_AUDIT_EVENT_PROFILES = frozenset( + {AGENT_POLICY_AUDIT_EVENT_PROFILE_V1_1} +) _AUDIT_EVENT_PROFILE_RE = re.compile(r"^[a-z][a-z0-9._-]{0,127}$") _AUDIT_EVENT_DIGEST_RE = re.compile(r"^b[a-z2-7]{52}$") +_AUDIT_EVENT_SESSION_ID_RE = re.compile(r"^[A-Za-z0-9._:@/+~-]+$") +_AUDIT_EVENT_DECISION_MODES = frozenset( + {"deny", "require_approval", "auto_allow"} +) +_AUDIT_EVENT_DECISION_REASONS = frozenset( + { + "hard_guardrail", + "repo_policy", + "default_mode", + "condition_match", + "no_match", + } +) class _JSONNumber(str): @@ -65,7 +82,10 @@ def safe_artifact_path(path: str, *, root: Path | None = None) -> str: def validate_agent_policy_audit_event_profile(profile: str) -> str: normalized = str(profile).strip() - if not _AUDIT_EVENT_PROFILE_RE.fullmatch(normalized): + if ( + not _AUDIT_EVENT_PROFILE_RE.fullmatch(normalized) + or normalized not in SUPPORTED_AGENT_POLICY_AUDIT_EVENT_PROFILES + ): raise ValueError(ERROR_AUDIT_EVENT_PROFILE) public_profile = {"event_profile": normalized} try: @@ -117,6 +137,88 @@ def _canonical_json_value(value: object) -> bytes: raise ValueError(ERROR_AUDIT_EVENT_INVALID) +def _is_json_string(value: object) -> bool: + return type(value) is str + + +def _contains_control_character(value: str) -> bool: + return any(ord(character) <= 0x1F for character in value) + + +def _validate_agent_policy_audit_event_v1_1(payload: dict[str, Any]) -> None: + required_fields = {"repo", "capability", "context", "decision"} + optional_fields = {"session_id", "command", "path"} + if not required_fields <= set(payload) or not set(payload) <= ( + required_fields | optional_fields + ): + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + + for field in ("repo", "capability"): + value = payload.get(field) + if not _is_json_string(value) or not value: + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + if not isinstance(payload.get("context"), dict): + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + + decision = payload.get("decision") + if not isinstance(decision, dict) or set(decision) != { + "mode", + "reason", + "matched_repo", + }: + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + mode = decision.get("mode") + reason = decision.get("reason") + matched_repo = decision.get("matched_repo") + if not _is_json_string(mode) or mode not in _AUDIT_EVENT_DECISION_MODES: + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + if not _is_json_string(reason) or reason not in _AUDIT_EVENT_DECISION_REASONS: + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + if matched_repo is not None and ( + not _is_json_string(matched_repo) + or not matched_repo + or len(matched_repo) > 256 + ): + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + + if "session_id" in payload: + session_id = payload["session_id"] + if ( + not _is_json_string(session_id) + or not 1 <= len(session_id) <= 256 + or not _AUDIT_EVENT_SESSION_ID_RE.fullmatch(session_id) + ): + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + if "command" in payload: + command = payload["command"] + if ( + not _is_json_string(command) + or not 1 <= len(command) <= 4096 + or _contains_control_character(command) + ): + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + if "path" in payload: + event_path = payload["path"] + if ( + not _is_json_string(event_path) + or not 1 <= len(event_path) <= 1024 + or event_path.startswith("/") + or _contains_control_character(event_path) + ): + raise ValueError(ERROR_AUDIT_EVENT_INVALID) + + +def _validate_agent_policy_audit_event_payload( + payload: dict[str, Any], + *, + event_profile: str, +) -> None: + if event_profile == AGENT_POLICY_AUDIT_EVENT_PROFILE_V1_1: + _validate_agent_policy_audit_event_v1_1(payload) + return + raise ValueError(ERROR_AUDIT_EVENT_PROFILE) + + def _repo_relative_audit_event_path(path: Path, repo_root: Path) -> tuple[Path, Path]: """Return a lexical in-root path without dereferencing its components.""" @@ -298,7 +400,7 @@ def _read_agent_policy_audit_event( return raw, relative_path -def _canonical_agent_policy_audit_event(raw: bytes) -> bytes: +def _canonical_agent_policy_audit_event(raw: bytes, *, event_profile: str) -> bytes: try: text = raw.decode("utf-8") payload = json.loads( @@ -310,6 +412,10 @@ def _canonical_agent_policy_audit_event(raw: bytes) -> bytes: ) if not isinstance(payload, dict): raise TypeError(ERROR_AUDIT_EVENT_INVALID) + _validate_agent_policy_audit_event_payload( + payload, + event_profile=event_profile, + ) canonical = _canonical_json_value(payload) except (UnicodeError, json.JSONDecodeError, RecursionError, TypeError, ValueError): raise ValueError(ERROR_AUDIT_EVENT_INVALID) from None @@ -324,7 +430,7 @@ def _build_agent_policy_audit_event_binding( ) -> tuple[dict[str, str], str | None]: profile = validate_agent_policy_audit_event_profile(event_profile) raw, relative_path = _read_agent_policy_audit_event(path, repo_root=repo_root) - canonical = _canonical_agent_policy_audit_event(raw) + canonical = _canonical_agent_policy_audit_event(raw, event_profile=profile) domain = ( AGENT_POLICY_AUDIT_EVENT_BINDING_SCHEMA_VERSION.encode("ascii") + b"\0" diff --git a/src/agent_guard/schemas/agent-guard.evidence_pack_manifest.v2.schema.json b/src/agent_guard/schemas/agent-guard.evidence_pack_manifest.v2.schema.json index 04ba693..fc05433 100644 --- a/src/agent_guard/schemas/agent-guard.evidence_pack_manifest.v2.schema.json +++ b/src/agent_guard/schemas/agent-guard.evidence_pack_manifest.v2.schema.json @@ -99,8 +99,7 @@ "properties": { "schema_version": {"const": "agent-guard.agent_policy_audit_event_binding.v1"}, "event_profile": { - "type": "string", - "pattern": "^[a-z][a-z0-9._-]{0,127}$" + "const": "agent-policy.audit_event.v1.1" }, "canonicalization": {"const": "canonical-json-v1"}, "digest_algorithm": {"const": "sha256"}, diff --git a/src/agent_guard/schemas/agent-guard.report_evidence.v2.schema.json b/src/agent_guard/schemas/agent-guard.report_evidence.v2.schema.json index 40cb2c5..fc2559f 100644 --- a/src/agent_guard/schemas/agent-guard.report_evidence.v2.schema.json +++ b/src/agent_guard/schemas/agent-guard.report_evidence.v2.schema.json @@ -344,8 +344,7 @@ "properties": { "schema_version": {"const": "agent-guard.agent_policy_audit_event_binding.v1"}, "event_profile": { - "type": "string", - "pattern": "^[a-z][a-z0-9._-]{0,127}$" + "const": "agent-policy.audit_event.v1.1" }, "canonicalization": {"const": "canonical-json-v1"}, "digest_algorithm": {"const": "sha256"}, diff --git a/tests/audit_event_helpers.py b/tests/audit_event_helpers.py new file mode 100644 index 0000000..f344ffb --- /dev/null +++ b/tests/audit_event_helpers.py @@ -0,0 +1,34 @@ +"""Shared valid agent-policy audit-event fixtures for binding contract tests.""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + + +def audit_event_payload( + *, + capability: str = "read", + context: dict[str, Any] | None = None, + mode: str = "auto_allow", + reason: str = "repo_policy", + matched_repo: str | None = "example/repo", +) -> dict[str, Any]: + return { + "repo": "example/repo", + "capability": capability, + "context": {} if context is None else context, + "decision": { + "mode": mode, + "reason": reason, + "matched_repo": matched_repo, + }, + } + + +def write_audit_event(path: Path, **overrides: Any) -> None: + path.write_text( + json.dumps(audit_event_payload(**overrides), sort_keys=True) + "\n", + encoding="utf-8", + ) diff --git a/tests/cli/test_evidence_pack.py b/tests/cli/test_evidence_pack.py index bb197f6..add34c6 100644 --- a/tests/cli/test_evidence_pack.py +++ b/tests/cli/test_evidence_pack.py @@ -17,6 +17,7 @@ build_agent_policy_audit_event_binding, build_evidence_pack_manifest, ) +from tests.audit_event_helpers import audit_event_payload, write_audit_event from tests.cli.helpers import run_cli AUDIT_EVENT_PROFILE = "agent-policy.audit_event.v1.1" @@ -31,7 +32,7 @@ def test_evidence_pack_manifest_cli_is_sanitized(tmp_path: Path) -> None: report = tmp_path / "report.json" event = tmp_path / ".agent-guard" / "evidence" / "policy-admission-event.json" event.parent.mkdir(parents=True) - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) report_payload = json.loads(EVIDENCE_SAMPLE_REPORT.read_text(encoding="utf-8")) report_payload["report"]["schema_version"] = "agent-guard.report_evidence.v2" embedded_manifest = report_payload["evidence_pack_manifest"] @@ -107,7 +108,7 @@ def test_evidence_pack_manifest_cli_rejects_unverified_bound_v2_report( report = tmp_path / "report.json" event = tmp_path / "reviewed" / "event.json" event.parent.mkdir() - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) report_payload = json.loads(EVIDENCE_SAMPLE_REPORT.read_text(encoding="utf-8")) report_payload["report"]["schema_version"] = "agent-guard.report_evidence.v2" manifest = report_payload["evidence_pack_manifest"] @@ -148,7 +149,7 @@ def test_evidence_pack_manifest_cli_rejects_mismatched_bound_v2_event( report = tmp_path / "report.json" event = tmp_path / "reviewed" / "event.json" event.parent.mkdir() - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) report_payload = json.loads(EVIDENCE_SAMPLE_REPORT.read_text(encoding="utf-8")) report_payload["report"]["schema_version"] = "agent-guard.report_evidence.v2" manifest = report_payload["evidence_pack_manifest"] @@ -166,7 +167,7 @@ def test_evidence_pack_manifest_cli_rejects_mismatched_bound_v2_event( ) report.write_text(json.dumps(report_payload), encoding="utf-8") private_marker = "synthetic-unreviewed-event-body" - event.write_text(json.dumps({"marker": private_marker}), encoding="utf-8") + write_audit_event(event, context={"marker": private_marker}) result = run_cli( "evidence-pack", @@ -287,7 +288,7 @@ def test_evidence_pack_manifest_cli_rejects_unbound_or_external_audit_event( encoding="utf-8", ) event = tmp_path / "event.json" - event.write_text("{}\n", encoding="utf-8") + write_audit_event(event) missing_profile = run_cli( "evidence-pack", @@ -328,8 +329,8 @@ def test_audit_event_artifacts_preserve_nested_paths_for_duplicate_basenames( second = root / "evidence" / "second" / "policy-admission-event.json" first.parent.mkdir(parents=True) second.parent.mkdir(parents=True) - first.write_text('{"status":"first"}\n', encoding="utf-8") - second.write_text('{"status":"second"}\n', encoding="utf-8") + write_audit_event(first, context={"status": "first"}) + write_audit_event(second, context={"status": "second"}) artifacts = build_agent_policy_audit_event_artifacts( [ @@ -350,15 +351,16 @@ def test_audit_event_artifacts_preserve_nested_paths_for_duplicate_basenames( def test_audit_event_binding_is_canonical_and_detects_content_change(tmp_path: Path) -> None: event = tmp_path / "event.json" - event.write_text('{"decision":{"mode":"auto_allow"},"capability":"read"}\n', encoding="utf-8") + payload = audit_event_payload(capability="read") + event.write_text(json.dumps(payload, separators=(",", ":")), encoding="utf-8") first = build_agent_policy_audit_event_binding(event, event_profile=AUDIT_EVENT_PROFILE) event.write_text( - '{\n "capability": "read",\n "decision": {"mode": "auto_allow"}\n}\n', + json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8", ) equivalent = build_agent_policy_audit_event_binding(event, event_profile=AUDIT_EVENT_PROFILE) - event.write_text('{"capability":"write","decision":{"mode":"auto_allow"}}\n', encoding="utf-8") + write_audit_event(event, capability="write") changed = build_agent_policy_audit_event_binding(event, event_profile=AUDIT_EVENT_PROFILE) assert equivalent == first @@ -369,13 +371,23 @@ def test_audit_event_binding_preserves_distinct_large_number_lexemes( tmp_path: Path, ) -> None: event = tmp_path / "event.json" - event.write_text('{"sequence":9007199254740992.0}\n', encoding="utf-8") + event.write_text( + '{"capability":"read","context":{"sequence":9007199254740992.0},' + '"decision":{"matched_repo":"example/repo","mode":"auto_allow",' + '"reason":"repo_policy"},"repo":"example/repo"}\n', + encoding="utf-8", + ) first = build_agent_policy_audit_event_binding( event, event_profile=AUDIT_EVENT_PROFILE, ) - event.write_text('{"sequence":9007199254740993.0}\n', encoding="utf-8") + event.write_text( + '{"capability":"read","context":{"sequence":9007199254740993.0},' + '"decision":{"matched_repo":"example/repo","mode":"auto_allow",' + '"reason":"repo_policy"},"repo":"example/repo"}\n', + encoding="utf-8", + ) changed = build_agent_policy_audit_event_binding( event, event_profile=AUDIT_EVENT_PROFILE, @@ -384,12 +396,77 @@ def test_audit_event_binding_preserves_distinct_large_number_lexemes( assert changed["digest"] != first["digest"] +@pytest.mark.parametrize( + "payload", + ( + {"case_id": "not-an-audit-event", "expected_findings": []}, + { + key: value + for key, value in audit_event_payload().items() + if key != "repo" + }, + audit_event_payload() | {"extra": "not allowed"}, + audit_event_payload() | {"capability": 7}, + audit_event_payload() + | { + "decision": { + "mode": "auto_allow", + "reason": "repo_policy", + "matched_repo": "example/repo", + "extra": True, + } + }, + audit_event_payload() | {"path": "/private/repository"}, + ), +) +def test_audit_event_binding_rejects_payloads_outside_recognized_profile_schema( + tmp_path: Path, + payload: dict[str, object], +) -> None: + marker = "not-an-audit-event" + event = tmp_path / "event.json" + event.write_text(json.dumps(payload), encoding="utf-8") + + with pytest.raises( + ValueError, + match="^agent-policy audit event is not valid bounded JSON$", + ) as exc_info: + build_agent_policy_audit_event_binding( + event, + event_profile=AUDIT_EVENT_PROFILE, + ) + + assert marker not in str(exc_info.value) + + +def test_audit_event_binding_rejects_unsupported_profile_before_digest( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + event = tmp_path / "event.json" + write_audit_event(event) + + def unexpected_digest(_value: bytes) -> object: + raise AssertionError("digest computation must not run") + + monkeypatch.setattr(evidence_pack.hashlib, "sha256", unexpected_digest) + + with pytest.raises( + ValueError, + match="^agent-policy audit event profile is invalid$", + ): + build_agent_policy_audit_event_binding( + event, + event_profile="agent-policy.audit_event.v1.2", + ) + + def test_manifest_rejects_extra_fields_in_prebuilt_audit_event_artifact( tmp_path: Path, ) -> None: marker = "synthetic-private-passphrase" event = tmp_path / "event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) binding = build_agent_policy_audit_event_binding( event, event_profile=AUDIT_EVENT_PROFILE, @@ -419,7 +496,7 @@ def test_manifest_rejects_invalid_prebuilt_audit_event_artifact_shape( tmp_path: Path, ) -> None: event = tmp_path / "event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) binding = build_agent_policy_audit_event_binding( event, event_profile=AUDIT_EVENT_PROFILE, @@ -456,7 +533,7 @@ def test_manifest_rejects_invalid_prebuilt_audit_event_artifact_shape( def test_bound_manifest_rejects_v1_report_without_leak(tmp_path: Path) -> None: event = tmp_path / "event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) artifact = { "path": "reviewed/event.json", "role": "agent-policy-audit-event", @@ -528,7 +605,7 @@ def test_audit_event_profile_rejects_public_sanitization_changes_before_digest( event_profile: str, ) -> None: event = tmp_path / "event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) assert evidence_pack._AUDIT_EVENT_PROFILE_RE.fullmatch(event_profile) def unexpected_digest(_value: bytes) -> object: @@ -555,7 +632,8 @@ def test_audit_event_binding_enforces_one_mib_read_bound_before_parse( event = tmp_path / "event.json" event.write_bytes(b"x" * (evidence_pack.MAX_AGENT_POLICY_AUDIT_EVENT_BYTES + 1)) - def unexpected_parse(_raw: bytes) -> bytes: + def unexpected_parse(_raw: bytes, *, event_profile: str) -> bytes: + del event_profile raise AssertionError("oversized event must not be parsed") monkeypatch.setattr( @@ -583,7 +661,7 @@ def test_audit_event_binding_reads_opened_descriptor_after_final_path_swap( event = root / "reviewed" / "event.json" external = tmp_path / "external-event.json" event.parent.mkdir(parents=True) - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) external.write_text('{"status":"external"}\n', encoding="utf-8") expected = build_agent_policy_audit_event_binding( event, @@ -630,7 +708,7 @@ def test_audit_event_binding_rejects_final_file_symlink_swap_without_leak( external = tmp_path / "external-event.json" external_marker = "synthetic-external-event-marker" event.parent.mkdir(parents=True) - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) external.write_text(json.dumps({"marker": external_marker}), encoding="utf-8") original_open = evidence_pack._open_agent_policy_audit_event_posix @@ -671,7 +749,7 @@ def test_audit_event_binding_rejects_ancestor_symlink_swap_without_leak( external_dir = tmp_path / "external" external_marker = "synthetic-external-ancestor-marker" event_dir.mkdir(parents=True) - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) external_dir.mkdir() (external_dir / "event.json").write_text( json.dumps({"marker": external_marker}), @@ -763,7 +841,12 @@ def test_evidence_pack_manifest_rejects_non_unicode_audit_event_json(tmp_path: P encoding="utf-8", ) event = tmp_path / "event.json" - event.write_bytes(b'{"value":"\\ud800"}\n') + event.write_bytes( + b'{"repo":"example/repo","capability":"read",' + b'"context":{"value":"\\ud800"},' + b'"decision":{"mode":"auto_allow","reason":"repo_policy",' + b'"matched_repo":"example/repo"}}\n' + ) result = run_cli( "evidence-pack", diff --git a/tests/cli/test_report_output.py b/tests/cli/test_report_output.py index fd5effd..b972791 100644 --- a/tests/cli/test_report_output.py +++ b/tests/cli/test_report_output.py @@ -9,6 +9,7 @@ from agent_guard import __version__ as AGENT_GUARD_VERSION +from tests.audit_event_helpers import write_audit_event from tests.cli.helpers import assert_shared_envelope, create_report_violation_fixture_repo, read_report_fixture, run_cli, write @@ -202,7 +203,7 @@ def test_report_cli_json_output_writes_file_and_suppresses_stdout(tmp_path: Path output = tmp_path / "evidence" / "agent-guard-report.json" event = tmp_path / "evidence" / "policy-admission-event.json" event.parent.mkdir(parents=True) - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) result = run_cli( "report", @@ -255,7 +256,7 @@ def test_report_cli_audit_event_implies_evidence_pack_manifest(tmp_path: Path) - ) event = tmp_path / "evidence" / "nested" / "policy-admission-event.json" event.parent.mkdir(parents=True) - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) result = run_cli( "report", @@ -313,7 +314,7 @@ def test_report_cli_error_after_valid_audit_event_remains_v1(tmp_path: Path) -> event_marker = "synthetic-reviewed-event-marker" event = tmp_path / "evidence" / "policy-admission-event.json" event.parent.mkdir(parents=True) - event.write_text(json.dumps({"marker": event_marker}), encoding="utf-8") + write_audit_event(event, context={"marker": event_marker}) result = run_cli( "report", diff --git a/tests/test_contract_stability.py b/tests/test_contract_stability.py index 087931c..a21b838 100644 --- a/tests/test_contract_stability.py +++ b/tests/test_contract_stability.py @@ -170,7 +170,7 @@ def test_changelog_records_latest_release_entry() -> None: assert normalized_unreleased == " ".join( [ "- Isolated repository-controlled context-policy regular-expression matching behind the existing bounded scanner worker and added fixed pattern-count and pattern-length limits. Timeout and limit failures remain deterministic, sanitized configuration errors; no raw pattern or context text is emitted.", - "- Content-bound optional `agent-policy` audit-event references with a canonical-JSON, profile-bound, public-safe digest. Producers require a caller-designated repo-local JSON event and explicit profile; maintainer review and event-schema validation remain external. Consumers require the separately supplied event and reject missing, malformed, or replaced content. Audit-event binding uses report and manifest v2; the released v1 schemas remain unchanged and their path-and-role references remain readable as explicitly unbound legacy metadata. The event body remains outside the fixed seven-file public bundle.", + "- Content-bound optional `agent-policy` audit-event references with a canonical-JSON, profile-bound, public-safe digest. Producers require a caller-designated repo-local JSON event and the recognized `agent-policy.audit_event.v1.1` profile. Producers and consumers validate that profile's published event shape and reject unsupported profiles, malformed events, or replaced content. Maintainer review remains external. Audit-event binding uses report and manifest v2; the released v1 schemas remain unchanged and their path-and-role references remain readable as explicitly unbound legacy metadata. The event body remains outside the fixed seven-file public bundle.", "- Defined a bounded demand-validation window through 2026-09-20 and froze feature releases pending an explicit maintainer decision after the 2026-09-21 review. Marketplace publication remains separately prohibited without explicit authorization.", "- Locked the release build toolchain, pinned copyable GitHub Action examples to the immutable v0.3.4 release commit, and documented the post-release pin refresh contract.", "- Simplified reviewed bootstrap and monorepo onboarding, added explicit Python interpreter checks, and tightened guidance for copying public-safe evidence.", diff --git a/tests/test_evidence_consumer.py b/tests/test_evidence_consumer.py index d180c98..2571992 100644 --- a/tests/test_evidence_consumer.py +++ b/tests/test_evidence_consumer.py @@ -5,6 +5,8 @@ from __future__ import annotations +import base64 +import hashlib import importlib.util import json import os @@ -31,6 +33,7 @@ from agent_guard.consumer._bundle import MAX_MARKDOWN_BYTES from agent_guard.evidence_pack import build_agent_policy_audit_event_binding from agent_guard.report_render import emit_report_output, render_report_output +from tests.audit_event_helpers import write_audit_event REPO_ROOT = Path(__file__).resolve().parents[1] SRC = REPO_ROOT / "src" @@ -101,7 +104,7 @@ def test_packaged_consumer_accepts_legacy_unbound_audit_event_reference( assert summary["report_schema_version"] == "agent-guard.report_evidence.v1" event = tmp_path / "synthetic-legacy-event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) with pytest.raises( ValueError, match=r"^agent-policy audit event binding is invalid$", @@ -118,7 +121,7 @@ def test_packaged_consumer_accepts_legacy_unbound_audit_event_reference( def test_v1_content_binding_field_never_counts_as_bound(tmp_path: Path) -> None: event = tmp_path / "synthetic-v1-event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) payload = json.loads(SAMPLE.read_text(encoding="utf-8")) payload["evidence_pack_manifest"]["artifacts"].append( { @@ -149,7 +152,7 @@ def test_packaged_consumer_rejects_extra_audit_event_artifact_fields_without_lea ) -> None: marker = "synthetic-private-passphrase" event = tmp_path / "event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) binding = build_agent_policy_audit_event_binding( event, event_profile=AUDIT_EVENT_PROFILE, @@ -184,7 +187,7 @@ def test_packaged_consumer_rejects_final_audit_event_symlink_without_leak( event = tmp_path / "event.json" external = tmp_path / "external.json" external_marker = "synthetic-consumer-external-marker" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) report = _bound_v2_report(event) external.write_text(json.dumps({"marker": external_marker}), encoding="utf-8") event.unlink() @@ -208,10 +211,20 @@ def test_packaged_consumer_rejects_large_number_audit_event_substitution( tmp_path: Path, ) -> None: event = tmp_path / "event.json" - event.write_text('{"sequence":9007199254740992.0}\n', encoding="utf-8") + event.write_text( + '{"capability":"read","context":{"sequence":9007199254740992.0},' + '"decision":{"matched_repo":"example/repo","mode":"auto_allow",' + '"reason":"repo_policy"},"repo":"example/repo"}\n', + encoding="utf-8", + ) report = _bound_v2_report(event) - event.write_text('{"sequence":9007199254740993.0}\n', encoding="utf-8") + event.write_text( + '{"capability":"read","context":{"sequence":9007199254740993.0},' + '"decision":{"matched_repo":"example/repo","mode":"auto_allow",' + '"reason":"repo_policy"},"repo":"example/repo"}\n', + encoding="utf-8", + ) with pytest.raises( ValueError, @@ -226,7 +239,7 @@ def test_packaged_consumer_rejects_large_number_audit_event_substitution( def test_current_consumer_accepts_matching_bound_v2_evidence(tmp_path: Path) -> None: event = tmp_path / "synthetic-reviewed-event.json" - event.write_text('{"status":"reviewed","sequence":7}\n', encoding="utf-8") + write_audit_event(event, context={"sequence": 7}) payload = _bound_v2_report(event) summary = validate_report(payload, select_report_schema(payload)) @@ -253,10 +266,72 @@ def test_current_consumer_accepts_matching_bound_v2_evidence(tmp_path: Path) -> ) +def test_current_consumer_rejects_non_event_json_with_recognized_profile( + tmp_path: Path, +) -> None: + event = tmp_path / "synthetic-non-event.json" + marker = "not-an-audit-event" + event_payload = {"case_id": marker, "expected_findings": []} + event.write_text(json.dumps(event_payload), encoding="utf-8") + canonical = json.dumps( + event_payload, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + domain = ( + b"agent-guard.agent_policy_audit_event_binding.v1\0" + + AUDIT_EVENT_PROFILE.encode("ascii") + + b"\0" + + canonical + ) + forged_digest = ( + "b" + + base64.b32encode(hashlib.sha256(domain).digest()) + .decode("ascii") + .rstrip("=") + .lower() + ) + + payload = json.loads(SAMPLE.read_text(encoding="utf-8")) + payload["report"]["schema_version"] = "agent-guard.report_evidence.v2" + manifest = payload["evidence_pack_manifest"] + manifest["schema_version"] = "agent-guard.evidence_pack_manifest.v2" + manifest["report"]["schema_version"] = "agent-guard.report_evidence.v2" + manifest["artifacts"].append( + { + "path": "reviewed/event.json", + "role": "agent-policy-audit-event", + "content_binding": { + "schema_version": "agent-guard.agent_policy_audit_event_binding.v1", + "event_profile": AUDIT_EVENT_PROFILE, + "canonicalization": "canonical-json-v1", + "digest_algorithm": "sha256", + "digest_encoding": "base32-lower-no-padding", + "digest": forged_digest, + }, + } + ) + + validate_report(payload, select_report_schema(payload)) + with pytest.raises( + ValueError, + match=r"^agent-policy audit event binding is invalid$", + ) as exc_info: + validate_agent_policy_audit_event_files( + payload, + (event,), + event_profile=AUDIT_EVENT_PROFILE, + ) + + assert marker not in str(exc_info.value) + assert str(event) not in str(exc_info.value) + + def test_bound_v2_verification_failures_are_sanitized(tmp_path: Path) -> None: event_marker = "synthetic-event-body-marker" event = tmp_path / "synthetic-sensitive-event-name.json" - event.write_text(json.dumps({"marker": event_marker}), encoding="utf-8") + write_audit_event(event, context={"marker": event_marker}) payload = _bound_v2_report(event) mismatched_profile = "agent-policy.audit_event.v1.2" @@ -299,7 +374,7 @@ def test_current_consumer_rejects_mixed_unbound_v2_entries_without_leak( tmp_path: Path, ) -> None: event = tmp_path / "synthetic-reviewed-event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) payload = _bound_v2_report(event) marker = "synthetic-private-unbound-path.json" payload["evidence_pack_manifest"]["artifacts"].append( @@ -365,7 +440,7 @@ def test_report_schema_loader_is_bounded_and_v1_default_is_preserved() -> None: def test_v1_schema_consumer_fails_closed_on_v2_report(tmp_path: Path) -> None: event = tmp_path / "synthetic-reviewed-event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) payload = _bound_v2_report(event) with pytest.raises( @@ -400,7 +475,7 @@ def test_evidence_pack_manifest_rejects_incompatible_version_pair( manifest["report"]["schema_version"] = report_version if manifest_version == "agent-guard.evidence_pack_manifest.v2": event = tmp_path / "synthetic-reviewed-event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) manifest["artifacts"].append( { "path": "reviewed/event.json", @@ -720,7 +795,7 @@ def test_packaged_consumer_accepts_bound_v2_standalone_manifest_bundle( evidence_dir = tmp_path / "evidence" evidence_dir.mkdir() event = tmp_path / "synthetic-reviewed-event.json" - event.write_text('{"status":"reviewed"}\n', encoding="utf-8") + write_audit_event(event) payload = _bound_v2_report(event) report = evidence_dir / "agent-guard-report.json" report.write_text(json.dumps(payload), encoding="utf-8") diff --git a/tests/test_evidence_contract_examples.py b/tests/test_evidence_contract_examples.py index 949fde5..267dead 100644 --- a/tests/test_evidence_contract_examples.py +++ b/tests/test_evidence_contract_examples.py @@ -15,6 +15,7 @@ import pytest import yaml +from tests.audit_event_helpers import audit_event_payload from tests.cli.helpers import mcp_policy_text, sha256_text, write @@ -398,7 +399,7 @@ def test_reviewed_audit_event_handoff_produces_consistent_public_bundle( repo.mkdir() write_contract_repo(repo) event_path = "reviewed/policy-admission-event.json" - write(repo / event_path, '{"status":"reviewed"}\n') + write(repo / event_path, json.dumps(audit_event_payload()) + "\n") report = generate_recommended_report( repo, agent_policy_audit_event=event_path, @@ -512,7 +513,10 @@ def test_reviewed_audit_event_handoff_produces_consistent_public_bundle( assert wrong_profile_result.stdout == "" assert wrong_profile_result.stderr.strip() == "agent-guard evidence bundle invalid" - write(repo / event_path, '{"status":"replaced"}\n') + write( + repo / event_path, + json.dumps(audit_event_payload(context={"status": "replaced"})) + "\n", + ) replaced_result = subprocess.run( [ sys.executable, diff --git a/tests/test_schemas.py b/tests/test_schemas.py index 05b0827..9fac4c6 100644 --- a/tests/test_schemas.py +++ b/tests/test_schemas.py @@ -262,6 +262,12 @@ def test_v2_evidence_schemas_require_exact_bound_audit_event_entries() -> None: ) assert validator.is_valid(value) + manifest["artifacts"][-1]["content_binding"] = binding | { + "event_profile": "agent-policy.audit_event.v1.2" + } + assert not validator.is_valid(value) + manifest["artifacts"][-1]["content_binding"] = binding + manifest["artifacts"][-1]["review_metadata"] = "synthetic-public-metadata" assert not validator.is_valid(value) del manifest["artifacts"][-1]["review_metadata"] diff --git a/tests/test_windows_file_boundaries.py b/tests/test_windows_file_boundaries.py index 748b1da..8b72841 100644 --- a/tests/test_windows_file_boundaries.py +++ b/tests/test_windows_file_boundaries.py @@ -26,7 +26,12 @@ def test_windows_repo_bound_readers_accept_in_root_regular_files(tmp_path: Path) (api_path, "def handler():\n return 'ok'\n"), (content_path, "Reviewed documentation.\n"), (workflow_path, "name: ci\njobs: {}\n"), - (audit_event_path, '{"status":"reviewed"}\n'), + ( + audit_event_path, + '{"repo":"example/repo","capability":"read","context":{},' + '"decision":{"mode":"auto_allow","reason":"repo_policy",' + '"matched_repo":"example/repo"}}\n', + ), ): path.parent.mkdir(parents=True, exist_ok=True) path.write_bytes(text.encode("utf-8"))