Agent Safety Evidence Demo

This repository is a public-safe demo for one narrow handoff: static repository evidence from agent-guard plus a deterministic admission artifact from agent-policy.

It is not a comprehensive agent safety toolkit. It shows a publishable evidence shape that maintainers can inspect, copy, and adapt. For copying the pattern into another repository, use docs/adoption-recipe.md.

What It Demonstrates

agent-policy handles runtime admission:

• normalize a requested agent action into a small capability name
• evaluate the capability against a repo policy matrix
• return one of auto_allow, require_approval, or deny
• map that decision to a process exit code that callers can enforce

agent-guard handles static repository gates:

• reject unsafe agent context file instructions
• emit redacted agent context inventory metadata for review evidence
• emit agent surface inventory v2 metadata for documented guard commands and evidence artifacts
• verify that discovered agent context files are pinned by digest policy
• reject private artifact paths before publication
• reject unsafe public-demo content patterns
• reject forbidden API endpoint references
• pin safety-critical file digests so drift is visible in CI
• emit a sanitized JSON evidence report for reviewers and automation

Together they cover different layers. agent-policy answers "may this agent action continue now?" while agent-guard answers "does this repository satisfy the selected deterministic static evidence profile?" The demo pairs one runtime admission audit event with one static guard evidence report so maintainers can review both sides without storing raw prompts, repository contents, hashes, tokens, or local paths.

Runtime Admission Demo

The shared contract in scripts/policy_event_contract.py keeps the demo action-to-capability vocabulary small and explicit, and gives the producer and validator one public-safe field grammar. The wrapper in scripts/policy_admit.py uses that contract for runtime decisions, and scripts/validate_policy_event.py uses it to validate public audit events:

Demo action	Capability	Expected mode	Exit
read_docs	read	auto_allow	0
edit_docs	write	require_approval	2
publish_release	artifact.publish	require_approval	2
force_push	push.force	deny	3

Invalid invocations and program errors exit 1; exit 2 is reserved for a validated require_approval decision.

Run a single admission check:

python3 scripts/policy_admit.py --action read_docs --repo yui-stingray/agent-safety-toolkit-example

Emit the deterministic audit event shape used by wrappers and CI:

python3 scripts/policy_admit.py --action read_docs --repo yui-stingray/agent-safety-toolkit-example --repo-alias agent-safety-toolkit-example-public --audit-event --command read_docs --path README.md

--repo is the identifier evaluated against .agent-policy/policy.toml. Audit events are public evidence, so --repo-alias is required with --audit-event; the raw repository identifier is never used as an audit-event fallback. scripts/validate_policy_event.py validates the committed public audit-event artifact and rejects raw repository identifiers, local paths, unsupported fields, and secret-shaped values before agent-guard references it. The installed yui-agent-policy 0.1.11 distribution also includes an opt-in generic agent-policy.audit_event.v1.1 JSON schema, but this demo intentionally keeps its stricter public-artifact profile. The generic schema does not replace the demo's raw repo identifier, local path, or secret-shaped value checks.

Local Verification

This demo's checked-in lock targets CPython 3.12 on GitHub-hosted Ubuntu Linux x86_64, which is also the CI target. Generate a separate hash lock and CI job before claiming support for another platform.

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install --require-hashes -r requirements/agent-safety-tools.txt
python -m pytest -q
bash scripts/run_demo.sh

scripts/run_demo.sh rejects non-3.12 interpreters. Set PYTHON to an explicit Python 3.12 executable when the activated environment is not first on PATH. It also requires GNU timeout. The script runs the published agent-guard 0.3.4 context check, context inventory, surface inventory, context lock, and report behind a 12-second external supervisor. This limits a known CPU-exhaustion risk from an unreviewed custom context-policy regular expression; timeout output is discarded and the demo fails closed. Review repository policy changes before running the demo. This wrapper is a mitigation, not a fixed agent-guard release.

The end-to-end script runs:

• expected pass and fail runtime admission checks
• public-safe audit-event schema validation
• path guard
• context guard
• redacted context inventory
• context lock coverage against the committed digest policy
• content guard
• API guard
• MCP config guard with a reviewed repo policy
• digest guard
• workflow drift guard
• policy/spec drift guard
• recommended-profile conformance check
• sanitized JSON evidence report and evidence-pack manifest
• downstream evidence consumer validation

The static guard portion is intentionally deterministic and can be inspected as these core commands:

agent-guard() {
  PYTHON="$(command -v python)" bash scripts/run_agent_guard_bounded.sh \
    python -m agent_guard.cli "$@"
}
agent-guard context check --root . --policy .agent-guard/context-policy.yaml --json
agent-guard surface inventory --root . --context-policy .agent-guard/context-policy.yaml --schema-version v2 --json
agent-guard mcp check --root . --policy .agent-guard/mcp-policy.yaml --json
agent-guard workflow check --root . --policy .agent-guard/workflow-policy.yaml --json
agent-guard drift check --root . --profile recommended --schema-version v2 --json
agent-guard report --root . --context-policy .agent-guard/context-policy.yaml --evidence-preset recommended --api-policy .agent-guard/api-policy.yaml --mcp-policy .agent-guard/mcp-policy.yaml --digest-policy .agent-guard/context-digest-policy.yaml --agent-policy-audit-event .agent-policy/evidence/policy-admission-event.json --format json --output .agent-guard/evidence/agent-guard-report.json
unset -f agent-guard

The temporary agent-guard shell function above routes every displayed command through the bounded wrapper; it does not invoke the installed executable directly.

Treat the individual per-scanner --json outputs above as local inspection or CI-internal diagnostics. The public handoff is the sanitized report and evidence-pack output under .agent-guard/evidence/; do not upload raw scanner JSON from a private repository unless a maintainer has reviewed that exact output. The MCP config guard reads committed configuration metadata only. It does not execute MCP servers, validate live OAuth flows, inspect MCP tool results, or detect MCP tool-poisoning behavior.

It writes the runtime admission artifact separately under .agent-policy/evidence/:

• policy-admission-event.json: deterministic agent-policy runtime admission evidence for one normalized action.

The fixed public agent-guard bundle under .agent-guard/evidence/ contains:

• agent-guard-report.json: sanitized agent-guard static repository evidence, including surface inventory v2, context lock coverage, workflow drift, profile conformance, and an embedded evidence-pack manifest with a sanitized agent-policy audit-event artifact reference.
• agent-guard-evidence-pack.json: compact artifact index for reviewer handoff, including the report and agent-policy audit-event artifact references.

The runner snapshots the previous complete evidence set, generates and validates its replacement with the installed agent-guard consumer, and restores the snapshot after ordinary catchable failures. It refuses to replace a bundle directory containing unexpected entries, so unrelated local evidence is not deleted implicitly. Publication spans .agent-guard and .agent-policy and is not an atomic multi-file transaction. SIGKILL, host power loss, and concurrent readers are outside this guarantee; run the demo in an isolated checkout with one writer, and publish or consume the resulting set only after the command succeeds. The standalone surface inventory command remains a local/CI check; this demo uses the identical section embedded in the report as the public handoff. The manifest intentionally references the separately stored, sanitized runtime admission event so reviewers can correlate the two evidence layers without co-locating the event in the static bundle directory.

Audit-event reference limitation and migration

In the current agent-guard 0.3.4 v1 report and manifest, the audit-event artifact reference records only the sanitized path and role. It does not bind or verify event content identity, schema/profile semantics, or substitution. The demo's own event schema validation is separate and is not a cryptographic or content binding.

This v2 migration is dependency-gated. Only after a formally published compatible agent-guard release should maintainers update the exact hash pin, pass the identical event path and a recognized profile to the report/manifest producer and packaged consumer, regenerate v2 evidence, and test substitution rejection. Until then, retain the current 0.3.4 pin and v1 evidence.

Updating Digests

The digest policy pins files that define the public demo contract:

• AGENTS.md
• README.md
• scripts/run_agent_guard_bounded.sh
• scripts/policy_event_contract.py
• scripts/policy_admit.py
• scripts/validate_policy_event.py
• .agent-policy/policy.toml
• .agent-guard/mcp-policy.yaml
• .agent-guard/workflow-policy.yaml

After an intentional change to one of those files:

python3 scripts/update_digests.py
agent-guard digest check --root . --policy .agent-guard/context-digest-policy.yaml
bash scripts/run_agent_guard_bounded.sh python -m agent_guard.cli context lock --root . --policy .agent-guard/context-policy.yaml --check --digest-policy .agent-guard/context-digest-policy.yaml --json
bash scripts/run_agent_guard_bounded.sh python -m agent_guard.cli report --root . --context-policy .agent-guard/context-policy.yaml --evidence-preset recommended --api-policy .agent-guard/api-policy.yaml --mcp-policy .agent-guard/mcp-policy.yaml --digest-policy .agent-guard/context-digest-policy.yaml --agent-policy-audit-event .agent-policy/evidence/policy-admission-event.json --format json --output .agent-guard/evidence/agent-guard-report.json

Public Safety Scope

The repository intentionally avoids private corpora, local automation state, credentials, and private repository examples. Guard-regression payloads that need executable checks are generated inside tests at runtime rather than stored as committed payload files. The small committed fixtures/adversarial/ corpus is inert, dummy-valued, and fenced for documentation and review only; production scripts and the demo runner do not import or execute it.

The policy choices here are examples, not a universal safety model. Real maintainers should adapt capability names, review thresholds, and static guard patterns to their own repositories.