feat: assess primitive — callable pre-flight risk check (CLI + MCP tool) (#149)#150
Merged
Conversation
…ion (#149) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cted from mcp_scan (#149) Extracts shared structural-detection primitives (is_shell, is_inline_exec_flag, launcher_basename, effective_shell_target, is_transient_path, is_env_assignment, starts_with_seq, contains_seq, first_destructive_after_shell_flag) from mcp_scan into a new ai_guard::command_scan module. scan_command scans a raw (command, args) pair for destructive content directly against the full preview string (C5: bare rm -rf /tmp/x without bash -c MUST emit DestructiveInInlineCommand). mcp_scan delegates to command_scan — zero duplicated logic. All 30 existing mcp_scan parity tests green; 7 new command_scan tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#149) Add public helper `DenyEvaluator::evaluate_bash_preview` that constructs a `HookAction::Bash` with the canonical blake3 hash of the preview and delegates to `evaluate()`, guaranteeing the assess primitive's deny verdict matches what sigil-hook enforcement would produce. Three TDD tests cover: deny match, no-match (benign), and parity with direct `evaluate()`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `AiGuardBucket::{PartialOrd,Ord}` (variants already ascending), create
`ai_guard::assess::{AssessCtx, assess}` with 6 TDD tests covering destructive
deny, safe allow, MCP shell launcher, deny-rule-forces-deny, url+command parity,
and determinism. Register module in `ai_guard/mod.rs`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…. rule-pack bundle (#149) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…odes (#149) Add `sigil assess` subcommand (operator-cli gated) that evaluates a proposed command or MCP server definition against the host's loaded cold-disk policy and exits 0 (allow/warn), 2 (deny), or 1 (usage/input/policy-load error). Includes fail-closed input-size limits, XOR mode validation, and a pure `exit_code(Decision, fail_on_warn)` function covered by unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Request::Assess / Response.assess_verdict to the shared control protocol, handle it in sigil-agent (snapshot-clone Rubric + DenyEvaluator under short read guards, evaluate synchronously, return AssessVerdict), and expose pub async fn assess() in sigil-mcp's LocalUpstream so the MCP layer can ask the running daemon for a live-policy verdict. TDD: assess_round_trips_against_canned_agent added to local.rs and passes along with the full sigil-mcp + sigil-agent control test suites. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#149) Adds an `assess` MCP tool to SigilLocal in local_tools.rs. Accepts either a shell command (command + optional args) or an MCP server definition (mcp_server object + server_name) via an AssessParams struct, enforces XOR at runtime (fail-closed: neither or both → invalid_params), validates mcp_server is a JSON object, delegates to LocalUpstream::assess over IPC, and returns the full AssessVerdict as JSON. Updates get_info instructions to mention the new tool. TDD: three new tests (verdict round-trip, XOR enforcement, non-object rejection) all green; 35/35 sigil-mcp tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…y-parity docs (#149) Final holistic review follow-ups: - effective_policy: treat an empty (zero-byte/whitespace) rule-packs.yaml like absent (no bundle) instead of erroring, matching the daemon's boot/reload tolerance (#135) — a benign empty file no longer makes `sigil assess` exit 1. Only a Corrupt (parse-failed) bundle is fail-loud on cold load. - Document on the `--command` CLI help and the assess MCP tool description that the full command line should go in `command` for faithful deny-rule parity with the hook (splitting into args can re-join with different spacing). - MCP assess tool now rejects an empty `server_name` (symmetry with the CLI). - Drop the now-stale dead_code allow on LocalUpstream::assess (it is consumed by the MCP tool). - CHANGELOG: document the assess primitive. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #149.
What
A callable
assessprimitive that scores a proposed shell command or a single MCP server definition against this host's loaded policy (the same rubric + rule-pack deny rules the agent enforces), returning{ bucket, score, reasons, deny_match, decision }. Where the existing read surfaces report standing posture ("what is my risk now?"),assessanswers "is this action risky / would Sigil block it?" before it runs.Two surfaces over one pure engine (
ai_guard::assess):sigil assessCLI (cold-disk policy, no daemon):--command "<cmd>"or--mcp-config <file>/--mcp-stdin --mcp-name <name>. One-line JSON verdict; exit codes (allow/warn → 0, deny → 2, usage/policy-load error → 1;--fail-on-warn→ warn = 2) for shell pre-flight gating.assessMCP tool (local surface) → control IPC → daemon evaluates against its live loaded policy. Read-only.Design
Built via brainstorm → spec (rev2, codex adversarial review folded) → plan → subagent-driven implementation (8 tasks) → holistic review. Spec/plan are in
docs/superpowers/(gitignored).Key correctness properties:
assess.hook_deny::evaluate_bash_previewbuilds the sameHookAction::Bashthe hook enforces with, so the assess deny verdict matches runtime enforcement (shared canonical preview; parity tests).command_scanextracts the destructive / attack-shape primitives (ai-guard: score MCP stdio launchers by attack shape (shell / transient-path) above the benign baseline #127) frommcp_scanso the MCP-args path and the new raw-command path share one source (rawrm -rf /tmp/xwith nobash -cis now caught — a gap codex flagged). The 30 existingmcp_scanparity tests stay green.load_effective_policymerges defaults < policy.yaml < rule-packs.yaml bundle (not the doctor path that ignores the bundle), tolerating an empty bundle like the daemon does (rule-packs.yaml watcher: re-arm if the config dir is created/replaced after boot #135).AiGuardBucket+rubric::bucket(no invented band type).codex / holistic review
codex adversarial review of the spec caught 13 issues (all folded into rev2), including the raw-destructive gap, the wrong band type, the deny-evaluator signature, and fail-open-on-malformed-input. The final holistic review found 0 blocking issues; 2 should-fix items (empty-bundle CLI tolerance, deny-parity docs) and 2 nice-to-haves were applied in the last commit.
Tests / gates
cargo fmt+cargo clippy --workspace --all-targets -D warnings+cargo test --workspaceall green (84 test-result sections, 0 failures). New coverage: engine (command/MCP/deny/override/determinism),command_scan(raw-destructive regression guard + mcp_scan parity),evaluate_bash_previewhook parity,load_effective_policy(bundle layer + empty/absent tolerance), CLI exit-code matrix + fail-closed, IPC round-trip, MCP tool (verdict + fail-closed validation).Scope (deferred)
config-snippet input; MCP-def stdio-command deny evaluation; fleet/central assess; an
enforce_thresholdpolicy field (enforce bucket is currentlyHighin both paths with a TODO).🤖 Generated with Claude Code