Skip to content

Make governance enforcement deterministic (dismiss/close gate + containment deny-list are prompt/snippet-only) #3

Closed

Description

@virtualsteve-exa

Source: Praxen audit of socxen v0.3.3 — PRAX-2026-07-01-002 (High), PRAX-2026-07-01-007 (Medium); compound PRAX-2026-07-01-003 (High).

Problem. Every governance control is soft. The dismiss/close gate (exabeam_update_alert / exabeam_update_case require a human "yes") is enforced only by (a) the settings.snippet.json ask rule — inert until an operator merges it — and (b) the SKILL.md "ask and WAIT" instruction — which SKILL.md itself notes is bypassable via --dangerously-skip-permissions / auto-accept. Likewise the 17-tool containment deny-list ships as an unmerged snippet and must be hand-synced with containment-tools.md. Net: nothing survives a bypassed permission mode except the model's own compliance, and a future response tool under a name not in the hand-maintained list would fire ungated.

Evidence. settings.snippet.json:23 (ask rules in a *.snippet template); settings.snippet.json:29-47 (deny block, inert until merged); SKILL.md governance tier 2 (concedes the prompt "can be switched off"); containment-tools.md:56 ("keeping the two in sync is what makes the gate real").
OWASP: LLM06 Excessive Agency · ASI02/ASI03. RAISE: Implement Zero Trust.

Recommended fix.

  • Ship the approval gate + deny-list as active configuration or a deterministic Claude Code PreToolUse hook, so update_alert/update_case cannot fire without a real human response even under auto-accept, and containment stays blocked regardless of merge state.
  • Generate the deny-list from a single source of truth so containment-tools.md and the active settings can't drift.
  • Interim (in-skill): require the agent to restate the pending suppression + the received "yes" verbatim in the case note before calling the update tool.

Decision needed: this grows socxen a thin code layer (a hook) vs. staying pure-skill. Praxen (and Claude) lean toward the hook for the one irreversible-ish action. Priority: High (structural) — the architecturally-correct fix for the "bypassable gate."

Activity

  1. virtualsteve-exa commented on Jul 1, 2026

    @virtualsteve-exa
    CollaboratorAuthor

    Concrete proposal for the deterministic-enforcement decision (so it's not buried in this issue): #9 — a bundled PreToolUse hook that hard-denies containment and gates dismiss/close independent of permission mode, plus structured logging (#5).

  2. virtualsteve-exa commented on Jul 1, 2026

    @virtualsteve-exa
    CollaboratorAuthor

    Correction: this is a hard gate, not "soft enforcement"

    On review, the "soft enforcement" framing (inherited from the Praxen finding) is imprecise. Claude Code's permission system is a hard, deterministic, harness-enforced control: when update_alert/update_case are in the active settings.json ask list, Claude Code prompts the human before the tool runs and the model cannot execute it without a real approval; deny-listed containment tools are hard-blocked. That's the harness, not the model's compliance.

    The real weaknesses are operator-controlled activation gaps, not softness:

    1. Off by default — socxen ships settings.snippet.json as a copy-me template, so there's no gate until the operator merges it.
    2. --dangerously-skip-permissions / auto-accept globally disable the permission system.

    So the accurate problem statement is "the hard gate is opt-in to activate and operator-disableable," and the fix is to make it active-by-default and bypass-resistant. Two paths worth verifying:

  3. virtualsteve-exa commented on Aug 13, 2026

    @virtualsteve-exa
    CollaboratorAuthor

    Status check, 2026-08-13 (v0.7.0) — reviewed against current code during a tracker sweep. Still open. I had expected #73 to close this; it doesn't, and the difference is worth stating precisely because it's easy to mistake one for the other.

    What this issue asked for: the gate as active configuration or a deterministic PreToolUse hook, such that dismiss/close can't fire without a real human response even under auto-accept, and containment stays blocked regardless of merge state.

    What shipped instead: an assisted merge of the permissions snippet — merge_permissions.py plus install.sh --merge-permissions and an interactive offer when the gate reads OFF (v0.7.0, #70/#73). Materially better than warning-only, but a different mechanism. There is no PreToolUse hook anywhere in the repo (grep -rn PreToolUse hits only a test fixture asserting an operator's unrelated hooks survive the merge), and plugin.json declares no hooks and no permissions.

    Against the literal asks:

    Genuinely satisfied: the single-source-of-truth ask for the deny-list. It's still hand-maintained, but tests/test_repo_invariants.py::test_deny_list_matches_containment_doc now derives the expected rule set from containment-tools.md and fails on missing or extra rules, in both spellings and both namespaces. Drift is caught in CI rather than made structurally impossible — I'd accept that as meeting the intent. (v0.6.9, #72/#74, which also fixed the bare-name bug: 17 → 68 rules.)

    Never shipped, and cheap: the interim in-skill ask — require the agent to restate the pending suppression and the received "yes" verbatim in the case note before calling the update tool. SKILL.md requires stating the action and waiting for a yes, but nothing records the approval. reference/report-template.md:18 has only "Analyst action taken:" with no approval-capture field. That's a ten-minute change and it's the only part of this issue that doesn't depend on a hook.

    Suggested re-scope — retitle to "the gate is opt-in to activate and defeated by permission bypass," credit #72/#74 and #70/#73 for what landed, and carry three items: (a) survive --dangerously-skip-permissions → that's #9's hook, (b) decide whether the gate should be active-by-default, (c) ship the verbatim-approval interim rule now.

  4. virtualsteve-exa commented on Sep 5, 2026

    @virtualsteve-exa
    CollaboratorAuthor

    #148 makes the dismiss/close gate and the containment deny deterministic on Claude Code without an operator merge: a PreToolUse hook bundled in the plugin, active the moment the plugin is enabled, reading the same tier file the permission snippet and the Codex map are built from. "Inert until merged" no longer applies — the permission pack is an optional second lock. Details, evidence and the review record are on the PR; proposing to close this with it.

  5. virtualsteve-exa commented on Sep 6, 2026

    @virtualsteve-exa
    CollaboratorAuthor

    Fixed on dev by #148 (squash e234815), 2026-09-06. Ships with the next release to main. Dismiss/close ask and the containment deny are enforced by the bundled hook the moment the plugin is enabled, on both hosts; the permission snippet is an optional second lock.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions