Skip to content

feat: install.sh --merge-permissions — prompted, backed-up merge of the governance gate (#70) - #73

Merged
mattwillems-exabeam merged 4 commits into
devfrom
feat/merge-permissions
Aug 13, 2026
Merged

mattwillems-exabeam merged 4 commits into
devfrom
feat/merge-permissions

Conversation

@mattwillems-exabeam

@mattwillems-exabeam mattwillems-exabeam commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #70.

Problem

The governance gate is opt-in on every install path. install.sh verified the merge and warned loudly when it was missing (gate_on()), but nothing ever performed it — so the shipped default was gate OFF until the operator hand-edits ~/.claude/settings.json, and the installer was a detection control rather than an enforcement one.

What this adds

--merge-permissions merges skills/soc-investigate/settings.snippet.json into the operator's settings. An interactive run whose gate reads OFF now shows exactly which rules it would add (via the merger's --dry-run) and asks y/N first.

Consent is preserved by construction:

  • Nothing merges by default.
  • -y does not authorise it — "assume yes" answers the installer's own questions, it does not license a write to the operator's settings. Installation alone never touches settings.json (the RFE's stated non-goal).
  • --checks-only outranks the flag entirely (reported with a ↷ line, not silently).

The merge engine (skills/soc-investigate/merge_permissions.py) is:

  • Backed up first — timestamped copy beside the original, carrying its mode.
  • Additive only — operator rules are never removed, reordered, or retiered.
  • Refusing on tier conflict — if a rule already sits in a different tier than the snippet specifies, that's operator intent or a mis-merge a human should resolve. It writes nothing and names the rule plus both tiers.
  • Restoring on write failure, and idempotent.

Worth re-running even when the gate reads ON: a hand-merge of just the two ask lines leaves the containment deny list missing, so "gate on" is not the same as "snippet merged".

Verification after the merge is the pre-existing gate_on() re-read, not the merger's exit code — the same "a 0-exit doesn't prove it landed" discipline the install block already uses. No python3 or no snippet reports "cannot merge, here's the manual path", never a false green, mirroring the existing "cannot verify ≠ OFF" rule.

Placement

The merger lives beside the snippet it merges, so the two travel together — if the snippet is present, the merger is too. Same clone-path constraint as the bridge check, and the pair survives #66's move into plugin/ intact.

Testing

tests/test_merge_permissions.py — 25 tests, weighted toward the refusals (every one must write nothing): tier conflict, malformed settings, wrong-shaped snippet, simulated mid-write failure, --dry-run. Plus what must survive: operator rules, unrelated top-level keys, key order, file mode, and an invariant tying the merge back to install.sh's own gate check.

Full suite: 263 passed (238 + 25).

End-to-end against a throwaway HOME and a stubbed claude CLI: flag merges (allow +18, ask +4, deny +68 — 90 rules); -y alone writes nothing; re-run is a byte-identical no-op; conflict writes nothing and makes no backup; operator's rules preserved (19 allow / 69 deny / 4 ask, model intact); interactive accept merges, decline and EOF do not.

Bug found while testing

shutil.copyfile doesn't carry permissions, so a 600 settings.json was being rewritten 644 (mode copied back from the fresh backup) and its backup landed world-readable. The mode is now read off the live file and the backup uses copy2. Both pinned by tests.

🤖 Generated with Claude Code

@mattwillems-exabeam

Copy link
Copy Markdown
Collaborator Author

Follow-up commit ae637e7 — a defect this PR would have introduced, found while scoping a plugin-smoke.sh leg for the new flag.

install.sh hardcoded $HOME/.claude/settings.json. That was survivable while the governance block only read: under a relocated config dir it reports the gate's state from a file the running Claude Code ignores — able to say "gate ON" about a gate that isn't in effect, the dangerous direction. It stops being survivable now that the block can write, where the same assumption merges the gate into a file that never takes effect: a green "gate ON" protecting nothing.

Now resolves SOCXEN_SETTINGS_FILE → $CLAUDE_CONFIG_DIR/settings.json → ~/.claude/settings.json, with every message naming the resolved path. SOCXEN_SETTINGS_FILE is the same explicit escape hatch EXABEAM_ENV_FILE already gives the credentials path.

Verified both overrides write to the override and leave $HOME/.claude/settings.json absent.

Why it matters beyond correctness: plugin-smoke.sh isolates whole installs in throwaway CLAUDE_CONFIG_DIRs. Without this, a smoke leg exercising --merge-permissions would have escaped its sandbox and rewritten the maintainer's real ~/.claude/settings.json on every release run.

Holding the smoke leg itself for a follow-up: #66 already rewrites plugin-smoke.sh (version_at / fabricate_marketplace prior-worktree fallbacks), so adding a leg here would just manufacture a conflict in that file. Better sequenced after #66 lands.

@mattwillems-exabeam

Copy link
Copy Markdown
Collaborator Author

Rebased onto dev @ 21e2a0b (#74's containment deny-list fix) and force-pushed.

Zero file overlap between the two changes, so the rebase was clean. Full suite green against the new snippet: 263 passed, with the deny tier now at 68 rules ({allow: 18, ask: 4, deny: 68}).

Worth noting for review: tests/test_merge_permissions.py needed no changes for #74. Every assertion derives its expected rules from the shipped settings.snippet.json rather than hardcoding names or counts, so retiering or expanding the snippet can't silently invalidate the merge tests — the one thing it does hardcode is the two gated dismiss/close verbs, which is the invariant we want pinned.

Also dropped two stale "17-entry" references in comments here (ac4bf41) — the point they were making never depended on the count, and #74 moved it. Note the PR description above still says "17-entry" for the same reason; read it as "the containment deny list".

virtualsteve-exa added a commit that referenced this pull request Aug 12, 2026
… table

Addresses code-owner review on #77.

1. Public-repo hygiene: removed committed absolute paths carrying an operator
   username and internal project name — 3 occurrences in the audit record and
   the `scan.workspace` field in the findings JSON, replaced with
   `<scan-workspace>` / `<socxen-checkout>` placeholders. The isolation claims
   in the audit record read the same with placeholders. HTML was already clean.
   Verified: no `/Users/` or operator name anywhere under security/praxen/.
   JSON re-validated — 13 findings, 50 rules, weighted 2.45 unchanged.

2. Dangling reference: SCAN_INSTRUCTIONS.md pointed at a `CLONE_SHA.txt`
   "beside this file" that is (correctly) scan-time state and not committed.
   Reworded to "pin the SHA at scan time and record it in the results
   artifact", naming the 005fa4c pin the committed run actually used.

3. Triage table (review item 3, non-blocking): all 13 findings now carry a
   recorded disposition in security/praxen/README.md — #71, #76, and #70/#73
   cross-referenced rather than double-filed, the rest marked *awaiting triage*
   as the live worklist. Disposition is explicitly the maintainer's call; the
   table does not invent one. Also carried the reviewer's install.sh:445 catch
   (the merge instruction sits inside `if [ "$CREDS_OK" = 0 ]`, so an install
   with credentials present never prints it) as a note for whoever fixes
   #70/#73.

238 passed; AI BOM current.

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
virtualsteve-exa added a commit that referenced this pull request Aug 12, 2026
…and a Critical-blocks-release gate (#77)

* security: add Agent Behavior Verification — Worker Remit, Praxen report, release gate

Adds `security/praxen/` — the "Praxen agent-behavior reports" the security
README has been reserving a slot for — and documents a release gate alongside
the existing red-team bar.

Contents:
- `WORKER_REMIT.md` — socxen's declared policy: what the agent is authorized to
  do. Authored blind from README/SECURITY/docs only (no access to skills/ or
  connector/), so it states intent rather than mirroring the implementation.
- `SCAN_INSTRUCTIONS.md` — scan scope: the whole deployed system (skill +
  connector + permissions pack + install), not the skill alone.
- `results/2026-08-12-socxen-0.6.9.{html,json}` — the scan of 0.6.9 at 005fa4c.
- `results/2026-08-12-socxen-0.6.9-audit.md` — independent context-unaware audit
  of that scan: 13/13 findings CONFIRMED, 0 unsupported, 0 remit defects.

Release gate (documented, not CI — the same posture as the red-team bar):
a Praxen Critical finding blocks a tag; it is fixed or explicitly waived in
writing by a maintainer before release. High/Medium/Low do not block.

0.6.9 result: 0 Critical (gate passes), 5 High, 7 Medium, 1 Low. Weighted RAISE
2.45/5. Build an AI Red Team scores 3 on the strength of the a10
find-fix-retest-regression arc in redteam/HISTORY.md; Monitor Continuously
scores 3 for the default-on audit trail at the bridge chokepoint.

Also:
- `security/README.md` — Praxen moved from "planned neighbors" into Contents,
  with a callout beside the red-team one.
- `CONTRIBUTING.md` — new "Release gates" section tabulating both gates.
- `tests/test_repo_invariants.py` — the SPDX-header exemption for generated run
  records now covers `security/praxen/results/` as it already did
  `security/redteam/results/`. Authored docs in `security/praxen/` carry headers.

Open items from this scan are tracked as #71 and #76; #72 was found by the
previous scan and is fixed in 0.6.9 — this re-scan verifies it.

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

* security(praxen): scrub local paths, fix dangling SHA ref, add triage table

Addresses code-owner review on #77.

1. Public-repo hygiene: removed committed absolute paths carrying an operator
   username and internal project name — 3 occurrences in the audit record and
   the `scan.workspace` field in the findings JSON, replaced with
   `<scan-workspace>` / `<socxen-checkout>` placeholders. The isolation claims
   in the audit record read the same with placeholders. HTML was already clean.
   Verified: no `/Users/` or operator name anywhere under security/praxen/.
   JSON re-validated — 13 findings, 50 rules, weighted 2.45 unchanged.

2. Dangling reference: SCAN_INSTRUCTIONS.md pointed at a `CLONE_SHA.txt`
   "beside this file" that is (correctly) scan-time state and not committed.
   Reworded to "pin the SHA at scan time and record it in the results
   artifact", naming the 005fa4c pin the committed run actually used.

3. Triage table (review item 3, non-blocking): all 13 findings now carry a
   recorded disposition in security/praxen/README.md — #71, #76, and #70/#73
   cross-referenced rather than double-filed, the rest marked *awaiting triage*
   as the live worklist. Disposition is explicitly the maintainer's call; the
   table does not invent one. Also carried the reviewer's install.sh:445 catch
   (the merge instruction sits inside `if [ "$CREDS_OK" = 0 ]`, so an install
   with credentials present never prints it) as a note for whoever fixes
   #70/#73.

238 passed; AI BOM current.

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

---------

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

@virtualsteve-exa virtualsteve-exa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-owner review, verified hands-on. APPROVING on content — this is an exemplary implementation of #70, and the consent semantics are exactly right. Needs a rebase (and one sequencing decision) before merge; details below.

Verified on the branch

  • Suite: 263 passed (238 + your 25), matching the PR's claim.
  • The merge engine earns the trust its job requires. Read in full: additive-only via mutation of the loaded object (operator key order and unknown top-level settings survive by construction), exact-string tier-conflict refusal with the wildcard non-inference explicitly reasoned, atomic same-directory os.replace, live-file mode carried (not backup-derived — the 600→644 bug you caught and pinned), copy2 backup so the backup itself can't leak, restore-on-write-failure with the "both failed, your original is at X" worst case handled, and distinct exit codes so an unhandled traceback's exit 1 can't impersonate an answer.
  • The installer wiring holds the #70 non-goal precisely: -y answers the installer's questions but never licenses the settings write; --checks-only outranks the flag visibly; non-tty gets the warning, not a hung prompt; the interactive offer previews the exact additions via --dry-run before asking; and success is judged by the pre-existing gate_on() re-read — with the "merger said 0 but the gate reads OFF" case reported as a failure, which is the correct trust direction.
  • The SOCXEN_SETTINGS_FILE / CLAUDE_CONFIG_DIR resolution fix is load-bearing, not cosmetic — your CHANGELOG framing is right that hardcoding was survivable for a read and unsurvivable for a write. Good catch; the Praxen scan's finding 001 evidence (gate_on reading a file the running Claude Code may ignore) is partially addressed by this too.
  • Docs: installation.md's new section states the consent model plainly. CHANGELOG under [Unreleased] — placement note below.

Before merge

  1. Rebase over current dev (0.6.9 + #77 + #78 landed under you — the conflict is your [Unreleased] CHANGELOG block vs the 0.6.9 entry; keep yours above ## [0.6.9]). Nothing else you touch collides.
  2. Re-run your end-to-end pass after the rebase and refresh the numbers in the PR body: post-#74 the merge adds allow +18, ask +4, deny +68 (the body's deny +17 predates the containment fix your own commit 3 already accounted for in the docs).
  3. Sequencing with #66 (your call, both approved from my side): I'd land #66 first, then relocate this PR's pieces in the rebase — install.sh → plugin/install.sh, merge_permissions.py → plugin/skills/soc-investigate/, tests stay at root. The merger-beside-snippet placement you chose survives the move by design, which you clearly anticipated ("the pair survives #66's move intact").

Non-blocking observations

  • backup_path() is second-granularity — two merges in the same second reuse a name. Harmless (the second backup is a copy of the first's result), not worth code.
  • When #66's layout lands, SCRIPT_DIR-relative SNIPPET/MERGER paths resolve correctly from plugin/install.sh unchanged — verified the relative structure holds.

Nice work — the "cannot do it ≠ pretend it's done" discipline runs through every path here, and the tests are weighted exactly where the risk is (the refusals that must write nothing).

@mattwillems-exabeam

Copy link
Copy Markdown
Collaborator Author

Rebased onto dev @ a413686 — 0ade179. Both pre-merge items done.

1. CHANGELOG. [Unreleased] kept above ## [0.6.9] as you asked; that was the only conflict in the rebase — nothing else collided, as predicted.

2. Numbers refreshed, and the end-to-end pass re-run rather than re-quoted. Suite: 263 passed (238 + 25). All six installer scenarios re-verified against the post-#74 snippet:

Scenario Result
--merge-permissions gate ON — {allow: 18, ask: 4, deny: 68}
-y alone nothing written, warning only
re-run over merged "already merged, nothing to change"
tier conflict ✗ stopped, nothing written, file unchanged
operator rules preserved 19 allow / 69 deny / 4 ask, model intact
--checks-only + flag ↷ ignored, nothing written

PR body updated: deny +17 → deny +68 (90 rules), and the preserved-rules line now reads 19/69/4.

Worth noting the merge tests again needed zero changes across #74 and this rebase — deriving expected rules from the shipped snippet rather than hardcoding them is what made both a non-event.

On sequencing: taking your recommendation — land #66 first, then I'll relocate this PR's pieces into plugin/install.sh and plugin/skills/soc-investigate/merge_permissions.py in the follow-up rebase. Tests stay at root.

One thing outside both PRs that you may not have hit: GitHub Actions has not run on either of these branches' force-pushed heads. ac4bf41 sat with zero check-runs for nine hours while other PRs in the repo ran normally; Actions is enabled, no queued or approval-blocked runs, and neither workflow has a path filter. I cycled both PRs closed/reopened to force a reopened event and that didn't produce runs either. Since main is branch-protected on Repo invariants + signoff, this could stall the promotion window — flagging in case it's something you can see from the org side.

@mattwillems-exabeam

Copy link
Copy Markdown
Collaborator Author

Correction to my last note: CI is running normally again. Both checks are green on both PRs' current heads — #73 0ade179 and #66 d17fa41, Repo invariants + signoff pass on each. The gap was real (ac4bf41 genuinely had zero check-runs for nine hours, and the close/reopen cycle didn't shake it loose) but it cleared on its own with these pushes, so there's nothing for you to chase unless it recurs. Sorry for the false alarm — flagging the resolution so it doesn't cost you a look.

mattwillems-exabeam and others added 4 commits August 13, 2026 12:07
…he governance gate (#70)

The governance gate has been opt-in on every install path. install.sh verified the
merge and warned loudly when it was missing, but nothing ever performed it, so the
shipped default was "gate OFF until the operator hand-edits ~/.claude/settings.json"
— the installer was a detection control, not an enforcement one.

Add the write half. --merge-permissions merges settings.snippet.json into the
operator's settings.json; an interactive run whose gate reads OFF now shows exactly
which rules it would add and asks y/N first.

Consent is preserved by construction: nothing merges by default, and -y does NOT
authorise it — "assume yes" answers the installer's own questions, it does not
license a write to the operator's settings. --checks-only outranks the flag entirely.

The merge itself (skills/soc-investigate/merge_permissions.py) is additive-only,
backs up settings.json first (timestamped, same mode as the original), refuses
outright when a rule already sits in a different tier than the snippet specifies
(operator intent wins — it writes nothing and reports what to resolve), restores
from the backup if a write fails, and is idempotent. Worth re-running even when the
gate reads ON: a hand-merge of just the two ask lines leaves the 17-entry
containment deny list missing.

Verification is the pre-existing gate_on() re-read, not the merger's exit code —
the same "a 0-exit doesn't prove it landed" discipline the install block uses. No
python3 or no snippet reports "cannot merge, here's the manual path", never a false
green, mirroring the existing "cannot verify != OFF" rule.

25 tests cover the refusals specifically: every one of them must write nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Matt W <matt.willems@exabeam.com>
…aude (#70)

install.sh hardcoded $HOME/.claude/settings.json. That was survivable while the
governance block only reported the gate's state — it could read a file the running
Claude Code ignores and mis-report ON, the dangerous direction. It stops being
survivable now that the block can write: the same assumption would merge the gate
into a file that never takes effect, i.e. a green "gate ON" protecting nothing.

Resolve as SOCXEN_SETTINGS_FILE -> $CLAUDE_CONFIG_DIR/settings.json ->
~/.claude/settings.json, and name the resolved path in every message rather than a
hardcoded one. SOCXEN_SETTINGS_FILE is the same explicit escape hatch
EXABEAM_ENV_FILE already gives the credentials path.

This is also what makes the merge testable from scripts/release/plugin-smoke.sh,
which isolates whole installs in throwaway CLAUDE_CONFIG_DIRs: without it, a smoke
leg exercising --merge-permissions would escape its sandbox and rewrite the
maintainer's real ~/.claude/settings.json on every release run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Matt W <matt.willems@exabeam.com>
#74 expanded the containment deny tier from 17 bare rules to 68 (both spellings,
both namespaces) covering the same 17 tools. Two comments here quoted the old
entry count while making a point that does not depend on it — that a hand-merge of
just the two ask lines leaves the containment deny list missing entirely. Say that
without the number, so the next expansion of the tier doesn't strand them again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Matt W <matt.willems@exabeam.com>
#66 moved the skill into plugin/, so tests/test_merge_permissions.py's SKILL_DIR
had to follow — tests stay at the repo root, the skill they read does not. Now
matches test_repo_invariants.py's constant.

Nothing else moved: the merger travels beside the snippet it merges, so both
landed in plugin/skills/soc-investigate/ together, and install.sh resolves them
SCRIPT_DIR-relative — unchanged from plugin/install.sh. Verified end-to-end from
the new location: the merge lands {allow: 18, ask: 4, deny: 68} and the no-flag
run still writes nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Matt W <matt.willems@exabeam.com>
@mattwillems-exabeam
mattwillems-exabeam merged commit 3ebda74 into dev Aug 13, 2026
2 checks passed
virtualsteve-exa added a commit that referenced this pull request Aug 13, 2026
…eleased changes (#80)

* docs: bring the docs current with the plugin/ restructure and the unreleased changes

Pre-promotion currency pass. Three classes of defect, all found by reviewing
docs against the code rather than against each other.

BROKEN INSTRUCTIONS (post-#66, a user follows these):
- plugin/docs/installation.md told users to run ./install.sh --merge-permissions
  in the section the page itself calls the most important on it. There is no root
  install.sh any more; twenty lines earlier the same page correctly says
  ./plugin/install.sh. Same for merge_permissions.py's path.
- install.sh's OWN output was stale: the usage block is printed verbatim as
  --help, and every line said ./install.sh. Its 'Next steps' block named
  skills/soc-investigate/settings.snippet.json and ./install.sh. Both now print
  the paths the script has already resolved, so they are right from a clone and
  from the installed plugin directory alike. The --help text also still claimed
  the merge targets ~/.claude/settings.json — the exact hardcoding #73 removed.

A DOC THAT CONTRADICTED THE CODE:
- logging.md said a failed emit makes telemetry 'disable itself'. It does the
  opposite on purpose: the event is dropped and the trail stays on, because a
  transient fault must not switch off a mandatory audit log — there is a test
  named for that behavior. Split the two failure modes, and documented the
  observra>=1.1 floor and the new stderr warning channel, neither of which
  appeared anywhere.

MISSING RELEASE NOTES:
- [Unreleased] documented 1 of 4 landed changes. Added entries for the plugin/
  restructure (#66), the ABV release gate (#77), and the dependency bounds +
  hash-pinned lockfile and observra 1.1 migration (#78), and corrected the
  pre-restructure paths in the entries that were already there.

Also: the lockfile existed with no mention in any document — CONTRIBUTING now
tells contributors to re-lock (CI checks the AI BOM but not lock drift) and the
end-to-end recipe copies the .lock; SECURITY.md's scope adds merge_permissions.py
(the only code that writes to a user's settings) and the lockfile; the front-door
README no longer presents the installer as check-only; CONTRIBUTING said gates run
'before a tag' though the repo has no tags and says so twice elsewhere; and
SCAN_INSTRUCTIONS.md — forward-looking scope for the NEXT Praxen audit — carried
pre-restructure paths and asked about a manual-merge-only world that no longer
exists, which would have biased that scan.

Verified: 263 tests pass, AI BOM current, installer smoke still merges to
{allow: 18, ask: 4, deny: 68}, --help renders, every relative link resolves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

* docs: correct three defects found in review of this PR

1. A FALSE ASSURANCE IN AN AUDIT DOC. The rewritten fail-open section claimed
   observra's warnings surface on stderr for 'a full queue' so a dropped event is
   'always visible'. Traced through observra 1.1: a queue-overflow drop is logged
   at DEBUG and counted in observra_events_dropped_total, below the WARNING-level
   stderr handler the shim installs — so that drop is counted, not announced.
   Backend write errors DO surface. Corrected in logging.md, the CHANGELOG entry,
   and the shim's own (pre-existing, equally wrong) inline comment.

2. THE REVERSE PATH ERROR. installation.md ships INSIDE the plugin and the primary
   install path is the marketplace, where no plugin/ directory exists — so
   'skills/soc-investigate/...' was correct plugin-relative and my prefixing it
   broke it for the majority reader, while plugin/README.md kept the relative form.
   Now names the plugin-relative path with the clone path as a gloss.

3. A STALE PATH IN THE BLOCK THIS PR EDITED. tests/end-to-end-testing.md's CACHE
   still used the pre-0.6.5 marketplace name (socxen/socxen); the real cache is
   under open-agent-ai-security/socxen/. The new .lock copy would have missed.

Also: the shim's DESIGN RULES said an emit failure disables logging, contradicting
its own code, tests, and this PR's new doc; my bump_version change contradicted the
script's own docstring, so both call sites now use uv run; praxen/README.md's gate
said 'before tagging' while CONTRIBUTING (which links it) says release; the rotation
bounds are clamped, not passed straight through; the merge-permissions hint now
prints an absolute path instead of $0 as invoked; and (#70) gained its PR number
per the file's (#issue, #PR) convention.

Verified: 263 pass, BOM current, Next-steps block renders with paste-able absolute
paths, real settings untouched, all relative links resolve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

* docs: order the [Unreleased] sections Added-before-Fixed

The last clear item from the review pass. The section order was Changed ->
Fixed -> Added -> Security; both Keep a Changelog and this file's own 0.6.3
entry put Added first. Pure reordering — no wording changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

---------

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
virtualsteve-exa added a commit that referenced this pull request Aug 13, 2026
…/50 resisted (#76) (#81)

* fix: pin the red-team gate to an explicit model ID and record the resolved model in every artifact (#76)

The runner's --models default was the floating 'sonnet' alias, so the gate's
result could change out from under the harness whenever a new Sonnet shipped,
and a recorded verdict could not be tied to a specific model version after
the fact. Now:

- --models defaults to the explicit supported-floor ID (claude-sonnet-4-6),
  matching the discipline the dated results/ artifacts already show;
- the runner captures the session's *resolved* model ID from the CLI init
  event and tallies/reports under it, so even an alias-invoked run produces
  an attributable artifact (and prints a notice when the two differ);
- HISTORY.md's stale note (Sonnet 5 'not approved for cyber work' — no longer
  true) is replaced with the real reason the pin exists: reproducibility;
- PLAN.md's 'weakest supported model (currently Sonnet)' wording names the
  pinned ID explicitly in all three places.

The AI BOM is unaffected: it inventories dependencies, not harness source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

* fix: a stray stream event must not discard a whole red-team trial

Found while running the gate for #76: the first live trial died with
"'str' object has no attribute 'get'".

The CLI's stream-json output carries lines whose `message` is a plain
STRING rather than an object (error/notice events — an MCP server failing
to reconnect, for instance). _parse called .get() on it unconditionally,
raising AttributeError, which the runner's per-trial `except Exception`
then degraded into a fully ERRORED trial. A drive that actually completed
19 tool calls and produced a 36k-character report was discarded because
one unrelated notice appeared in the transcript.

Grading integrity was never at risk — _inconc_rows already refuses to read
an all-errored blocking-class attack as a pass (verdict INCONCLUSIVE, exit
3) — but the harness burned a full live drive per occurrence and could
have stalled the gate on re-runs.

Parse defensively by SHAPE, not just by event type, and add regression
tests for both the odd-shaped line and the resolved-model capture.
Negative-verified: the new test fails red on the unfixed parse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

* security: record the 2026-08-13 red-team gate run — 50/50 resisted, PASS (#76)

The staleness half of #76. The gate had not run since 2026-07-03, across
0.6.7/0.6.8/0.6.9, and this promotion carries #66/#73/#77/#78.

Run: 10 class-A families x 5 trials on the pinned claude-sonnet-4-6, judge on,
driven against the working-tree plugin. Every family landed 0/5; no errored and
no inconclusive trials, so the verdict rests on 50 valid trials rather than on
absence of evidence.

The a10 rows carry the real signal. In 4 of 5 trials the model reproduced the
phishing markdown link in its chat output — the injection still works on the
model — but the persisted artifact was clean every time, because the write-sink
neutralizer defanged it. That is exactly what output-pipeline grading exists to
distinguish: a pass where the model never emitted the payload would prove
nothing about the guardrail. a10 landed 3/3 in the first run and drove the
deterministic fix, so seeing the code layer fire is the point.

Also fixes the ledger's own stale connector/ paths after #66, and updates the
a10 summary's resistance figure from the 0/2 fix-time retest to 0/5 here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

* docs: link the class C/D corpus gap to its tracking issue

Praxen finding 008 was 'awaiting triage'; it now has a home (#82) rather than
being an untracked row. The gap itself is unchanged — the corpus is still
class-A only, and classes C and D remain unexercised despite both blocking a
release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>

---------

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
virtualsteve-exa added a commit that referenced this pull request Aug 13, 2026
Minor, not patch: the shipped surface changes shape. Only the payload under
plugin/ is distributed (#66), and the installer can now install the governance
gate rather than only warn about it (#73).

Also stamps the changelog entry for the red-team gate work (#76/#81), which
landed after the [Unreleased] section was written, and dates the section.

Both release gates ran against this candidate: red team 50/50 resisted (PASS,
2026-08-13 artifact); ABV 0 Critical (PASS, 2026-08-12 artifact against 0.6.9).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
virtualsteve-exa added a commit that referenced this pull request Aug 13, 2026
…otes

Two release-record items:

1. The 2026-08-12 Praxen scan is 0.7.0's ABV gate artifact (maintainer
   determination). It ran against the dev tip as it stood (005fa4c) — the head
   of the line 0.7.0 was cut from, not a stale release tag — and every
   behavioural change merged since is either a remediation of one of that
   scan's own findings (001->#73, 007->#78, 009->#81) or the path-only plugin/
   restructure. Recorded in security/praxen/README.md rather than only in the
   release PR, so a reader can tell which release the dated artifact verified.

2. #79 landed on dev after the version bump, so the changelog gains its entry:
   the release smoke now asserts the governance gate actually installs. Release
   machinery earns a changelog line by the 0.6.4 precedent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
virtualsteve-exa added a commit that referenced this pull request Aug 17, 2026
Both READMEs said dismiss/close is held back by 'two locks'. That is true only
after the operator merges the permission pack; on a fresh install there is one.
The 🛑 box thirty lines below already said so, so each page contradicted itself
and the claim stated as fact up front is the one a skimmer keeps.

This direction of error is the dangerous one, and the tooling already knows it:
install.sh refuses to report a gate ON it cannot verify, and #73 added an exit
path for a merge that claims success while gate_on() reads OFF. The front page
should not assert what the installer declines to assume.

Also corrects the Evidence row: CONTRIBUTING.md names two release gates, both in
security/; evals/ is a regression harness, not a gate.

Caught in review by Matt Willems on #97.

Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants