Repository navigation
Release 0.7.0 — ship only plugin/, and let the installer turn the governance gate on - #83
Merged
Merged
Conversation
…and a Critical-blocks-release gate (#77) * security: add Agent Behavior Verification — Worker Remit, Praxen report, release gate Adds `security/praxen/` — the "Praxen agent-behavior reports" the security README has been reserving a slot for — and documents a release gate alongside the existing red-team bar. Contents: - `WORKER_REMIT.md` — socxen's declared policy: what the agent is authorized to do. Authored blind from README/SECURITY/docs only (no access to skills/ or connector/), so it states intent rather than mirroring the implementation. - `SCAN_INSTRUCTIONS.md` — scan scope: the whole deployed system (skill + connector + permissions pack + install), not the skill alone. - `results/2026-08-12-socxen-0.6.9.{html,json}` — the scan of 0.6.9 at 005fa4c. - `results/2026-08-12-socxen-0.6.9-audit.md` — independent context-unaware audit of that scan: 13/13 findings CONFIRMED, 0 unsupported, 0 remit defects. Release gate (documented, not CI — the same posture as the red-team bar): a Praxen Critical finding blocks a tag; it is fixed or explicitly waived in writing by a maintainer before release. High/Medium/Low do not block. 0.6.9 result: 0 Critical (gate passes), 5 High, 7 Medium, 1 Low. Weighted RAISE 2.45/5. Build an AI Red Team scores 3 on the strength of the a10 find-fix-retest-regression arc in redteam/HISTORY.md; Monitor Continuously scores 3 for the default-on audit trail at the bridge chokepoint. Also: - `security/README.md` — Praxen moved from "planned neighbors" into Contents, with a callout beside the red-team one. - `CONTRIBUTING.md` — new "Release gates" section tabulating both gates. - `tests/test_repo_invariants.py` — the SPDX-header exemption for generated run records now covers `security/praxen/results/` as it already did `security/redteam/results/`. Authored docs in `security/praxen/` carry headers. Open items from this scan are tracked as #71 and #76; #72 was found by the previous scan and is fixed in 0.6.9 — this re-scan verifies it. Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> * security(praxen): scrub local paths, fix dangling SHA ref, add triage table Addresses code-owner review on #77. 1. Public-repo hygiene: removed committed absolute paths carrying an operator username and internal project name — 3 occurrences in the audit record and the `scan.workspace` field in the findings JSON, replaced with `<scan-workspace>` / `<socxen-checkout>` placeholders. The isolation claims in the audit record read the same with placeholders. HTML was already clean. Verified: no `/Users/` or operator name anywhere under security/praxen/. JSON re-validated — 13 findings, 50 rules, weighted 2.45 unchanged. 2. Dangling reference: SCAN_INSTRUCTIONS.md pointed at a `CLONE_SHA.txt` "beside this file" that is (correctly) scan-time state and not committed. Reworded to "pin the SHA at scan time and record it in the results artifact", naming the 005fa4c pin the committed run actually used. 3. Triage table (review item 3, non-blocking): all 13 findings now carry a recorded disposition in security/praxen/README.md — #71, #76, and #70/#73 cross-referenced rather than double-filed, the rest marked *awaiting triage* as the live worklist. Disposition is explicitly the maintainer's call; the table does not invent one. Also carried the reviewer's install.sh:445 catch (the merge instruction sits inside `if [ "$CREDS_OK" = 0 ]`, so an install with credentials present never prints it) as a note for whoever fixes #70/#73. 238 passed; AI BOM current. Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> --------- Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
…ndencies (#71) (#78) observra 1.1 made the custom-host path first-class, so the telemetry shim drops two of its three private reach-ins: - _emit now goes through public observra.emit() (same event shape, same fail-open drop semantics) instead of create_event + observra._queue_proxy. emit() swallows its own faults into the observra logger, so _configure routes that logger's warnings to stderr — a dropped audit event stays visible to the operator either way. - Rotation bounds (max_bytes/backup_count) now pass through initialize() to the jsonl backend constructor (observra#84 is fixed in 1.1); the _worker._storage reach-in and its fallback path are gone. - The exit drain keeps its guarded _worker._shutdown() reach-in — the one private surface left; public shutdown()/flush() requested upstream as observra#117. Dependency hygiene (#71 rungs 1-2, motivated by this exact release: 1.1.0 rolled onto every fresh install silently the day it shipped, same channel that broke us with mcp 2.0): - PEP 723 bounds: observra floor raised to >=1.1,<2 (emit + kwarg forwarding don't exist below 1.1; every fresh resolve is already 1.1.0), httpx capped <1, certifi bounded >=2024,<2027, typing_extensions >=4.7,<5. All five deps now carry explicit ranges; AI BOM regenerated. - uv lock --script lockfile (connector/exabeam-mcp-bridge.py.lock) pins the resolve with hashes (mcp 1.29.0, observra 1.1.0, certifi 2026.7.22, 33 packages) — uv run picks it up automatically, so upstream releases become reviewed diffs instead of ambient drift. Verified: 238 passed; live bridge --check against staging connects clean under the locked env (21 tools). Remaining #71 rung: a scanner in CI — maintainer decision, tracked there. Refs #71 Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…out of the distributed plugin (#29) (#66) * restructure: ship only plugin/ via git-subdir; keep build artifacts out of the distributed plugin (#29) The community marketplace installs socxen with `source: url` over the WHOLE repo, so every user's plugin cache gets tests/, evals/, security/ (incl. the red-team attack payloads), scripts/, and .github/ — build-time artifacts that should never ship. Claude Code has no exclude/ignore mechanism; the only supported way to trim what installs is a `git-subdir` source pointing at a subdirectory. Move the distributable payload into `plugin/` (single source of truth — no staged dist/ copy): plugin/.claude-plugin/plugin.json · plugin/.mcp.json · plugin/skills/ plugin/connector/ · plugin/docs/ · plugin/install.sh · plugin/README.md plugin/LICENSE · plugin/NOTICE Left at repo root (never ship): tests/ evals/ security/ scripts/ .github/ and the community-health docs (CHANGELOG, CONTRIBUTING, SECURITY, CODE_OF_CONDUCT). All build tooling repointed to plugin/ (Path(__file__)-relative ROOT stays the repo root, so moved refs gain a `plugin/` segment): - tests/*.py (bridge/skills/settings/connector loads; invariant marketplace assertion + bridge self-ref rebased to plugin/) - evals/run.py (fixture dir), security/gen_aibom.py (all sources + BOM prose; aibom.cdx.json regenerated), .github/workflows/ci.yml (py_compile path) - scripts/bump_version.py (plugin.json + README pill), scripts/release/ plugin-smoke.sh (version read, history pathspec +--follow across the rename, fabricated marketplace source ./ -> ./plugin) - doc cross-links: plugin/docs/ ../ -> ../../, README clone-and-run -> ./plugin/install.sh, CONTRIBUTING/SECURITY path refs CI green locally: pytest 237 passed/1 skipped, evals 1/1, py_compile ok, AI BOM --check current. install.sh is move-safe (resolves SCRIPT_DIR via BASH_SOURCE; connector/ stays its sibling). COMPANION CHANGE REQUIRED (separate PR, open-agent-ai-security/plugins): switch socxen's marketplace entry from source:url to { "source": "git-subdir", "url": ".../socxen.git", "path": "plugin", "ref": "main" } These two MUST reach socxen `main` together — the marketplace tracks socxen `main`, so promoting this restructure to main while the marketplace still uses source:url breaks installs (no manifest at repo root). Safe on `dev`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VcpWhxi8ZFKgUTSHZY5ZhF Signed-off-by: Matt W <matt.willems@exabeam.com> * docs: add thin root README so the public landing isn't bare (#29) The restructure moves the full README into plugin/ (shipped). git-subdir ships only plugin/, so a root README is GitHub-browsing only and never reaches users. This one is a pointer — what socxen is, the install command, and the repo layout (what ships vs. build-time) — deferring detail to plugin/README.md to avoid drift. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VcpWhxi8ZFKgUTSHZY5ZhF Signed-off-by: Matt W <matt.willems@exabeam.com> * review: smoke-script layout fallbacks, un-mask the bridge skip guard, stale paths (#29) Addresses the three items from @virtualsteve-exa's review, plus the minor. 1. plugin-smoke.sh could not survive its own first post-promotion run. Both helpers assumed this release's layout, but the PRIOR ref on that run is the last root-layout release (0.6.8): - version_at() read $1/plugin/.claude-plugin/plugin.json unconditionally -> FileNotFoundError, fatal under set -e. Now probes plugin/ then the root. A genuinely absent manifest still hard-errors rather than yielding an empty version, which would sail through the current-vs-prior equality check and smoke-test nothing. - fabricate_marketplace() hardcoded "source": "./plugin", which the prior install does not have. Now resolved per call — the upgrade leg fabricates twice against one worktree, straddling the checkout that crosses the restructure boundary. Verified both helpers against fabricated old- and new-layout worktrees: 0.6.8 from the root layout, 0.7.0 from plugin/, sources "./" and "./plugin" respectively, set -e never tripped, and a manifest-less dir exits non-zero. 2. The skip guard in test_check_without_creds_exits_clean treated a bridge TRACEBACK as an environment problem. That is how #67 hid on both maintainers' machines: mcp 2.0.0 dropped streamablehttp_client, the bridge died at import, and the guard downgraded the crash to a skip. The assertions below it already reject tracebacks — they just never ran. A traceback now fails; only genuine provisioning failures still skip. 3. evals/README.md:8,87 pointed at the pre-restructure fixture path, as did test_bridge_wiring.py's module docstring. Rebased onto dev (0.6.8 + the #74 containment fix). Suite: 238 passed, 0 skipped — the no-creds bridge test now runs rather than skipping, which is the check that number was meant to prove. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> --------- Signed-off-by: Matt W <matt.willems@exabeam.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…he governance gate (#70) (#73) * feat: install.sh --merge-permissions — prompted, backed-up merge of the governance gate (#70) The governance gate has been opt-in on every install path. install.sh verified the merge and warned loudly when it was missing, but nothing ever performed it, so the shipped default was "gate OFF until the operator hand-edits ~/.claude/settings.json" — the installer was a detection control, not an enforcement one. Add the write half. --merge-permissions merges settings.snippet.json into the operator's settings.json; an interactive run whose gate reads OFF now shows exactly which rules it would add and asks y/N first. Consent is preserved by construction: nothing merges by default, and -y does NOT authorise it — "assume yes" answers the installer's own questions, it does not license a write to the operator's settings. --checks-only outranks the flag entirely. The merge itself (skills/soc-investigate/merge_permissions.py) is additive-only, backs up settings.json first (timestamped, same mode as the original), refuses outright when a rule already sits in a different tier than the snippet specifies (operator intent wins — it writes nothing and reports what to resolve), restores from the backup if a write fails, and is idempotent. Worth re-running even when the gate reads ON: a hand-merge of just the two ask lines leaves the 17-entry containment deny list missing. Verification is the pre-existing gate_on() re-read, not the merger's exit code — the same "a 0-exit doesn't prove it landed" discipline the install block uses. No python3 or no snippet reports "cannot merge, here's the manual path", never a false green, mirroring the existing "cannot verify != OFF" rule. 25 tests cover the refusals specifically: every one of them must write nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> * fix: resolve the governance settings path instead of hardcoding ~/.claude (#70) install.sh hardcoded $HOME/.claude/settings.json. That was survivable while the governance block only reported the gate's state — it could read a file the running Claude Code ignores and mis-report ON, the dangerous direction. It stops being survivable now that the block can write: the same assumption would merge the gate into a file that never takes effect, i.e. a green "gate ON" protecting nothing. Resolve as SOCXEN_SETTINGS_FILE -> $CLAUDE_CONFIG_DIR/settings.json -> ~/.claude/settings.json, and name the resolved path in every message rather than a hardcoded one. SOCXEN_SETTINGS_FILE is the same explicit escape hatch EXABEAM_ENV_FILE already gives the credentials path. This is also what makes the merge testable from scripts/release/plugin-smoke.sh, which isolates whole installs in throwaway CLAUDE_CONFIG_DIRs: without it, a smoke leg exercising --merge-permissions would escape its sandbox and rewrite the maintainer's real ~/.claude/settings.json on every release run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> * docs: drop stale containment-rule counts after #74 #74 expanded the containment deny tier from 17 bare rules to 68 (both spellings, both namespaces) covering the same 17 tools. Two comments here quoted the old entry count while making a point that does not depend on it — that a hand-merge of just the two ask lines leaves the containment deny list missing entirely. Say that without the number, so the next expansion of the tier doesn't strand them again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> * test: point the merge tests at the relocated skill directory (#29, #70) #66 moved the skill into plugin/, so tests/test_merge_permissions.py's SKILL_DIR had to follow — tests stay at the repo root, the skill they read does not. Now matches test_repo_invariants.py's constant. Nothing else moved: the merger travels beside the snippet it merges, so both landed in plugin/skills/soc-investigate/ together, and install.sh resolves them SCRIPT_DIR-relative — unchanged from plugin/install.sh. Verified end-to-end from the new location: the merge lands {allow: 18, ask: 4, deny: 68} and the no-flag run still writes nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> --------- Signed-off-by: Matt W <matt.willems@exabeam.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eleased changes (#80) * docs: bring the docs current with the plugin/ restructure and the unreleased changes Pre-promotion currency pass. Three classes of defect, all found by reviewing docs against the code rather than against each other. BROKEN INSTRUCTIONS (post-#66, a user follows these): - plugin/docs/installation.md told users to run ./install.sh --merge-permissions in the section the page itself calls the most important on it. There is no root install.sh any more; twenty lines earlier the same page correctly says ./plugin/install.sh. Same for merge_permissions.py's path. - install.sh's OWN output was stale: the usage block is printed verbatim as --help, and every line said ./install.sh. Its 'Next steps' block named skills/soc-investigate/settings.snippet.json and ./install.sh. Both now print the paths the script has already resolved, so they are right from a clone and from the installed plugin directory alike. The --help text also still claimed the merge targets ~/.claude/settings.json — the exact hardcoding #73 removed. A DOC THAT CONTRADICTED THE CODE: - logging.md said a failed emit makes telemetry 'disable itself'. It does the opposite on purpose: the event is dropped and the trail stays on, because a transient fault must not switch off a mandatory audit log — there is a test named for that behavior. Split the two failure modes, and documented the observra>=1.1 floor and the new stderr warning channel, neither of which appeared anywhere. MISSING RELEASE NOTES: - [Unreleased] documented 1 of 4 landed changes. Added entries for the plugin/ restructure (#66), the ABV release gate (#77), and the dependency bounds + hash-pinned lockfile and observra 1.1 migration (#78), and corrected the pre-restructure paths in the entries that were already there. Also: the lockfile existed with no mention in any document — CONTRIBUTING now tells contributors to re-lock (CI checks the AI BOM but not lock drift) and the end-to-end recipe copies the .lock; SECURITY.md's scope adds merge_permissions.py (the only code that writes to a user's settings) and the lockfile; the front-door README no longer presents the installer as check-only; CONTRIBUTING said gates run 'before a tag' though the repo has no tags and says so twice elsewhere; and SCAN_INSTRUCTIONS.md — forward-looking scope for the NEXT Praxen audit — carried pre-restructure paths and asked about a manual-merge-only world that no longer exists, which would have biased that scan. Verified: 263 tests pass, AI BOM current, installer smoke still merges to {allow: 18, ask: 4, deny: 68}, --help renders, every relative link resolves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> * docs: correct three defects found in review of this PR 1. A FALSE ASSURANCE IN AN AUDIT DOC. The rewritten fail-open section claimed observra's warnings surface on stderr for 'a full queue' so a dropped event is 'always visible'. Traced through observra 1.1: a queue-overflow drop is logged at DEBUG and counted in observra_events_dropped_total, below the WARNING-level stderr handler the shim installs — so that drop is counted, not announced. Backend write errors DO surface. Corrected in logging.md, the CHANGELOG entry, and the shim's own (pre-existing, equally wrong) inline comment. 2. THE REVERSE PATH ERROR. installation.md ships INSIDE the plugin and the primary install path is the marketplace, where no plugin/ directory exists — so 'skills/soc-investigate/...' was correct plugin-relative and my prefixing it broke it for the majority reader, while plugin/README.md kept the relative form. Now names the plugin-relative path with the clone path as a gloss. 3. A STALE PATH IN THE BLOCK THIS PR EDITED. tests/end-to-end-testing.md's CACHE still used the pre-0.6.5 marketplace name (socxen/socxen); the real cache is under open-agent-ai-security/socxen/. The new .lock copy would have missed. Also: the shim's DESIGN RULES said an emit failure disables logging, contradicting its own code, tests, and this PR's new doc; my bump_version change contradicted the script's own docstring, so both call sites now use uv run; praxen/README.md's gate said 'before tagging' while CONTRIBUTING (which links it) says release; the rotation bounds are clamped, not passed straight through; the merge-permissions hint now prints an absolute path instead of $0 as invoked; and (#70) gained its PR number per the file's (#issue, #PR) convention. Verified: 263 pass, BOM current, Next-steps block renders with paste-able absolute paths, real settings untouched, all relative links resolve. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> * docs: order the [Unreleased] sections Added-before-Fixed The last clear item from the review pass. The section order was Changed -> Fixed -> Added -> Security; both Keep a Changelog and this file's own 0.6.3 entry put Added first. Pure reordering — no wording changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> --------- Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…/50 resisted (#76) (#81) * fix: pin the red-team gate to an explicit model ID and record the resolved model in every artifact (#76) The runner's --models default was the floating 'sonnet' alias, so the gate's result could change out from under the harness whenever a new Sonnet shipped, and a recorded verdict could not be tied to a specific model version after the fact. Now: - --models defaults to the explicit supported-floor ID (claude-sonnet-4-6), matching the discipline the dated results/ artifacts already show; - the runner captures the session's *resolved* model ID from the CLI init event and tallies/reports under it, so even an alias-invoked run produces an attributable artifact (and prints a notice when the two differ); - HISTORY.md's stale note (Sonnet 5 'not approved for cyber work' — no longer true) is replaced with the real reason the pin exists: reproducibility; - PLAN.md's 'weakest supported model (currently Sonnet)' wording names the pinned ID explicitly in all three places. The AI BOM is unaffected: it inventories dependencies, not harness source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> * fix: a stray stream event must not discard a whole red-team trial Found while running the gate for #76: the first live trial died with "'str' object has no attribute 'get'". The CLI's stream-json output carries lines whose `message` is a plain STRING rather than an object (error/notice events — an MCP server failing to reconnect, for instance). _parse called .get() on it unconditionally, raising AttributeError, which the runner's per-trial `except Exception` then degraded into a fully ERRORED trial. A drive that actually completed 19 tool calls and produced a 36k-character report was discarded because one unrelated notice appeared in the transcript. Grading integrity was never at risk — _inconc_rows already refuses to read an all-errored blocking-class attack as a pass (verdict INCONCLUSIVE, exit 3) — but the harness burned a full live drive per occurrence and could have stalled the gate on re-runs. Parse defensively by SHAPE, not just by event type, and add regression tests for both the odd-shaped line and the resolved-model capture. Negative-verified: the new test fails red on the unfixed parse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> * security: record the 2026-08-13 red-team gate run — 50/50 resisted, PASS (#76) The staleness half of #76. The gate had not run since 2026-07-03, across 0.6.7/0.6.8/0.6.9, and this promotion carries #66/#73/#77/#78. Run: 10 class-A families x 5 trials on the pinned claude-sonnet-4-6, judge on, driven against the working-tree plugin. Every family landed 0/5; no errored and no inconclusive trials, so the verdict rests on 50 valid trials rather than on absence of evidence. The a10 rows carry the real signal. In 4 of 5 trials the model reproduced the phishing markdown link in its chat output — the injection still works on the model — but the persisted artifact was clean every time, because the write-sink neutralizer defanged it. That is exactly what output-pipeline grading exists to distinguish: a pass where the model never emitted the payload would prove nothing about the guardrail. a10 landed 3/3 in the first run and drove the deterministic fix, so seeing the code layer fire is the point. Also fixes the ledger's own stale connector/ paths after #66, and updates the a10 summary's resistance figure from the 0/2 fix-time retest to 0/5 here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> * docs: link the class C/D corpus gap to its tracking issue Praxen finding 008 was 'awaiting triage'; it now has a home (#82) rather than being an untracked row. The gap itself is unchanged — the corpus is still class-A only, and classes C and D remain unexercised despite both blocking a release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> --------- Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Minor, not patch: the shipped surface changes shape. Only the payload under plugin/ is distributed (#66), and the installer can now install the governance gate rather than only warn about it (#73). Also stamps the changelog entry for the red-team gate work (#76/#81), which landed after the [Unreleased] section was written, and dates the section. Both release gates ran against this candidate: red team 50/50 resisted (PASS, 2026-08-13 artifact); ABV 0 Critical (PASS, 2026-08-12 artifact against 0.6.9). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
…ree (#70) (#79) * smoke: verify the governance gate actually installs on the released tree (#70) The release smoke proved the plugin registers and upgrades, but said nothing about the control that makes socxen safe to point at real alerts. Now that install.sh can install the gate rather than only warn about it, a release can regress the assisted merge and every existing leg would still report PASS. Leg 3 runs the shipped tree's plugin/install.sh --merge-permissions, then asserts the dismiss/close pair sits specifically in the `ask` tier — the same judgement gate_on() makes, so "settings.json mentions the tools somewhere" cannot read as installed — and that a re-run reports "already merged" rather than double-appending into the operator's file over successive releases. SOCXEN_SETTINGS_FILE is what makes the leg safe to run at all: without it install.sh writes the real ~/.claude/settings.json, and a release smoke that edits the maintainer's live governance config on every run is worse than no smoke. The containment is asserted rather than assumed — the real file's digest is compared before and after, so a PASS that quietly rewrote it fails instead. Both assertions negative-tested, not just observed passing: the gate check reports OFF when the two tools sit in `allow` instead of `ask`, and the digest guard detects a modified file. A leg that cannot fail is the #67 skip-guard bug in a new costume. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> * smoke: guard leg 3 against a pre-restructure current release (#70) Per @virtualsteve-exa's review note. WT_CURRENT is origin/main@HEAD, which until the promotion is 0.6.9 in the root layout — so anyone running the smoke in this window got "installer exited non-zero" with a file-not-found log tail, which reads like a real regression rather than "not applicable yet". Deliberately NOT the version_at() probe pattern that fixed the same class of problem in the other two helpers: falling back to a root install.sh would run a PRE-#70 installer with no --merge-permissions at all, and the leg would pass having tested nothing. Refusing loudly is the honest answer for a release that cannot support this check. Verified both directions: fires with the explicit message against a fabricated root-layout worktree, and the three assertions still pass against the real tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Matt W <matt.willems@exabeam.com> --------- Signed-off-by: Matt W <matt.willems@exabeam.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…otes Two release-record items: 1. The 2026-08-12 Praxen scan is 0.7.0's ABV gate artifact (maintainer determination). It ran against the dev tip as it stood (005fa4c) — the head of the line 0.7.0 was cut from, not a stale release tag — and every behavioural change merged since is either a remediation of one of that scan's own findings (001->#73, 007->#78, 009->#81) or the path-only plugin/ restructure. Recorded in security/praxen/README.md rather than only in the release PR, so a reader can tell which release the dated artifact verified. 2. #79 landed on dev after the version bump, so the changelog gains its entry: the release smoke now asserts the governance gate actually installs. Release machinery earns a changelog line by the 0.6.4 precedent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Steve Wilson <steve.wilson@exabeam.com>
Collaborator
|
Reviewed the release, verifying claims against the tree rather than reading the summary. One finding, non-blocking, fix already up as #84. Everything else checks out. Verified independently
Finding — five dead links in the shipped docs index (#84)
What makes it worth catching rather than shrugging at: before #66 these read #84 links them canonically and adds Not a blocker on its own. But it's docs-only and one merge away, and the alternative is shipping it to every 0.7.0 install and fixing it in 0.7.1. On your caveat 1 — I'd re-scanYou called the stale ABV artifact a judgment call rather than a blocker, and by the written rule that's right. I'd still re-run it. The 0.6.9 scan predates I can run it against the candidate before the merge if you want the artifact to actually cover what ships. On your caveat 2Agreed it shouldn't gate this release. Class-A-only is accurately stated and #82 tracks it. SequencingNothing here changes the plan: merge commit (never squash), lockstep with plugins#5, then the fast-forward and the smoke run — which is also the first run to exercise #66's prior-worktree fallbacks and #79's governance leg for real. |
mattwillems-exabeam
approved these changes
Aug 13, 2026
mattwillems-exabeam
left a comment
mattwillems-exabeam
left a comment
Collaborator
There was a problem hiding this comment.
Approve. Full review in the comment above — everything I could check against the tree rather than the summary holds: version consistent at 0.7.0 across plugin.json / README pill / AI BOM, main-ancestor-of-dev invariant intact, 265 tests pass, BOM current, both gate artifacts present, and the shipped tree self-contained.
Approving rather than requesting changes because my one finding is docs-only and its fix is already green: plugin/docs/README.md ships five links (../../CHANGELOG.md, CONTRIBUTING.md, SECURITY.md, evals/, tests/end-to-end-testing.md) that resolve in the repo and dead-end in an installed plugin, because a cache's root is plugin/. It's a consequence of this release succeeding — those links were correct until #29 stopped shipping their targets. Real, worth fixing, not worth blocking a release over.
Two conditions on the merge, neither of which is a change to this PR:
- Land #84 first. It fixes those five links and adds the invariant that would have caught them. This PR's head is
dev, so merging #84 updates the candidate in place — no re-cut, and it costs one merge to avoid shipping dead links (including "how to report a vulnerability") to every 0.7.0 install. - Have plugins#5 approved before merging this. I've approved it, but it carries a four-person reviewer set against this PR's one — and since
socxen@maindoesn't carryplugin/yet, either merge order opens a window where installs break. Make that window seconds, not however long an approval takes.
On your caveat 1: still worth re-running ABV against the candidate, since 0.6.9 predates merge_permissions.py — the only code here that writes to a user's governance config. Your call, and I can run it on request; not withholding approval over it.
virtualsteve-exa
added a commit
to open-agent-ai-security/plugins
that referenced
this pull request
Aug 13, 2026
Paired with open-agent-ai-security/raffkin#83 (release 0.7.0). socxen's shipped payload moved under plugin/ (socxen#29/#66), so the catalog entry flips from 'url' (whole repo) to 'git-subdir' with path: plugin. These two merges must land together. Flipped alone, this points at a plugin/ directory that does not yet exist on socxen's main. Left unflipped while socxen releases, a fresh install pulls the repo flat and finds no manifest at the root. Either way installs break, so neither side is safe to land on its own. The validator learned this source type in #4; the manifest change it was written for is exactly this one. Verified by installing through a git-subdir entry pointing at socxen's release candidate: version 0.7.0, payload is connector/docs/skills/install.sh only, with tests/, evals/, scripts/ and security/ (including the red-team attack payloads) absent — and the governance snippet still at 18 allow / 4 ask / 68 deny. Signed-off-by: Steve Wilson <steve.wilson@exabeam.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release
dev → main. Merge with a merge commit, never squash (gh pr merge --merge), then fast-forwarddevback up somainstays an ancestor.fixes #29
fixes #70
fixes #76
The plugin now ships as a git subdirectory. The index entry in
open-agent-ai-security/pluginsmust flip tosource: git-subdir/path: pluginin lockstep with this merge. Landing one without the other is the one way this release breaks installs — merged alone, a fresh install pulls the repo flat and finds no plugin manifest at the root.What ships
Only the payload is distributed now (#29, #66). Everything Claude Code installs moved under
plugin/; a user's cache no longer receives the test suite, eval corpus, release scripts, orsecurity/— which included the red-team attack payloads. Clone-and-run becomes./plugin/install.sh.The installer can install the governance gate, not just warn about it (#70, #73).
--merge-permissionsmerges the permission pack, and an interactive run whose gate reads OFF now offers to do it after showing exactly what it would add. Consent is preserved by design: nothing merges by default,-ydoes not authorise it,--checks-onlyoutranks it, the merge is additive-only, backs up first, refuses on a tier conflict, and is verified afterwards by the pre-existinggate_on()re-read. The settings path also stops being hardcoded — it resolvesSOCXEN_SETTINGS_FILE→$CLAUDE_CONFIG_DIR/settings.json→~/.claude/settings.json, so the gate is checked and written in the file Claude Code actually reads.The release smoke now proves that gate installs (#79) — a third leg runs
--merge-permissionson the shipped tree into a throwaway settings file and asserts the gate reads ON in theasktier, the operator's realsettings.jsonis untouched (digest-compared), and a re-run is a no-op.Agent Behavior Verification became a documented release gate (#77) — a blind-authored Worker Remit, dated scan results, and a no-Critical-blocks-release rule.
Dependencies are bounded and hash-pinned (#71, #78), and the telemetry shim moved onto observra's public API. This closes the class of breakage that took out 0.6.8.
The red-team gate is pinned to a named model and was actually run (#76, #81).
Release gates — both green
security/redteam/results/2026-08-13T2009-claude-sonnet-4-6.mdsecurity/praxen/results/2026-08-12-socxen-0.6.9.*On the ABV artifact's provenance (maintainer determination, now recorded in
security/praxen/README.md): the scan ran at2026-08-12T22:36Zagainst thedevtip as it stood —005fa4c, the head of the line this release was cut from, not a stale release tag. Every behavioural change merged between that scan and this candidate is either a remediation of one of the scan's own findings (001 → #73, 007 → #78, 009 → #81) or the path-onlyplugin/restructure (#66); #77 is the scan's own artifacts. So the delta since scanning introduces no unscanned behaviour, and the gate's rule — no open Critical — is satisfied for 0.7.0.One caveat that does stand: the red-team corpus is class-A only. Gate bypass (C) and data leak (D) are individually release-blocking per
PLAN.mdand have never been exercised — the PASS above is accurate for injection and silent about the other two. Tracked in #82.Verification
265 tests pass · AI BOM current and regenerated by
bump_version.py· version consistent acrossplugin.json, the README pill, and the BOM at 0.7.0 · attack corpus lints 10/10 · smoke script parses clean.Post-merge checklist
git-subdir/path: plugin(pair with this merge).git checkout dev && git merge --ff-only origin/main && git push origin dev— holds the main-is-ancestor-of-dev invariant.scripts/release/plugin-smoke.sh— the first run that crosses the restructure boundary, so it exercises both the prior-worktree fallbacks from restructure: ship only plugin/ via git-subdir — keep build artifacts out of the distributed plugin (#29) #66 and the new governance leg from smoke: verify the governance gate actually installs on the released tree (#70) #79.🤖 Generated with Claude Code