diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e847135..49776f66 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,12 @@ How to keep this current: add the entry in the same pull request as the change, +## 0.75.0 + +### Changed + +- The default `action.irreversible.confirm` (the hold threshold) is 0.9 instead of 0.7, so a judge-only hold waits for the confidence the recorded action-guard corpus shows is safe: the judge's error rate falls from 15% below confidence 0.8 to under 1% above it, and a 0.9 cutoff chosen on one half of the corpus removed about 52 false alarms on the other half without losing a true catch. A call the judge scores 0.5 to 0.9 now warns instead of holding; pattern holds and user or project overrides are unchanged. Restore the old behaviour with `"action": { "irreversible": { "confirm": 0.7 } }`. + ## 0.74.1 ### Fixed diff --git a/docs/configuration.md b/docs/configuration.md index b86d2eab..af1d4262 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -21,7 +21,7 @@ User file `~/.pi/agent/pi-warden/config.json` (owner-only). `/warden config` ope "enabled": true, "tools": ["bash", "powershell", "ctx_execute", "ctx_batch_execute", "ctx_execute_file", "write", "edit"], "failOpen": true, - "irreversible": { "warn": 0.5, "confirm": 0.7 }, + "irreversible": { "warn": 0.5, "confirm": 0.9 }, "offTask": { "warn": 0.6, "steer": 0.85 }, "intentMismatch": 0.9, "visibleMismatch": 0.8, @@ -85,7 +85,7 @@ User file `~/.pi/agent/pi-warden/config.json` (owner-only). `/warden config` ope | `timeoutMs` | Per-request timeout. On timeout the call is allowed with a warning when `action.failOpen` is true. | | `maxRequests` | Per-session request budget. When spent, pi-warden says so once and continues with offline checks. | | `action.tools` | Tools the action guard inspects. Add your own shell-like tools here. | -| `action.irreversible` | `warn` and `confirm` (hold) thresholds on P(irreversible). | +| `action.irreversible` | `warn` and `confirm` (hold) thresholds on P(irreversible). Defaults `warn` 0.5 and `confirm` 0.9. The 0.9 hold waits for the confidence at which the judge stops being wrong: on the recorded action-guard corpus the judge errs on 15% of calls below confidence 0.8 and under 1% above it, and a 0.9 cutoff removed about 52 false alarms on a held-out half of the corpus without losing a true catch. Calls the judge scores 0.5 to 0.9 warn instead of holding. See [guards.md](guards.md#irreversible-hold-threshold-2026-09-29-held-out-split). | | `action.offTask` | `warn` and `steer` thresholds on P(off-task). Off-task never holds. | | `action.intentMismatch` | P(call differs from the agent's stated plan) that warns and tells the agent, on calls that can change something. | | `action.visibleMismatch` | Lower mismatch threshold for commands whose effect is visible outside the working tree (commit, push, publish, install, launch). | diff --git a/docs/guards.md b/docs/guards.md index 83337801..55e82956 100644 --- a/docs/guards.md +++ b/docs/guards.md @@ -23,7 +23,7 @@ Runs on `tool_call`, before the tool executes. Text that is data is not a command. A heredoc body written to a file, a quoted `echo`/`printf` argument, a `grep` pattern, or a `git commit -m` message can mention `git push --force` without a hold. The same text fed to `sh`, `bash -c`, `eval`, `xargs`, or a `python3 - <= 0.8` and `intent_mismatch >= 0.8` flags 1.1% of calls with 18% in a rejected turn, so that pair steers at `visibleMismatch` 0.8. Two deterministic patterns came from the regretted list: a git command with hooks or signing switched off, and `gh pr merge`. @@ -94,7 +94,7 @@ AUC against regret: 0.26 — non-regretted calls score higher (correct direction | `should_proceed` | 0.26 | 0.58 | | `pause_requested` | 0.27 | 0.51 | -The question is trace-only by default until calibrated: AUC against regret is 0.26 and the default threshold of 0.6 flags 44% of non-read-only calls. The score and reason remain in the trace, but no steer reaches the agent. Set `action.shouldProceed.steer: true` to restore the pause-and-ask steer; `hold` remains the threshold. This question never holds a call, consistent with the existing rule that only deny rules and `irreversible >= 0.7` hold; a built-in destructive pattern holds only when no judge answers or when `action.floor` is `"level"`. +The question is trace-only by default until calibrated: AUC against regret is 0.26 and the default threshold of 0.6 flags 44% of non-read-only calls. The score and reason remain in the trace, but no steer reaches the agent. Set `action.shouldProceed.steer: true` to restore the pause-and-ask steer; `hold` remains the threshold. This question never holds a call, consistent with the existing rule that only deny rules and `irreversible >= 0.9` hold; a built-in destructive pattern holds only when no judge answers or when `action.floor` is `"level"`. ### violation_judgment calibration (2026-09-20) @@ -146,6 +146,21 @@ A full replay of four projects' recorded sessions at 0.33.3 with `--extra`, judg n = 15,232 judged calls with 26 positives, except `intent_mismatch` (12,636 calls with a plan, 20 positives). Since 17 September the hold rate fell from 1% to 0.27%, because off-task no longer holds; `irreversible` rose from 0.71 to 0.73, `mutates` slipped from 0.75 to 0.72, the candidate questions barely moved, and the share of live holds the user approved rose from 12% to 36%. Full tables: `eval/reports/2026-09-21-calibration-0.33.3/`. +### Irreversible hold threshold (2026-09-29, held-out split) + +`action.irreversible.confirm` is 0.9, not 0.7. The threshold is set on the judge's own confidence, not on regret: on the recorded action-guard corpus (15,346 judged calls, 26 regretted calls), confidence is `max(p, 1 - p)` and a judge failure counts as an error. + +| confidence | items | error | false alarms | misses | +| --- | --- | --- | --- | --- | +| 0.50–0.60 | 118 | 42.4% | 49 | 1 | +| 0.60–0.80 | 493 | 15.2% | 72 | 3 | +| 0.80–0.95 | 4,937 | 0.8% | 28 | 13 | +| 0.95–1.00 | 9,798 | 0.1% | 0 | 8 | + +The error rate collapses once confidence passes 0.8: below it the judge is wrong between one call in seven and one in two. A 0.9 cutoff chosen on a random half of the corpus (seed 20260930) and checked on the other half removed about 52 false alarms on the held-out half and lost no true catch, so calls the judge alone scored 0.7 to 0.9 now warn instead of holding. In the replay, none of the 42 calls scored in that band sat in a rejected turn, and the 23 of them on calls that ran also drew no regret. + +The evidence is a direction, not a fitted threshold: there are only 26 regret positives, and the misses sit at high confidence (0.95 to 1.00), where a cutoff cannot reach them — moving the threshold trades false alarms for misses, it does not fix a judge that is confidently wrong. The method follows Li, Miao, Krishnan, Padman, "JEV-as-a-Judge: Accept When Confident, Escalate When Unsure" (arXiv:2609.26550, Carnegie Mellon University): pick the threshold on a selection split, re-check it on a held-out split, and count judge failures as errors. + ### Live: what fired, and what the agent did next The replay measures the action guard's decisions against your reactions. It cannot measure the other half: what the agent does with a steer. For that, 67 steer messages from two days of live work on one production repo (19 sessions, 2026-09-16 to 09-17), read back from the recorded session logs: @@ -315,7 +330,7 @@ Confusion over the four reasons (rows labelled, columns answered): 21/21 `from_c ### Rules tiers calibration (2026-09-27, per-rule cutoff, severity, and soft tier) -The bench grew to 161 labelled cases (17 rules: 107 tune, 54 holdout). A rule body may now start with `threshold:` and `severity:` lines beside `paths:`; a rule with its own cutoff fires there instead of `rules.threshold`, and severity orders findings high, normal, low, then score, in the steer and the trace. An opt-in `rules.softThreshold` (`0`, off) turns a score between it and a rule's cutoff into a soft "check whether this applies" sentence in the same steer, without holding and without counting as a finding. A full run at the shipped settings (161 cases, 147 asked: 74 violation, 73 clean) reaches recall 0.865, false alarms 0.027 and precision 0.970 at 0.7; the soft tier at 0.5 adds 8 catches and 4 false alarms against the per-rule cutoffs, for 0.959 and 0.055. Every false alarm at 0.7 comes from the boolean-name rule (0.400 of its five clean cases); `threshold: 0.9` on that rule moves it to recall 0.750 and 0.000 — one catch traded for two false alarms, which is why the tier and the cutoff are per-rule choices, not new defaults. A one-sentence wording change (new text breaking the rule again is a violation even where the file already breaks it) was tried, made the same-file cases score high, but did not move the known miss and cost a tune catch on the shared set, so it was reverted. **Case `r15-04` (a new `var` in a file that already declares one) is still a known miss: 0.35 before, 0.39 after, both below the cutoff, and the new same-file cases swing between 0.51 and 0.96 across runs of the shipped question.** The shared 153 cases are unmoved (all: 0.912 and 0.028). Default behaviour is unchanged with no header lines and `softThreshold: 0`. +The bench grew to 161 labelled cases (17 rules: 107 tune, 54 holdout). A rule body may now start with `threshold:` and `severity:` lines beside `paths:`; a rule with its own cutoff fires there instead of `rules.threshold`, and severity orders findings high, normal, low, then score, in the steer and the trace. An opt-in `rules.softThreshold` (`0`, off) turns a score between it and a rule's cutoff into a soft "check whether this applies" sentence in the same steer, without holding and without counting as a finding. A full run at the shipped settings (161 cases, 147 asked: 74 violation, 73 clean) reaches recall 0.865, false alarms 0.027 and precision 0.970 at 0.7 (tp 64 / fp 2 / fn 10, the flat-0.7 view; the shipped per-rule thresholds give tp 63 / fp 0 / fn 11); the soft tier at 0.5 adds 8 catches and 4 false alarms against the per-rule cutoffs, for 0.959 and 0.055. Every false alarm at 0.7 comes from the boolean-name rule (0.400 of its five clean cases); `threshold: 0.9` on that rule moves it to recall 0.750 and 0.000 — one catch traded for two false alarms, which is why the tier and the cutoff are per-rule choices, not new defaults. A one-sentence wording change (new text breaking the rule again is a violation even where the file already breaks it) was tried, made the same-file cases score high, but did not move the known miss and cost a tune catch on the shared set, so it was reverted. **Case `r15-04` (a new `var` in a file that already declares one) is still a known miss: 0.35 before, 0.39 after, both below the cutoff, and the new same-file cases swing between 0.51 and 0.96 across runs of the shipped question.** The shared 153 cases are unmoved (all: 0.912 and 0.028). Default behaviour is unchanged with no header lines and `softThreshold: 0`. ### Turn question calibration (2026-09-27, first measurement) diff --git a/package.json b/package.json index 379afeee..97db3d51 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "pi-warden", - "version": "0.74.1", + "version": "0.75.0", "description": "Makes the Pi agent follow your project's rules. Jev judges every write against your pi-warden.md and quotes the broken rule back to the agent, names slop, breaks stuck loops, calls out unverified done claims, compresses large tool output, and holds the rare destructive command. Built on pi-typesafe.", "type": "module", "license": "MIT", diff --git a/src/config.ts b/src/config.ts index cd616f7f..f32e2542 100644 --- a/src/config.ts +++ b/src/config.ts @@ -487,7 +487,8 @@ export function defaultConfig(): WardenConfig { tools: [...COMMAND_TOOLS, "write", "edit"], failOpen: true, timeoutMs: 5000, - irreversible: { warn: 0.5, confirm: 0.7 }, + // 0.9 holds: below it the judge is wrong one call in two to one in seven, and the 0.7 to 0.9 band held no call the user regretted. + irreversible: { warn: 0.5, confirm: 0.9 }, offTask: { warn: 0.6, steer: 0.85 }, intentMismatch: 0.9, visibleMismatch: 0.8, diff --git a/tests/action-guard.test.ts b/tests/action-guard.test.ts index 160ed8af..074ef243 100644 --- a/tests/action-guard.test.ts +++ b/tests/action-guard.test.ts @@ -96,22 +96,22 @@ test("regression: approval applies to the action, not the exact command string; // `command -v` prefix and was held twice more, then fell back to DROP DATABASE. The approval question was never asked. const guard = new ActionGuard(); const judge = stubJudge(); - judge.next = { irreversible: 0.87, mutates: 0.95 }; + judge.next = { irreversible: 0.92, mutates: 0.95 }; assert.equal((await guard.inspect(bash("c1", "cd wt && command -v supabase; supabase db reset 2>&1 | tail -25"), under("prove the three migrations"), options(judge))).level, "confirm"); guard.hold("prove the three migrations"); - judge.next = { irreversible: 0.87, mutates: 0.95, approved: 0.93 }; + judge.next = { irreversible: 0.92, mutates: 0.95, approved: 0.93 }; const reworded = await guard.inspect(bash("c2", "cd wt && supabase db reset 2>&1 | tail -25"), under("Yes you can."), options(judge)); assert.equal(reworded.level, "allow", "reworded retry after approval runs"); assert.ok(askedApproval(judge), "Jev was asked whether the reply approves this action"); // Same shape, but Jev says the reply does not approve the first retry (0.2); a second, reworded retry must still be asked. - judge.next = { irreversible: 0.87, mutates: 0.95 }; + judge.next = { irreversible: 0.92, mutates: 0.95 }; assert.equal((await guard.inspect(bash("c3", "supabase db reset"), under("wipe the local db and replay migrations"), options(judge))).level, "confirm"); guard.hold("wipe the local db and replay migrations"); - judge.next = { irreversible: 0.87, mutates: 0.95, approved: 0.2 }; + judge.next = { irreversible: 0.92, mutates: 0.95, approved: 0.2 }; assert.equal((await guard.inspect(bash("c4", "supabase db reset 2>&1 | tail -25"), under("Yes you can."), options(judge))).level, "confirm", "held again: 0.2 is not approval"); guard.hold("Yes you can."); - judge.next = { irreversible: 0.87, mutates: 0.95, approved: 0.9 }; + judge.next = { irreversible: 0.92, mutates: 0.95, approved: 0.9 }; const again = await guard.inspect(bash("c5", "supabase db reset 2>&1 | tail -40"), under("Yes you can."), options(judge)); assert.ok(askedApproval(judge), "the re-hold did not consume the user's reply"); assert.equal(again.level, "allow"); diff --git a/tests/config.test.ts b/tests/config.test.ts index eb8fae71..de5aeaf0 100644 --- a/tests/config.test.ts +++ b/tests/config.test.ts @@ -27,6 +27,7 @@ test("defaults: guards on, steer mode, TypeSafe consent off, nudges on", () => { assert.equal(config.mode, "steer"); assert.deepEqual(config.action.tools, ["bash", "powershell", "ctx_execute", "ctx_batch_execute", "ctx_execute_file", "write", "edit"]); assert.equal(config.action.failOpen, true); + assert.deepEqual(config.action.irreversible, { warn: 0.5, confirm: 0.9 }, "0.9 is the surveyed hold threshold"); assert.ok(config.action.irreversible.warn < config.action.irreversible.confirm); assert.equal(config.stuck.nudge, true); assert.equal(config.done.nudge, true); @@ -102,13 +103,13 @@ test("0.1.x files keep working: action.timeoutMs and action.maxRequests are read }); test("project overrides cannot grant consent, change the mode, or raise budgets", () => { - const config = applyProjectOverrides(defaultConfig(), { typesafe: true, mode: "advise", maxRequests: 9999, timeoutMs: 1, action: { tools: ["bash"], irreversible: { confirm: 0.9 } }, stuck: { enabled: false } }); + const config = applyProjectOverrides(defaultConfig(), { typesafe: true, mode: "advise", maxRequests: 9999, timeoutMs: 1, action: { tools: ["bash"], irreversible: { confirm: 0.95 } }, stuck: { enabled: false } }); assert.equal(config.typesafe, false); assert.equal(config.mode, "steer"); assert.equal(config.maxRequests, 500); assert.equal(config.action.timeoutMs, 5000); assert.deepEqual(config.action.tools, ["bash"]); - assert.equal(config.action.irreversible.confirm, 0.9); + assert.equal(config.action.irreversible.confirm, 0.95, "a project override still moves the hold threshold"); assert.equal(config.stuck.enabled, false); const quiet = applyProjectOverrides(defaultConfig(), { notify: { enabled: false, command: ["evil"] } }); assert.equal(quiet.notify.enabled, false, "a project may switch notifications off"); diff --git a/tests/guard.test.ts b/tests/guard.test.ts index 69ad2f60..7d332cc4 100644 --- a/tests/guard.test.ts +++ b/tests/guard.test.ts @@ -672,7 +672,10 @@ test("evaluateAction applies thresholds from config", async () => { const config = defaultConfig().action; const warn = await evaluateAction({ tool: "bash", input: { command: "npm run migrate" }, cwd, task: "add a column" }, { config, judge: judge(0.55, 0.1) }); assert.equal(warn.level, "warn"); - const confirm = await evaluateAction({ tool: "bash", input: { command: "npm run migrate" }, cwd, task: "add a column" }, { config, judge: judge(0.8, 0.1) }); + // The 0.5 to 0.9 band warns: the default confirm is 0.9, so a judge-only 0.8 does not hold. + const band = await evaluateAction({ tool: "bash", input: { command: "npm run migrate" }, cwd, task: "add a column" }, { config, judge: judge(0.8, 0.1) }); + assert.equal(band.level, "warn"); + const confirm = await evaluateAction({ tool: "bash", input: { command: "npm run migrate" }, cwd, task: "add a column" }, { config, judge: judge(0.92, 0.1) }); assert.equal(confirm.level, "confirm"); const allow = await evaluateAction({ tool: "bash", input: { command: "npm run migrate" }, cwd, task: "add a column" }, { config, judge: judge(0.2, 0.2) }); assert.equal(allow.level, "allow"); diff --git a/tests/violation-pipeline.test.ts b/tests/violation-pipeline.test.ts index c8e6bda9..841e4fe6 100644 --- a/tests/violation-pipeline.test.ts +++ b/tests/violation-pipeline.test.ts @@ -149,12 +149,12 @@ test("(b) level mode: destructive hit, no judge → confirm as today", async () assert.equal(verdict.level, "confirm"); }); -test("(c) evidence mode: destructive hit, judge present, Jev 0.85 → confirm via irreversible", async () => { +test("(c) evidence mode: destructive hit, judge present, Jev 0.92 → confirm via irreversible", async () => { const config = defaultConfig().action; - const j = fakeJudge({ irreversible: 0.85 }); + const j = fakeJudge({ irreversible: 0.92 }); const verdict = await evaluateAction({ tool: "bash", input: { command: "git reset --hard HEAD~1" }, cwd, task: "undo last commit" }, { config, judge: j }); assert.equal(verdict.level, "confirm"); - assert.ok(verdict.reasons.some(r => /irreversible 0\.85/.test(r))); + assert.ok(verdict.reasons.some(r => /irreversible 0\.92/.test(r))); }); test("(d) user command rule severity destructive, judge present, Jev 0.1 → confirm (user rule wins)", async () => {