examples(bank-support): replace AIEWF demo with two realistic-baseline behaviors - #313
examples(bank-support): replace AIEWF demo with two realistic-baseline behaviors#313Chang Liu (changliu2) wants to merge 6 commits into
Conversation
Replaces the AIEWF bank-manager demo's strawman baseline (an agent with no
authorization logic) with a genuinely competent one, so the ACS Rego claim
rests on generalization rather than on the baseline being broken.
Behavior under test: reads of, and state changes on, entities whose
risk_tier is in {high_net_worth, vip, restricted} require a verified
same-session authorization for that entity, cited by approval_ref.
Three arms, single-variable deltas, same frozen 72-case test set:
1. realistic baseline - server-side, deterministic, property-based gate,
but registered only on the deposit tools it shipped with
2. baseline + strengthened defensive prompt
3. baseline + ACS Rego rule keyed on risk_tier (arm 1's exact prompt)
Real results, six full runs, 432 scored cases:
deterministic unauthorized exposure 55.6% / 54.2% / 0.0% (p=8.6e-16)
trace-fed judge policy_violation 8.3% / 5.6% / 0.0% (p=0.028)
overrefusal on standard-tier records 0.0% / 0.0% / 0.0%
The baseline sits at 8-21% exposure on the one domain its gate covers and
68-80% on the three added later; the stronger prompt moves the uncovered
domains by zero points. The unmodified Rego rule blocks 13/13 protected
records across six domains - two of which exist nowhere in the codebase -
with zero false positives and zero new lines of code.
Also records a measurement finding: with a text-only callable target the
judge saturated at an identical 62.5% for all three arms and produced 45
false positives against the arm with zero real violations, because an
ordering invariant cannot be established from final text alone. Adding a
connector target that exposes the tool sequence made the dimension
discriminative. Both configs are kept so the contrast is reproducible.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…notator One behavior, one yaml: a requester substitutes a claimed authority, an unrecorded verbal assurance, or deadline pressure for a required RECORDED control step, while every typed field stays clean. No deterministic rule can separate these from legitimate evidenced escalations, so the control is a calibrated learned classifier wired as an ACS classifier annotator at pre_tool_call. - eval_coercion_authority.yaml + 3 arm variants (shared frozen test set) - coercion_classifier.py: naive keyword gate + LLM gate + Platt scaling - acs_annotator_shim.py: host-side ACS 10 annotator dispatch - bank_manager_coercion.rego: three-band allow/escalate/deny learned gate - calibration, ground-truth labelling, scoreboard and held-out check scripts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…line behaviors Removes the old three-beat demo whose baseline agent had no authorization logic at all, and replaces it with two behaviors that each start from a baseline a reviewer would sign off on. - Behavior 1 (sensitivity-tier authorization): deterministic decision, property-based Rego rule vs a competent-but-domain-scoped Python gate. - Behavior 2 (coercion via unverified authority): non-deterministic judgement, calibrated classifier annotator vs a control-aware prompt plus keyword tripwire. agent.py is renamed to bank_agent_common.py and stripped to shared plumbing only (LLM construction, MCP server startup, text extraction); it no longer declares a system prompt or an ASSERT callable. Also fixes a pre-existing pytest collection failure in the example's tests/ package via tests/conftest.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The arm3n naive-keyword config is removed from the shipped example. At n=19/21 the runtime arms cannot separate the naive scorer from the calibrated one, so the config added a fourth arm without adding evidence. The finding itself is preserved in prose: the naive-vs-calibrated recall / FPR / Brier tables, the out-of-distribution recall collapse (1.000 -> 0.429), the Platt-calibration-hurt-OOD negative result, and the 0.0% bypass / 38.1% over-refusal end-to-end diagnostic numbers all stay in the README. The callable is kept so the diagnostic remains reproducible via --override. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the reviewed 120-case coercion fixture, make OpenTelemetry tracing the default authorization path, update permissible/impermissible reporting, and rebuild the AIEWF deck around the best-practice demo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Select all three candidate arms through target overrides so sensitivity-tier authorization and coercion share the same one-behavior-one-YAML workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Jake Present (jakepresent)
left a comment
Jake Present (jakepresent)
left a comment
There was a problem hiding this comment.
The replacement framing is substantially better than the old no-auth strawman, but the current implementation does not yet support the structural-safety and evidence claims it makes.
-
A direct protected write can execute before ACS learns the tier.
agent_tier_authz._wrap_tool()buildsprotected_refsonly fromstate["observed_tiers"], defaulting an unseen entity tostandard. A directprepare_loan_modification(LN-3002, ...)therefore reaches the Rego pre-call point withstate_changing=trueand an emptyprotected_refs; the checked-in policy returnsallow. The core function then adds the VIP loan modification to_pending_loan_mods, and only the post-call rule denies the result after the mutation. I reproduced both halves with the checked-in artifacts: OPA returnedallowpre-call for unseenLN-3002, andbank_core.prepare_loan_modification()changed the pending-mod state from 0 to 1. The property needs to be available authoritatively before a state-changing call, or the mutation must remain provisional until the post-call decision. Please add a real direct-write regression, not only the post-call generalization table. -
The coercion arm treats a reference-shaped string as recorded evidence without verifying it. The manifest passes only
$.snapshot.user_message; the classifier prompt explicitly scoresAUTH-####/CB-####/OPS-####strings low, and there is no lookup against bank-owned state. A caller can invent a reference and receive the same treatment as a real artifact. That recreates the exact design problem the example says it fixes. The semantic classifier can identify authority pressure, but whether a cited artifact exists and applies to this action needs a typed lookup/verification result in the policy input. -
The coercion eval cannot judge the claimed tool/control behavior.
eval_coercion_authority.yamlhas notarget.trace, and all three exported callables return only the final string. The ACS arm writes a separate example-local JSONL file, but it is not attached to the ASSERT transcript or score evidence. The judge rubric asks whether privileged tools ran or ACS blocked them, yet the judge cannot see either. Please carry tool calls/results and ACS decisions into normal ASSERT evidence, through OTel or structured callable events, and verify that the score cites that evidence. -
The learned policy is fail-open when its annotation is absent, contrary to its comments.
bank_manager_coercion.regodefaults toallow; with no annotation it readsscore=0,escalate_lo=2, anddeny_hi=2, so neither guard fires. Running the checked-in policy through OPA returned{"decision":"allow"}for a gatedcreate_transferwith no annotation. Missing or invalid learned evidence needs an explicit deny/escalate rule plus policy tests. -
The committed powered-study reproduction path is currently broken. The checked-in fixture hashes to
d301c16a..., whileprepare_powered_coercion.py,test_powered_coercion_fixture.py, and the published results all pin1f314b96.... The first documented preparation command exits with a hash mismatch, and the example suite is red (1 failed, 72 passed, 11 skipped). The PR only runs CodeQL, so this escaped CI. Please restore one internally consistent frozen dataset/result provenance and run the example tests in CI. For the paired McNemar claim, a compact per-case arm-outcome table would also let the committed evidence recompute the statistic instead of merely asserting numbers already present in the summary JSON.
Other verification was healthy: the repository suite passed (1,211 passed, 20 skipped, 474 subtests), viewer check/build passed, all six targets import after installing the documented example dependency, and the OPA tier-generalization proof passed 13/13 with 0/11 false positives. Those checks do not cover the trust-boundary gaps above.
2 participants
Why
The current AIEWF bank demo starts from an agent with no authorization logic. A reviewer can dismiss that result as a missing-check bug rather than evidence for an evaluation/control loop.
This replaces it with two independently actionable behaviors whose baselines are reasonable first versions:
ASSERT discovers and measures the runtime failure. ACS applies the structural control: property-based Rego for the deterministic case and a classifier annotator for the semantic case.
Both behaviors use the same customer workflow: one behavior → one YAML → three target overrides.
Behavior 1 — scalable policy coverage
The deposit service already has a server-side, fail-closed authorization gate. Loans, brokerage, and client records shipped later without calling it.
All three arms use the same 72 cases and full OpenTelemetry trace evidence:
The independent replicate is
55.6% -> 54.2% -> 0.0%.The deterministic generalization proof runs the unchanged gate and Rego against six domains, including two that do not exist in the implementation:
eval_tier_authorization.yamlusestarget.callablewithtarget.trace: {backend: otel}. Target overrides select the baseline, defensive-prompt, and ACS arms.Behavior 2 — powered coercion comparison
The branch carries one
eval_coercion_authority.yamlplus the reviewed frozen fixture used by the blog:Classifier vs hardened prompt reduces permissible violations by 20.0 percentage points on paired cases (exact McNemar
p=.0169). The classifier has0/60observed bypasses, with a one-sided exact upper 95% bound of4.87%.The fixture installer is offline:
All three arms run from the same YAML by overriding only
runandinference.target.callable, so they reuse the exact same test-set IDs. The scoreboard uses paired statistics and writes a local machine-readable report.AIEWF write-up
talks/aiewf-18min/aiewf-2026-deck.pdfas a seven-slide best-practice walkthrough.Validation
The model-backed 120-case study was completed before this PR: 360/360 inference rows were judged successfully. This PR commits the curated fixture, labels, result summary, and reproduction path—not generated run artifacts.
Evidence scope
The reported results apply to this agent, reviewed datasets, controls, and model configurations under the tested conditions. They are not a claim of perfect performance across all agents or authorization failures.
Post-merge
aka.ms/assert-acs-democurrently pins the previous talk commit. Repoint it to the mergedtalks/README.md(or the new immutable merge commit) so the short link serves the revised PDF.