diff --git a/CHANGELOG.md b/CHANGELOG.md
index b4356b8..7777574 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,16 @@ changes, new features. **Patch:** fixes. Images are published to
 a release is validated. Release process: [runbook](docs/operate/runbook.md#release-a-new-version). Gateway/CLI-only releases
 (`make release-gateway`) reuse the previous serving images.
 
+## v0.1.2 (unreleased)
+
+Gateway and monitoring; serving images unchanged (v0.1.0).
+
+- Monitoring: scheduled health check (`scripts/probe.py`, `scripts/deploy_probe.sh`: Cloud Run jobs + Scheduler),
+  alert policies and log-based metrics (`scripts/setup_alerts.py`), guide `docs/operate/monitoring.md`.
+- Gateway decision logs record the requested backend (`dgem_backend_requested`) next to the one that answered, and
+  failed requests are logged with backend `none` instead of `cloudrun`, so failover can be measured.
+- `scripts/template_sweep.py`: run every policy template through a gateway with its sample variables.
+
 ## v0.1.1 (2026-09-28)
 
 Gateway, MCP and CLI only. **Serving images are unchanged**: the latest serving image is still `v0.1.0`
diff --git a/benchmarks/runs/20260925-serving-speed/deployments.md b/benchmarks/runs/20260925-serving-speed/deployments.md
index 479d67b..d24144d 100644
--- a/benchmarks/runs/20260925-serving-speed/deployments.md
+++ b/benchmarks/runs/20260925-serving-speed/deployments.md
@@ -29,6 +29,10 @@
 
 | 09-28 ~21:00 | Cloud Run `dgemma-gateway` → revision `00057` | **v0.1.1** (gateway/CLI-only release; serving stays v0.1.0): PRs #19, #22, #23 on top of #18; admin API off | `/health` version v0.1.1; all backends route; admin POSTs 403; unknown backend / foreign `vertex_url` 400; MCP tool schemas list `[vertex_first, vertex, cloudrun]` and reject unknown backends at validation; IAP custom domain; `dgem mcp --remote` | Revision `00055` |
 
+| 09-28 22:39–23:05 | Verification + rollback drill ([`20260928-v010-verification`](../20260928-v010-verification/README.md)) | All sample suites on v0.1.0; rollback model `504638d` redeployed on the endpoint and given 100% traffic for ~30 s, then traffic restored and the drill copy undeployed | All pass (two pre-existing harness issues noted) | — |
+| 09-28 23:10 | Cloud Run `dgemma`, `dgemma-gateway`; Vertex model registry | Pruned to n and n-1: 81 revisions deleted (kept gateway `00057`, `00055`; serving `00034`, `00032`); 3 older model resources deleted (kept v0.1.0 and `504638d`) | — | — |
+| 09-28 22:50–23:01 | Monitoring | Health check `dgem-probe-hourly` / `-daily` (Cloud Run jobs + Scheduler, SA `dgem-probe-sa`), 4 log-based metrics, 7 `dgem:` alert policies (email) | First runs: Vertex ok 78 ms, gateway ok 192 ms, Cloud Run ok 82 ms | — |
+
 Follow-ups:
 - ~~Rotate the Hugging Face token~~ done (version 2).
 - ~~Cloud Run health lacks `warmed`~~ not a bug: the earlier check ran before warmup finished; later checks show
diff --git a/benchmarks/runs/20260928-v010-verification/README.md b/benchmarks/runs/20260928-v010-verification/README.md
new file mode 100644
index 0000000..f5054da
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/README.md
@@ -0,0 +1,40 @@
+# Run 20260928-v010-verification
+
+Before pruning old revisions, every sample suite was run against production **v0.1.0** (Vertex G4 endpoint, via
+`--vertex-url`) in one session, and the retained rollback targets (n-1) were exercised. Numbers compare with the
+committed receipts in `benchmarks/` (earlier images and hardware) and with the image-parity run.
+
+## Version n: v0.1.0 on Vertex G4
+
+| Suite | Items | Result | Reference | Verdict |
+| :--- | :---: | :--- | :--- | :--- |
+| `bench-jev` | 231 | 187 correct (81.0%) | Band 182–192 (image parity) | Pass |
+| `bench-calibration` | 50 | 46 (92%) | 43–46 (image parity) | Pass |
+| `bench` (triage, code review, security) | 30 | 22 (73.3%) | 23–24 on L4 receipts | Pass (±2 items is noise on 30) |
+| `bench-intents` clinc150 (small set) | 30 | 29 (96.7%) after tie-break; domain 28/30 | Committed receipt 96.7% | Pass |
+| `bench-intents` banking77 (small set) | 30 | 0: every request rejected, `at most 26 alternatives` | Committed receipt (Sep 20, older server) | **Harness issue**, see below |
+| `bench-bbox` | 12 | mIoU 0.406 argmax / 0.422 expectation | 0.290 / 0.377 (Cloud Run L4) | Pass |
+| `bench-rerank` | 30 queries | nDCG@10 0.824, 0.856, 0.856 (3 runs); 0% ties, 100% poison quarantine | 0.9265 Cloud Run L4, 0.8502 Vertex L4 | Pass (in the range of earlier receipts) |
+| `bench-permutation` | 16 | 100% accuracy, 6.25% flip rate | EXP-13 | Pass |
+| `bench-decision-index` panel | 22 | 22/22 supported, index 98.89 | 98.89 | Pass |
+| Every policy template through the gateway (`scripts/template_sweep.py`) | 28 | 23 pass; 5 need harness-supplied variables (below) | — | Pass |
+| `bench-ecotone` | — | Not run (needs the C++ ecotone sidecar) | — | Skipped |
+
+**Two findings, both pre-existing, not regressions:**
+- `bench-intents --dataset banking77` sends all 30 candidate intents in one `choice` question. The server has
+  rejected more than 26 options since before v0.1.0 (`ab208dd` had the same check); the committed receipt predates
+  it. The harness should split into brackets (as `dgem systemone serve` does) or use the 26-option slice.
+- `intent_banking77`, `intent_clinc150`, `intent_tiebreak`, `jevbench_generic` and `tn_disambiguation` are harness
+  templates: they take option lists from `bench-intents`, `bench-jev` or `bench-ecotone`, so the gateway's sample
+  variables leave them with fewer than two options (400). The harnesses themselves pass (above).
+
+## Rollback targets (n-1)
+
+| Target | Check | Result |
+| :--- | :--- | :--- |
+| Vertex model `504638d` (redeployed on the production endpoint, then 100% of traffic for the drill, then back) | Contract (`contract_rollback_vertex.json`), calibration | Contract identical (the digit-string `samples` case returns 400, confirming the older model served); 44/50 |
+| Cloud Run revision on `504638d` (tagged URL) | Contract, calibration | Contract identical; 43/50 |
+| Gateway revision on `v0.1.0-5-g3216438` (tagged URL) | Routing, admin 403s, unknown backend / foreign `vertex_url` 400s | All pass |
+
+After the checks, older Cloud Run revisions (81) and three older Vertex model resources were deleted; the drill copy
+of the rollback model was undeployed (its model resource is kept).
diff --git a/benchmarks/runs/20260928-v010-verification/bbox.json b/benchmarks/runs/20260928-v010-verification/bbox.json
new file mode 100644
index 0000000..3480ce5
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/bbox.json
@@ -0,0 +1,835 @@
+{
+  "timestamp": "2026-09-28T22:40:19Z",
+  "mode": "live",
+  "target_url": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "total_cases": 12,
+  "mean_argmax_iou": 0.4060444440474246,
+  "mean_expectation_iou": 0.42223810670454526,
+  "subbin_iou_gain_pct": 1.6193662657120633,
+  "acc_at_50_argmax_pct": 45.45454545454545,
+  "acc_at_50_expectation_pct": 36.36363636363637,
+  "acc_at_75_argmax_pct": 0,
+  "acc_at_75_expectation_pct": 9.090909090909092,
+  "visible_edge_mean_h_norm": 0.510834959652089,
+  "occluded_edge_mean_h_norm": 0.6159108844174354,
+  "occlusion_entropy_multiplier": 1.2056944670288614,
+  "cases": [
+    {
+      "id": "bbox-t1-01-offgrid-cta",
+      "tier": "subbin_precision",
+      "target": "primary_cta_button",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        17.5,
+        32.5,
+        42.5,
+        67.5
+      ],
+      "argmax_box": [
+        5,
+        30,
+        45,
+        65
+      ],
+      "expectation_box": [
+        21.152157483400764,
+        27.21069477604085,
+        53.69358704939415,
+        60.86431133634359
+      ],
+      "argmax_iou": 0.5555555555555556,
+      "expectation_iou": 0.44372563606390053,
+      "iou_gain": -0.11182991949165505,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.7199198070784331,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "65",
+          "argmax_coord": 65,
+          "expected_coord": 60.86431133634359,
+          "confidence": 0.5686374669376082,
+          "raw_entropy_nats": 1.6705198435045123,
+          "normalized_entropy": 0.5486968408594364
+        },
+        "xmin": {
+          "argmax_bin": "30",
+          "argmax_coord": 30,
+          "expected_coord": 27.21069477604085,
+          "confidence": 0.4764840038037175,
+          "raw_entropy_nats": 1.7839541918052817,
+          "normalized_entropy": 0.5859553438335814
+        },
+        "ymax": {
+          "argmax_bin": "45",
+          "argmax_coord": 45,
+          "expected_coord": 53.69358704939415,
+          "confidence": 0.2611663064302673,
+          "raw_entropy_nats": 2.1918120060118076,
+          "normalized_entropy": 0.7199198070784331
+        },
+        "ymin": {
+          "argmax_bin": "05",
+          "argmax_coord": 5,
+          "expected_coord": 21.152157483400764,
+          "confidence": 0.22295563504574856,
+          "raw_entropy_nats": 2.1349499237271767,
+          "normalized_entropy": 0.701242959248351
+        }
+      },
+      "wall_time_ms": 365.91
+    },
+    {
+      "id": "bbox-t1-02-offgrid-badge",
+      "tier": "subbin_precision",
+      "target": "status_alert_badge",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        12.5,
+        62.5,
+        27.5,
+        87.5
+      ],
+      "argmax_box": [
+        5,
+        55,
+        45,
+        90
+      ],
+      "expectation_box": [
+        6.351772724293464,
+        43.542859291568625,
+        43.02579591386572,
+        76.41005761445113
+      ],
+      "argmax_iou": 0.26785714285714285,
+      "expectation_iou": 0.15210876243256746,
+      "iou_gain": -0.11574838042457539,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.6916004365376844,
+      "highest_entropy_edge": "xmax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "90",
+          "argmax_coord": 90,
+          "expected_coord": 76.41005761445113,
+          "confidence": 0.30712093610853064,
+          "raw_entropy_nats": 2.1055930469782944,
+          "normalized_entropy": 0.6916004365376844
+        },
+        "xmin": {
+          "argmax_bin": "55",
+          "argmax_coord": 55,
+          "expected_coord": 43.542859291568625,
+          "confidence": 0.4039405701227056,
+          "raw_entropy_nats": 1.7167176299683393,
+          "normalized_entropy": 0.5638709075345277
+        },
+        "ymax": {
+          "argmax_bin": "45",
+          "argmax_coord": 45,
+          "expected_coord": 43.02579591386572,
+          "confidence": 0.5288669820142594,
+          "raw_entropy_nats": 1.8900020184061392,
+          "normalized_entropy": 0.6207876792064013
+        },
+        "ymin": {
+          "argmax_bin": "05",
+          "argmax_coord": 5,
+          "expected_coord": 6.351772724293464,
+          "confidence": 0.9084044151535442,
+          "raw_entropy_nats": 0.5049431416064665,
+          "normalized_entropy": 0.16585298743406324
+        }
+      },
+      "wall_time_ms": 218.079
+    },
+    {
+      "id": "bbox-t1-03-offgrid-card",
+      "tier": "subbin_precision",
+      "target": "checkout_summary_card",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        22.5,
+        12.5,
+        77.5,
+        52.5
+      ],
+      "argmax_box": [
+        15,
+        5,
+        85,
+        55
+      ],
+      "expectation_box": [
+        12.156892991712704,
+        13.978138136389713,
+        79.5439288300282,
+        52.07456142388727
+      ],
+      "argmax_iou": 0.6285714285714286,
+      "expectation_iou": 0.7841992215920164,
+      "iou_gain": 0.15562779302058782,
+      "pass_acc_50": true,
+      "pass_acc_75": true,
+      "max_edge_normalized_entropy": 0.6850333485951351,
+      "highest_entropy_edge": "xmax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "55",
+          "argmax_coord": 55,
+          "expected_coord": 52.07456142388727,
+          "confidence": 0.3509998360868514,
+          "raw_entropy_nats": 2.0855994003867,
+          "normalized_entropy": 0.6850333485951351
+        },
+        "xmin": {
+          "argmax_bin": "05",
+          "argmax_coord": 5,
+          "expected_coord": 13.978138136389713,
+          "confidence": 0.3092035992525148,
+          "raw_entropy_nats": 1.709873541573761,
+          "normalized_entropy": 0.5616229068925301
+        },
+        "ymax": {
+          "argmax_bin": "85",
+          "argmax_coord": 85,
+          "expected_coord": 79.5439288300282,
+          "confidence": 0.6510512190406311,
+          "raw_entropy_nats": 1.4933279825462324,
+          "normalized_entropy": 0.49049662569177377
+        },
+        "ymin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 12.156892991712704,
+          "confidence": 0.4831072778580788,
+          "raw_entropy_nats": 1.2246555104875605,
+          "normalized_entropy": 0.402248804381718
+        }
+      },
+      "wall_time_ms": 182.153
+    },
+    {
+      "id": "bbox-t1-04-ongrid-control",
+      "tier": "subbin_precision",
+      "target": "primary_cta_button",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        20,
+        30,
+        45,
+        70
+      ],
+      "argmax_box": [
+        5,
+        30,
+        45,
+        65
+      ],
+      "expectation_box": [
+        12.030301471291079,
+        27.876116345053305,
+        47.23409128573254,
+        64.32939938588504
+      ],
+      "argmax_iou": 0.5737704918032787,
+      "expectation_iou": 0.6022453438862129,
+      "iou_gain": 0.02847485208293421,
+      "pass_acc_50": true,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.785676305406351,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "65",
+          "argmax_coord": 65,
+          "expected_coord": 64.32939938588504,
+          "confidence": 0.6589663569662159,
+          "raw_entropy_nats": 1.4412589785955279,
+          "normalized_entropy": 0.4733941063259977
+        },
+        "xmin": {
+          "argmax_bin": "30",
+          "argmax_coord": 30,
+          "expected_coord": 27.876116345053305,
+          "confidence": 0.544332153716671,
+          "raw_entropy_nats": 1.41435054389506,
+          "normalized_entropy": 0.46455579580246326
+        },
+        "ymax": {
+          "argmax_bin": "45",
+          "argmax_coord": 45,
+          "expected_coord": 47.23409128573254,
+          "confidence": 0.21723322755380828,
+          "raw_entropy_nats": 2.3920091405972763,
+          "normalized_entropy": 0.785676305406351
+        },
+        "ymin": {
+          "argmax_bin": "05",
+          "argmax_coord": 5,
+          "expected_coord": 12.030301471291079,
+          "confidence": 0.5260421111608521,
+          "raw_entropy_nats": 1.5410024103203175,
+          "normalized_entropy": 0.5061557081092232
+        }
+      },
+      "wall_time_ms": 223.732
+    },
+    {
+      "id": "bbox-t1-05-micro-pill",
+      "tier": "subbin_precision",
+      "target": "notification_counter_pill",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        47.5,
+        47.5,
+        57.5,
+        57.5
+      ],
+      "argmax_box": [
+        45,
+        45,
+        55,
+        65
+      ],
+      "expectation_box": [
+        32.99516153211613,
+        40.90615314290634,
+        52.152175982685144,
+        61.24776986612952
+      ],
+      "argmax_iou": 0.3333333333333333,
+      "expectation_iou": 0.10497666056723366,
+      "iou_gain": -0.22835667276609967,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.8138353419338493,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "65",
+          "argmax_coord": 65,
+          "expected_coord": 61.24776986612952,
+          "confidence": 0.25361536370300686,
+          "raw_entropy_nats": 2.471295308455041,
+          "normalized_entropy": 0.8117185401014752
+        },
+        "xmin": {
+          "argmax_bin": "45",
+          "argmax_coord": 45,
+          "expected_coord": 40.90615314290634,
+          "confidence": 0.3491627159693018,
+          "raw_entropy_nats": 2.1244014612535045,
+          "normalized_entropy": 0.6977782245684648
+        },
+        "ymax": {
+          "argmax_bin": "55",
+          "argmax_coord": 55,
+          "expected_coord": 52.152175982685144,
+          "confidence": 0.2047548150419721,
+          "raw_entropy_nats": 2.4777399591299183,
+          "normalized_entropy": 0.8138353419338493
+        },
+        "ymin": {
+          "argmax_bin": "45",
+          "argmax_coord": 45,
+          "expected_coord": 32.99516153211613,
+          "confidence": 0.31462996979507835,
+          "raw_entropy_nats": 1.969841960521481,
+          "normalized_entropy": 0.647011805895723
+        }
+      },
+      "wall_time_ms": 235.472
+    },
+    {
+      "id": "bbox-t2-01a-unoccluded-twin",
+      "tier": "paired_occlusion",
+      "target": "primary_cta_button",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        25,
+        20,
+        55,
+        65
+      ],
+      "argmax_box": [
+        15,
+        15,
+        55,
+        65
+      ],
+      "expectation_box": [
+        12.211895510344169,
+        16.292609661334588,
+        56.294149701293364,
+        61.38296049988708
+      ],
+      "argmax_iou": 0.675,
+      "expectation_iou": 0.5922581264962721,
+      "iou_gain": -0.08274187350372797,
+      "pass_acc_50": true,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.730057848245896,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "65",
+          "argmax_coord": 65,
+          "expected_coord": 61.38296049988708,
+          "confidence": 0.5826816183958837,
+          "raw_entropy_nats": 1.5243719448836976,
+          "normalized_entropy": 0.5006932864070348
+        },
+        "xmin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 16.292609661334588,
+          "confidence": 0.7754311527211478,
+          "raw_entropy_nats": 0.8687165649359347,
+          "normalized_entropy": 0.28533754725274013
+        },
+        "ymax": {
+          "argmax_bin": "55",
+          "argmax_coord": 55,
+          "expected_coord": 56.294149701293364,
+          "confidence": 0.3313345157232454,
+          "raw_entropy_nats": 2.2226774998207124,
+          "normalized_entropy": 0.730057848245896
+        },
+        "ymin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 12.211895510344169,
+          "confidence": 0.4785418946908347,
+          "raw_entropy_nats": 1.307121572666597,
+          "normalized_entropy": 0.42933550315497504
+        }
+      },
+      "wall_time_ms": 249.891
+    },
+    {
+      "id": "bbox-t2-01b-right-edge-occluded",
+      "tier": "paired_occlusion",
+      "target": "primary_cta_button",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "xmax",
+      "gt_box": [
+        25,
+        20,
+        55,
+        65
+      ],
+      "argmax_box": [
+        15,
+        15,
+        55,
+        30
+      ],
+      "expectation_box": [
+        17.36084014274473,
+        16.419232190847296,
+        57.31607188311723,
+        47.623091228605354
+      ],
+      "argmax_iou": 0.18181818181818182,
+      "expectation_iou": 0.4687004639626631,
+      "iou_gain": 0.2868822821444813,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.769172438948636,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "30",
+          "argmax_coord": 30,
+          "expected_coord": 47.623091228605354,
+          "confidence": 0.32154140214803,
+          "raw_entropy_nats": 2.1645269911834184,
+          "normalized_entropy": 0.7109578055210422
+        },
+        "xmin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 16.419232190847296,
+          "confidence": 0.762752388502344,
+          "raw_entropy_nats": 0.9557157870103731,
+          "normalized_entropy": 0.31391320200780676
+        },
+        "ymax": {
+          "argmax_bin": "55",
+          "argmax_coord": 55,
+          "expected_coord": 57.31607188311723,
+          "confidence": 0.24451519467865715,
+          "raw_entropy_nats": 2.341762748857572,
+          "normalized_entropy": 0.769172438948636
+        },
+        "ymin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 17.36084014274473,
+          "confidence": 0.4062646485269269,
+          "raw_entropy_nats": 1.763975731437384,
+          "normalized_entropy": 0.5793932439389139
+        }
+      },
+      "wall_time_ms": 207.698
+    },
+    {
+      "id": "bbox-t2-02a-unoccluded-twin",
+      "tier": "paired_occlusion",
+      "target": "warning_modal_banner",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        30,
+        25,
+        70,
+        75
+      ],
+      "argmax_box": [
+        15,
+        30,
+        65,
+        80
+      ],
+      "expectation_box": [
+        15.121390066591166,
+        25.410069942366256,
+        65.12557262255753,
+        73.26880118817054
+      ],
+      "argmax_iou": 0.5384615384615384,
+      "expectation_iou": 0.6198455334348014,
+      "iou_gain": 0.08138399497326299,
+      "pass_acc_50": true,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.7713338583826017,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "80",
+          "argmax_coord": 80,
+          "expected_coord": 73.26880118817054,
+          "confidence": 0.38213250983420644,
+          "raw_entropy_nats": 1.8222524419587,
+          "normalized_entropy": 0.5985347387754221
+        },
+        "xmin": {
+          "argmax_bin": "30",
+          "argmax_coord": 30,
+          "expected_coord": 25.410069942366256,
+          "confidence": 0.43158330206054835,
+          "raw_entropy_nats": 1.3600297982272256,
+          "normalized_entropy": 0.44671367219228103
+        },
+        "ymax": {
+          "argmax_bin": "65",
+          "argmax_coord": 65,
+          "expected_coord": 65.12557262255753,
+          "confidence": 0.27623853817833305,
+          "raw_entropy_nats": 2.348343238821612,
+          "normalized_entropy": 0.7713338583826017
+        },
+        "ymin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 15.121390066591166,
+          "confidence": 0.8247506930617594,
+          "raw_entropy_nats": 0.689422775282416,
+          "normalized_entropy": 0.22644693523689052
+        }
+      },
+      "wall_time_ms": 244.295
+    },
+    {
+      "id": "bbox-t2-02b-bottom-edge-occluded",
+      "tier": "paired_occlusion",
+      "target": "warning_modal_banner",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "ymax",
+      "gt_box": [
+        30,
+        25,
+        70,
+        75
+      ],
+      "argmax_box": [
+        15,
+        15,
+        55,
+        65
+      ],
+      "expectation_box": [
+        15.626596645135345,
+        19.617296007412545,
+        55.905529674558295,
+        71.72675122927838
+      ],
+      "argmax_iou": 0.3333333333333333,
+      "expectation_iou": 0.41907901368094724,
+      "iou_gain": 0.08574568034761393,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.6091443256495772,
+      "highest_entropy_edge": "xmax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "65",
+          "argmax_coord": 65,
+          "expected_coord": 71.72675122927838,
+          "confidence": 0.27749745513153784,
+          "raw_entropy_nats": 1.8545535672520415,
+          "normalized_entropy": 0.6091443256495772
+        },
+        "xmin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 19.617296007412545,
+          "confidence": 0.5214870721114357,
+          "raw_entropy_nats": 1.3385451532723465,
+          "normalized_entropy": 0.43965685280784433
+        },
+        "ymax": {
+          "argmax_bin": "55",
+          "argmax_coord": 55,
+          "expected_coord": 55.905529674558295,
+          "confidence": 0.5920855050995082,
+          "raw_entropy_nats": 1.5857820233105009,
+          "normalized_entropy": 0.5208639633138286
+        },
+        "ymin": {
+          "argmax_bin": "15",
+          "argmax_coord": 15,
+          "expected_coord": 15.626596645135345,
+          "confidence": 0.7845977796595538,
+          "raw_entropy_nats": 0.9070630973437438,
+          "normalized_entropy": 0.2979328009229621
+        }
+      },
+      "wall_time_ms": 232.768
+    },
+    {
+      "id": "bbox-t3-01-detr-dual-buttons",
+      "tier": "detr_and_negative",
+      "target": "dual_dialog_controls",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        60,
+        15,
+        78,
+        45
+      ],
+      "argmax_box": [
+        50,
+        0,
+        70,
+        30
+      ],
+      "expectation_box": [
+        47.59684262708259,
+        7.438262149505142,
+        69.62220401394319,
+        30.718752338783187
+      ],
+      "argmax_iou": 0.15151515151515152,
+      "expectation_iou": 0.16777260168530547,
+      "iou_gain": 0.016257450170153953,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.49703925195318266,
+      "highest_entropy_edge": "xmin",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "30",
+          "argmax_coord": 30,
+          "expected_coord": 30.718752338783187,
+          "confidence": 0.8452378463020024,
+          "raw_entropy_nats": 0.6994278423618554,
+          "normalized_entropy": 0.3037576524272513
+        },
+        "xmin": {
+          "argmax_bin": "00",
+          "argmax_coord": 0,
+          "expected_coord": 7.438262149505142,
+          "confidence": 0.4923523931705466,
+          "raw_entropy_nats": 1.1444751721803101,
+          "normalized_entropy": 0.49703925195318266
+        },
+        "ymax": {
+          "argmax_bin": "70",
+          "argmax_coord": 70,
+          "expected_coord": 69.62220401394319,
+          "confidence": 0.7981758774672804,
+          "raw_entropy_nats": 0.7586371017663648,
+          "normalized_entropy": 0.32947190706420787
+        },
+        "ymin": {
+          "argmax_bin": "50",
+          "argmax_coord": 50,
+          "expected_coord": 47.59684262708259,
+          "confidence": 0.8444984747513771,
+          "raw_entropy_nats": 0.6433705641218901,
+          "normalized_entropy": 0.2794122858171191
+        }
+      },
+      "wall_time_ms": 299.488
+    },
+    {
+      "id": "bbox-t3-02-detr-stacked-banner-cta",
+      "tier": "detr_and_negative",
+      "target": "banner_and_cta",
+      "object_present_gt": true,
+      "object_present_pred": true,
+      "occluded_edge": "none",
+      "gt_box": [
+        12,
+        20,
+        28,
+        80
+      ],
+      "argmax_box": [
+        0,
+        10,
+        20,
+        70
+      ],
+      "expectation_box": [
+        0.939351498627836,
+        10.430615456837323,
+        22.877168007248834,
+        64.17393195877757
+      ],
+      "argmax_iou": 0.22727272727272727,
+      "expectation_iou": 0.2897078099480777,
+      "iou_gain": 0.062435082675350445,
+      "pass_acc_50": false,
+      "pass_acc_75": false,
+      "max_edge_normalized_entropy": 0.4435819206621941,
+      "highest_entropy_edge": "ymax",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "70",
+          "argmax_coord": 70,
+          "expected_coord": 64.17393195877757,
+          "confidence": 0.7670497660301617,
+          "raw_entropy_nats": 0.9565610499259448,
+          "normalized_entropy": 0.4154291855864188
+        },
+        "xmin": {
+          "argmax_bin": "10",
+          "argmax_coord": 10,
+          "expected_coord": 10.430615456837323,
+          "confidence": 0.9425317746810916,
+          "raw_entropy_nats": 0.292550407248538,
+          "normalized_entropy": 0.12705302754658912
+        },
+        "ymax": {
+          "argmax_bin": "20",
+          "argmax_coord": 20,
+          "expected_coord": 22.877168007248834,
+          "confidence": 0.6852267124966106,
+          "raw_entropy_nats": 1.0213851180384357,
+          "normalized_entropy": 0.4435819206621941
+        },
+        "ymin": {
+          "argmax_bin": "00",
+          "argmax_coord": 0,
+          "expected_coord": 0.939351498627836,
+          "confidence": 0.9317602219647692,
+          "raw_entropy_nats": 0.28598646804285066,
+          "normalized_entropy": 0.1242023449700107
+        }
+      },
+      "wall_time_ms": 273.129
+    },
+    {
+      "id": "bbox-t3-03-absent-target-gate",
+      "tier": "detr_and_negative",
+      "target": "primary_cta_button",
+      "object_present_gt": false,
+      "object_present_pred": false,
+      "occluded_edge": "none",
+      "gt_box": [
+        0,
+        0,
+        0,
+        0
+      ],
+      "argmax_box": [
+        0,
+        0,
+        0,
+        0
+      ],
+      "expectation_box": [
+        0,
+        0,
+        0,
+        0
+      ],
+      "argmax_iou": 1,
+      "expectation_iou": 1,
+      "iou_gain": 0,
+      "pass_acc_50": true,
+      "pass_acc_75": true,
+      "max_edge_normalized_entropy": 0,
+      "highest_entropy_edge": "none",
+      "edges": {
+        "xmax": {
+          "argmax_bin": "00",
+          "argmax_coord": 0,
+          "expected_coord": 30.16774161359408,
+          "confidence": 0.6399789926161097,
+          "raw_entropy_nats": 1.2310229079739865,
+          "normalized_entropy": 0.4043402317292489
+        },
+        "xmin": {
+          "argmax_bin": "00",
+          "argmax_coord": 0,
+          "expected_coord": 6.235515423630125,
+          "confidence": 0.8843760791862776,
+          "raw_entropy_nats": 0.6125974212171433,
+          "normalized_entropy": 0.20121297633635446
+        },
+        "ymax": {
+          "argmax_bin": "00",
+          "argmax_coord": 0,
+          "expected_coord": 18.120432351254692,
+          "confidence": 0.7739090215790301,
+          "raw_entropy_nats": 0.9117672319807281,
+          "normalized_entropy": 0.29947791505275045
+        },
+        "ymin": {
+          "argmax_bin": "00",
+          "argmax_coord": 0,
+          "expected_coord": 5.953675971612923,
+          "confidence": 0.8810481689814413,
+          "raw_entropy_nats": 0.6414151780381093,
+          "normalized_entropy": 0.21067842039546109
+        }
+      },
+      "wall_time_ms": 209.249
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/bench.json b/benchmarks/runs/20260928-v010-verification/bench.json
new file mode 100644
index 0000000..6bbe33a
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/bench.json
@@ -0,0 +1,467 @@
+{
+  "timestamp": "2026-09-28T22:41:19Z",
+  "target_url": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "total_cases": 30,
+  "failed_cases": 0,
+  "total_retries": 0,
+  "workers": 4,
+  "total_elapsed_sec": 1.397251334,
+  "mode": "slot",
+  "slot_avg_denoise_ms": 128.5627285639445,
+  "slot_avg_wall_ms": 174.16666666666666,
+  "slot_accuracy_pct": 73.33333333333333,
+  "slot_multi_reads_triggered": 0,
+  "cases": [
+    {
+      "id": "sup-01",
+      "domain": "support",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 129.0457248687744,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 176,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "engineering",
+        "urgent": "yes"
+      }
+    },
+    {
+      "id": "sup-02",
+      "domain": "support",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 126.3132095336914,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 169,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "billing",
+        "urgent": "yes"
+      }
+    },
+    {
+      "id": "sup-03",
+      "domain": "support",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 127.51555442810059,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 182,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "support",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "sup-04",
+      "domain": "support",
+      "tier": "ambiguous",
+      "slot_accurate": false,
+      "slot_denoise_ms": 130.5530071258545,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 176,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "engineering",
+        "urgent": "yes"
+      }
+    },
+    {
+      "id": "sup-05",
+      "domain": "support",
+      "tier": "negation",
+      "slot_accurate": true,
+      "slot_denoise_ms": 125.60033798217773,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 174,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "support",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "sup-06",
+      "domain": "support",
+      "tier": "unambiguous",
+      "slot_accurate": false,
+      "slot_denoise_ms": 114.17627334594727,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 161,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "engineering",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "sup-07",
+      "domain": "support",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 120.44477462768555,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 166,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "billing",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "sup-08",
+      "domain": "support",
+      "tier": "negation",
+      "slot_accurate": false,
+      "slot_denoise_ms": 107.90753364562988,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 154,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "engineering",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "sup-09",
+      "domain": "support",
+      "tier": "ambiguous",
+      "slot_accurate": false,
+      "slot_denoise_ms": 146.63958549499512,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 188,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "engineering",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "sup-10",
+      "domain": "support",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 147.50981330871582,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 198,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "team": "support",
+        "urgent": "no"
+      }
+    },
+    {
+      "id": "code-01",
+      "domain": "code_review",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 144.8037624359131,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 194,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "security"
+      }
+    },
+    {
+      "id": "code-02",
+      "domain": "code_review",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 142.61794090270996,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 193,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "bugfix"
+      }
+    },
+    {
+      "id": "code-03",
+      "domain": "code_review",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 129.0433406829834,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 173,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "refactor"
+      }
+    },
+    {
+      "id": "code-04",
+      "domain": "code_review",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 122.89690971374512,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 166,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "feature"
+      }
+    },
+    {
+      "id": "code-05",
+      "domain": "code_review",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 117.36321449279785,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 160,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "no",
+        "category": "security"
+      }
+    },
+    {
+      "id": "code-06",
+      "domain": "code_review",
+      "tier": "ambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 120.90015411376953,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 168,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "refactor"
+      }
+    },
+    {
+      "id": "code-07",
+      "domain": "code_review",
+      "tier": "negation",
+      "slot_accurate": true,
+      "slot_denoise_ms": 130.95498085021973,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 178,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "refactor"
+      }
+    },
+    {
+      "id": "code-08",
+      "domain": "code_review",
+      "tier": "complex",
+      "slot_accurate": false,
+      "slot_denoise_ms": 122.30110168457031,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 167,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "no",
+        "category": "refactor"
+      }
+    },
+    {
+      "id": "code-09",
+      "domain": "code_review",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 126.31702423095703,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 171,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "no",
+        "category": "security"
+      }
+    },
+    {
+      "id": "code-10",
+      "domain": "code_review",
+      "tier": "negation",
+      "slot_accurate": true,
+      "slot_denoise_ms": 120.41521072387695,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 164,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "approved": "yes",
+        "category": "feature"
+      }
+    },
+    {
+      "id": "sec-01",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 138.76867294311523,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 186,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "revoke_key",
+        "data_compromise": "yes"
+      }
+    },
+    {
+      "id": "sec-02",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 136.15775108337402,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 183,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "isolate_host",
+        "data_compromise": "yes"
+      }
+    },
+    {
+      "id": "sec-03",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 133.9273452758789,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 181,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "monitor",
+        "data_compromise": "no"
+      }
+    },
+    {
+      "id": "sec-04",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": false,
+      "slot_denoise_ms": 132.19475746154785,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 180,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "isolate_host",
+        "data_compromise": "no"
+      }
+    },
+    {
+      "id": "sec-05",
+      "domain": "security",
+      "tier": "negation",
+      "slot_accurate": true,
+      "slot_denoise_ms": 133.91566276550293,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 176,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "monitor",
+        "data_compromise": "no"
+      }
+    },
+    {
+      "id": "sec-06",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": false,
+      "slot_denoise_ms": 137.54940032958984,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 181,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "isolate_host",
+        "data_compromise": "no"
+      }
+    },
+    {
+      "id": "sec-07",
+      "domain": "security",
+      "tier": "ambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 134.12189483642578,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 177,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "revoke_key",
+        "data_compromise": "no"
+      }
+    },
+    {
+      "id": "sec-08",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 134.4926357269287,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 183,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "monitor",
+        "data_compromise": "no"
+      }
+    },
+    {
+      "id": "sec-09",
+      "domain": "security",
+      "tier": "ambiguous",
+      "slot_accurate": false,
+      "slot_denoise_ms": 111.70411109924316,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 150,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "revoke_key",
+        "data_compromise": "yes"
+      }
+    },
+    {
+      "id": "sec-10",
+      "domain": "security",
+      "tier": "unambiguous",
+      "slot_accurate": true,
+      "slot_denoise_ms": 110.73017120361328,
+      "slot_prefill_ms": 0,
+      "slot_wall_time_ms": 150,
+      "slot_samples": 4,
+      "slot_extended": false,
+      "slot_answers": {
+        "action": "monitor",
+        "data_compromise": "no"
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/calibration.json b/benchmarks/runs/20260928-v010-verification/calibration.json
new file mode 100644
index 0000000..62d101a
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/calibration.json
@@ -0,0 +1,1724 @@
+{
+  "timestamp": "2026-09-28T22:40:03Z",
+  "idc_config": "none",
+  "target_url": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "workers": 4,
+  "total_cases": 50,
+  "total_correct": 46,
+  "overall_accuracy_pct": 92,
+  "chance_baseline_pct": 29.205128205128204,
+  "chance_corrected_acc_pct": 88.69974646867078,
+  "avg_confidence": 0.8994134443640456,
+  "avg_entropy_nats": 0.26222162823775724,
+  "avg_normalized_entropy": 0.19563988053335377,
+  "avg_brier_score": 0.17054863235383164,
+  "ece_10bin": 0.09073642238505861,
+  "p50_latency_ms": 232.99999999999997,
+  "p95_latency_ms": 682.2,
+  "avg_wall_time_ms": 238.04,
+  "total_elapsed_sec": 3.05781496,
+  "temperature_scale": 1,
+  "jev_parity": {
+    "protocol_version": "jevbench::v1.4",
+    "temperature_applied": 1,
+    "optimal_temperature": 1.5,
+    "chance_baseline_pct": 29.205128205128204,
+    "chance_corrected_acc_pct": 88.69974646867078,
+    "intelligence_score": 88.60248324590074,
+    "calibration_score": 83.15792876871623,
+    "speed_score": 80.30178686879835,
+    "cost_score": 57.55079956087546,
+    "composite_jevbench_score": 76.38912442713793,
+    "ece_10bin": 0.09073642238505861,
+    "brier_mean": 0.17054863235383164,
+    "soft_tvd_mean": 0.1553685798555579,
+    "probability_fidelity": 84.4631420144442,
+    "p50_latency_sec": 0.23299999999999998,
+    "p95_latency_sec": 0.6822,
+    "usd_per_1k_decisions": 0.026,
+    "presets": {
+      "Balanced 33:33:33 (no calibration)": 74.25758510636571,
+      "Emphasis on Accuracy 60:20:20": 79.69349080032276,
+      "Emphasis on Speed 20:60:20": 76.61866431379038,
+      "JevBench Score (25:25:25:25)": 76.38912442713793
+    },
+    "raw_t1_ece_10bin": 0.09073642238505861,
+    "raw_t1_brier_mean": 0.17054863235383164,
+    "raw_t1_calibration_score": 83.15792876871623,
+    "raw_t1_composite_score": 76.38912442713793,
+    "reliability_bins": [
+      {
+        "bin_index": 0,
+        "range_label": "[0.0, 0.1)",
+        "lower": 0,
+        "upper": 0.1,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 1,
+        "range_label": "[0.1, 0.2)",
+        "lower": 0.1,
+        "upper": 0.2,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 2,
+        "range_label": "[0.2, 0.3)",
+        "lower": 0.2,
+        "upper": 0.30000000000000004,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 3,
+        "range_label": "[0.3, 0.4)",
+        "lower": 0.30000000000000004,
+        "upper": 0.4,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 4,
+        "range_label": "[0.4, 0.5)",
+        "lower": 0.4,
+        "upper": 0.5,
+        "count": 3,
+        "avg_confidence": 0.49359589625456807,
+        "empirical_accuracy": 0.6666666666666666,
+        "abs_gap": 0.17307077041209856
+      },
+      {
+        "bin_index": 5,
+        "range_label": "[0.5, 0.6)",
+        "lower": 0.5,
+        "upper": 0.6000000000000001,
+        "count": 3,
+        "avg_confidence": 0.5593963107886296,
+        "empirical_accuracy": 1,
+        "abs_gap": 0.4406036892113704
+      },
+      {
+        "bin_index": 6,
+        "range_label": "[0.6, 0.7)",
+        "lower": 0.6000000000000001,
+        "upper": 0.7000000000000001,
+        "count": 2,
+        "avg_confidence": 0.6505973899600804,
+        "empirical_accuracy": 1,
+        "abs_gap": 0.34940261003991957
+      },
+      {
+        "bin_index": 7,
+        "range_label": "[0.7, 0.8)",
+        "lower": 0.7000000000000001,
+        "upper": 0.8,
+        "count": 1,
+        "avg_confidence": 0.7567541484249153,
+        "empirical_accuracy": 1,
+        "abs_gap": 0.24324585157508472
+      },
+      {
+        "bin_index": 8,
+        "range_label": "[0.8, 0.9)",
+        "lower": 0.8,
+        "upper": 0.9,
+        "count": 5,
+        "avg_confidence": 0.8620012354817673,
+        "empirical_accuracy": 0.6,
+        "abs_gap": 0.26200123548176735
+      },
+      {
+        "bin_index": 9,
+        "range_label": "[0.9, 1.0]",
+        "lower": 0.9,
+        "upper": 1,
+        "count": 36,
+        "avg_confidence": 0.9845483469810767,
+        "empirical_accuracy": 0.9722222222222222,
+        "abs_gap": 0.012326124758854529
+      }
+    ],
+    "endpoint_kind": "gpu",
+    "raw_speed_score": 87.98732363902603,
+    "adjusted_speed_score": 80.30178686879835,
+    "v14_intelligence_score": 88.60248324590074,
+    "v14_calibration_score": 83.15792876871623,
+    "v14_composite_score": 75.27193897249262,
+    "v14_public_only_estimate": true
+  },
+  "categories": [
+    {
+      "name": "agent-trajectory",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 35.714285714285715,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9991223320313772,
+      "avg_entropy_nats": 0.0070585108427689966,
+      "avg_normalized_entropy": 0.0057450885223036974,
+      "avg_brier_score": 0.0000027852315522696895,
+      "ece_10bin": 0.000877667968622764,
+      "avg_latency_ms": 128.57142857142858
+    },
+    {
+      "name": "emotion",
+      "total": 4,
+      "correct": 4,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 3.8461538461538463,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.8696355688675891,
+      "avg_entropy_nats": 0.3905797410864087,
+      "avg_normalized_entropy": 0.11987973239233507,
+      "avg_brier_score": 0.07025965587432705,
+      "ece_10bin": 0.1303644311324109,
+      "avg_latency_ms": 189.5
+    },
+    {
+      "name": "grounding",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9934005844859043,
+      "avg_entropy_nats": 0.038457489889493046,
+      "avg_normalized_entropy": 0.055482429948609066,
+      "avg_brier_score": 0.00011787924463482194,
+      "ece_10bin": 0.006599415514095708,
+      "avg_latency_ms": 103
+    },
+    {
+      "name": "guardrail",
+      "total": 4,
+      "correct": 4,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.997356869683358,
+      "avg_entropy_nats": 0.017468667803529264,
+      "avg_normalized_entropy": 0.025201960411088367,
+      "avg_brier_score": 0.00002331808755940371,
+      "ece_10bin": 0.002643130316641984,
+      "avg_latency_ms": 408.25
+    },
+    {
+      "name": "intent",
+      "total": 7,
+      "correct": 6,
+      "accuracy_pct": 85.71428571428571,
+      "chance_baseline_pct": 6.41025641025641,
+      "chance_corrected_pct": 84.7358121330724,
+      "avg_confidence": 0.9592409391435439,
+      "avg_entropy_nats": 0.16979941658450848,
+      "avg_normalized_entropy": 0.053921330160339995,
+      "avg_brier_score": 0.21896650197484907,
+      "ece_10bin": 0.10750616353734944,
+      "avg_latency_ms": 165.57142857142858
+    },
+    {
+      "name": "nli",
+      "total": 3,
+      "correct": 1,
+      "accuracy_pct": 33.33333333333333,
+      "chance_baseline_pct": 33.33333333333333,
+      "avg_confidence": 0.665383542660572,
+      "avg_entropy_nats": 0.6608306760652969,
+      "avg_normalized_entropy": 0.6015140035129659,
+      "avg_brier_score": 1.1668859011384538,
+      "ece_10bin": 0.6248783486062012,
+      "avg_latency_ms": 292.6666666666667
+    },
+    {
+      "name": "nli-soft",
+      "total": 6,
+      "correct": 5,
+      "accuracy_pct": 83.33333333333334,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 74.99999999999999,
+      "avg_confidence": 0.7880767333894773,
+      "avg_entropy_nats": 0.45602128067304504,
+      "avg_normalized_entropy": 0.41508845784521253,
+      "avg_brier_score": 0.36901655876457456,
+      "ece_10bin": 0.3154141033926428,
+      "avg_latency_ms": 218.33333333333334
+    },
+    {
+      "name": "ordinal",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 20,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.8720877132679477,
+      "avg_entropy_nats": 0.4134813123557858,
+      "avg_normalized_entropy": 0.2569103841542046,
+      "avg_brier_score": 0.05336304949111308,
+      "ece_10bin": 0.1279122867320524,
+      "avg_latency_ms": 218.14285714285714
+    },
+    {
+      "name": "reading-comprehension",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9428343560633439,
+      "avg_entropy_nats": 0.2155542343870916,
+      "avg_normalized_entropy": 0.31097902499287433,
+      "avg_brier_score": 0.007285310611920256,
+      "ece_10bin": 0.05716564393665613,
+      "avg_latency_ms": 247.5
+    },
+    {
+      "name": "retrieval",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9909316363069836,
+      "avg_entropy_nats": 0.05123900573805215,
+      "avg_normalized_entropy": 0.07392225947836897,
+      "avg_brier_score": 0.00017990431924373555,
+      "ece_10bin": 0.009068363693016446,
+      "avg_latency_ms": 476.5
+    },
+    {
+      "name": "toxicity",
+      "total": 6,
+      "correct": 6,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 35,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.851766898477893,
+      "avg_entropy_nats": 0.3362319875972049,
+      "avg_normalized_entropy": 0.30163157485769637,
+      "avg_brier_score": 0.10167502010117106,
+      "ece_10bin": 0.148233101522107,
+      "avg_latency_ms": 347.1666666666667
+    }
+  ],
+  "tiers": [
+    {
+      "name": "adversarial",
+      "total": 5,
+      "correct": 4,
+      "accuracy_pct": 80,
+      "chance_baseline_pct": 43.33333333333333,
+      "chance_corrected_pct": 64.70588235294119,
+      "avg_confidence": 0.8994641070181622,
+      "avg_entropy_nats": 0.23483353778814414,
+      "avg_normalized_entropy": 0.2180723049611834,
+      "avg_brier_score": 0.4273891690952767,
+      "ece_10bin": 0.27516099058553967,
+      "avg_latency_ms": 191
+    },
+    {
+      "name": "ambiguous",
+      "total": 9,
+      "correct": 8,
+      "accuracy_pct": 88.88888888888889,
+      "chance_baseline_pct": 27.891737891737893,
+      "chance_corrected_pct": 84.59107072303436,
+      "avg_confidence": 0.8767805178574573,
+      "avg_entropy_nats": 0.3110871949011742,
+      "avg_normalized_entropy": 0.26740599051973585,
+      "avg_brier_score": 0.17700279800216384,
+      "ece_10bin": 0.12168704958302232,
+      "avg_latency_ms": 345.3333333333333
+    },
+    {
+      "name": "easy",
+      "total": 17,
+      "correct": 17,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 31.138763197586727,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9411757578741802,
+      "avg_entropy_nats": 0.15729418321979322,
+      "avg_normalized_entropy": 0.08593797982121348,
+      "avg_brier_score": 0.03960423854082793,
+      "ece_10bin": 0.05882424212581975,
+      "avg_latency_ms": 237.41176470588235
+    },
+    {
+      "name": "held-out",
+      "total": 8,
+      "correct": 7,
+      "accuracy_pct": 87.5,
+      "chance_baseline_pct": 21.442307692307693,
+      "chance_corrected_pct": 84.08812729498165,
+      "avg_confidence": 0.857832966334099,
+      "avg_entropy_nats": 0.45694897833593734,
+      "avg_normalized_entropy": 0.2866674736479639,
+      "avg_brier_score": 0.23875662316043877,
+      "ece_10bin": 0.1697287073567741,
+      "avg_latency_ms": 226.875
+    },
+    {
+      "name": "high-entropy",
+      "total": 3,
+      "correct": 2,
+      "accuracy_pct": 66.66666666666666,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 49.99999999999999,
+      "avg_confidence": 0.6079627293236245,
+      "avg_entropy_nats": 0.7707160610519835,
+      "avg_normalized_entropy": 0.70153599136084,
+      "avg_brier_score": 0.7348664434710118,
+      "ece_10bin": 0.5990189442406156,
+      "avg_latency_ms": 239.66666666666666
+    },
+    {
+      "name": "localization",
+      "total": 3,
+      "correct": 3,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 25,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9984554003557768,
+      "avg_entropy_nats": 0.011942009755889197,
+      "avg_normalized_entropy": 0.008614339126534481,
+      "avg_brier_score": 0.000005989384545296953,
+      "ece_10bin": 0.0015445996442231547,
+      "avg_latency_ms": 131
+    },
+    {
+      "name": "low-entropy",
+      "total": 3,
+      "correct": 3,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.96819073745533,
+      "avg_entropy_nats": 0.14132650029410657,
+      "avg_normalized_entropy": 0.12864092432958507,
+      "avg_brier_score": 0.003166674058137151,
+      "ece_10bin": 0.03180926254466998,
+      "avg_latency_ms": 197
+    },
+    {
+      "name": "out-of-scope",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 8.333333333333332,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9979244032281467,
+      "avg_entropy_nats": 0.01779115705332429,
+      "avg_normalized_entropy": 0.00715968829446456,
+      "avg_brier_score": 0.000009114488526636751,
+      "ece_10bin": 0.0020755967718533164,
+      "avg_latency_ms": 142.5
+    }
+  ],
+  "cases": [
+    {
+      "id": "tox-01",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9997380056594533,
+      "entropy_nats": 0.0024226765081747153,
+      "normalized_entropy": 0.0034951833840218523,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 1.3728206895704768e-7,
+      "wall_time_ms": 274,
+      "top_probabilities": {
+        "no": 0.9997380056594533,
+        "yes": 0.00026199434054678233
+      }
+    },
+    {
+      "id": "tox-02",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9901378161024849,
+      "entropy_nats": 0.0553672891976805,
+      "normalized_entropy": 0.07987811355295873,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0001945253424568139,
+      "wall_time_ms": 274,
+      "top_probabilities": {
+        "no": 0.009862183897515104,
+        "yes": 0.9901378161024849
+      }
+    },
+    {
+      "id": "tox-03",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "ambiguous",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.689478181610305,
+      "entropy_nats": 0.6195175450874202,
+      "normalized_entropy": 0.8937748900413259,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.19284759939208537,
+      "tvd_gold": 0.06052181838969495,
+      "stderr": 0.04688206430349549,
+      "wall_time_ms": 897,
+      "top_probabilities": {
+        "no": 0.689478181610305,
+        "yes": 0.310521818389695
+      }
+    },
+    {
+      "id": "marco-01",
+      "metric": "calibration/ms-marco-passage-relevance-check",
+      "category": "retrieval",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9937095743108,
+      "entropy_nats": 0.03815504694312889,
+      "normalized_entropy": 0.05504609700973766,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00007913891070269506,
+      "wall_time_ms": 273,
+      "top_probabilities": {
+        "no": 0.9937095743108,
+        "yes": 0.006290425689199977
+      }
+    },
+    {
+      "id": "marco-02",
+      "metric": "calibration/ms-marco-passage-relevance-check",
+      "category": "retrieval",
+      "tier": "ambiguous",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.988153698303167,
+      "entropy_nats": 0.06432296453297541,
+      "normalized_entropy": 0.0927984219470003,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00028066972778477607,
+      "tvd_gold": 0.23815369830316702,
+      "wall_time_ms": 680,
+      "top_probabilities": {
+        "no": 0.011846301696832994,
+        "yes": 0.988153698303167
+      }
+    },
+    {
+      "id": "pi-01",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9986313536296104,
+      "entropy_nats": 0.010392471920703066,
+      "normalized_entropy": 0.014993167702576115,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0000037463857743611495,
+      "wall_time_ms": 684,
+      "top_probabilities": {
+        "no": 0.9986313536296104,
+        "yes": 0.0013686463703896164
+      }
+    },
+    {
+      "id": "pi-02",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9938454186493256,
+      "entropy_nats": 0.03746585959133382,
+      "normalized_entropy": 0.05405180983505951,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00007575774320413753,
+      "wall_time_ms": 685,
+      "top_probabilities": {
+        "no": 0.00615458135067437,
+        "yes": 0.9938454186493256
+      }
+    },
+    {
+      "id": "pi-03",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "adversarial",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9974182309386191,
+      "entropy_nats": 0.017963919272030893,
+      "normalized_entropy": 0.025916457248688646,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.000013331062972606776,
+      "wall_time_ms": 138,
+      "top_probabilities": {
+        "no": 0.0025817690613808256,
+        "yes": 0.9974182309386191
+      }
+    },
+    {
+      "id": "pi-04",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "adversarial",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9995324755158768,
+      "entropy_nats": 0.004052420430049268,
+      "normalized_entropy": 0.0058464068580292,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 4.3715828650940043e-7,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "no": 0.9995324755158768,
+        "yes": 0.00046752448412329834
+      }
+    },
+    {
+      "id": "ad-01",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9999104310088098,
+      "entropy_nats": 0.0009243928856136462,
+      "normalized_entropy": 0.0013336170319078462,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 1.6045208365649435e-8,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "no": 0.9999104310088098,
+        "yes": 0.00008956899119016347
+      }
+    },
+    {
+      "id": "ad-02",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9998584057531458,
+      "entropy_nats": 0.0013964696074067237,
+      "normalized_entropy": 0.002014679777357838,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 4.0097861484431204e-8,
+      "wall_time_ms": 121,
+      "top_probabilities": {
+        "no": 0.000141594246854272,
+        "yes": 0.9998584057531458
+      }
+    },
+    {
+      "id": "ad-03",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "ambiguous",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9995366689049231,
+      "entropy_nats": 0.0040202483658262015,
+      "normalized_entropy": 0.00579999238051942,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 4.293514073303225e-7,
+      "tvd_gold": 0.2495366689049231,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "no": 0.9995366689049231,
+        "yes": 0.0004633310950769815
+      }
+    },
+    {
+      "id": "ground-01",
+      "metric": "calibration/grounding-claim-support-check",
+      "category": "grounding",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9894779148350512,
+      "entropy_nats": 0.0583870430997275,
+      "normalized_entropy": 0.08423469753214703,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00022142855243687087,
+      "wall_time_ms": 105,
+      "top_probabilities": {
+        "no": 0.010522085164948802,
+        "yes": 0.9894779148350512
+      }
+    },
+    {
+      "id": "ground-02",
+      "metric": "calibration/grounding-claim-support-check",
+      "category": "grounding",
+      "tier": "adversarial",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9973232541367574,
+      "entropy_nats": 0.018527936679258592,
+      "normalized_entropy": 0.0267301623650711,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.000014329936832773005,
+      "wall_time_ms": 101,
+      "top_probabilities": {
+        "no": 0.9973232541367574,
+        "yes": 0.002676745863242619
+      }
+    },
+    {
+      "id": "boolq-01",
+      "metric": "calibration/boolq-passage-answer-check",
+      "category": "reading-comprehension",
+      "tier": "held-out",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.962192673636614,
+      "entropy_nats": 0.16091198374847496,
+      "normalized_entropy": 0.2321469209735303,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0028587878534951667,
+      "stderr": 0.011284180834627466,
+      "wall_time_ms": 222,
+      "top_probabilities": {
+        "no": 0.03780732636338604,
+        "yes": 0.962192673636614
+      }
+    },
+    {
+      "id": "boolq-02",
+      "metric": "calibration/boolq-passage-answer-check",
+      "category": "reading-comprehension",
+      "tier": "held-out",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9234760384900738,
+      "entropy_nats": 0.27019648502570826,
+      "normalized_entropy": 0.38981112901221837,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.011711833370345346,
+      "stderr": 0.00825910432575482,
+      "wall_time_ms": 273,
+      "top_probabilities": {
+        "no": 0.9234760384900738,
+        "yes": 0.0765239615099262
+      }
+    },
+    {
+      "id": "emo-01",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "easy",
+      "expected": "sadness",
+      "actual": "sadness",
+      "accurate": true,
+      "confidence": 0.5960139103256583,
+      "entropy_nats": 1.0745576026503056,
+      "normalized_entropy": 0.3298114681717944,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.2599412000743276,
+      "stderr": 0.01576077588547065,
+      "wall_time_ms": 254,
+      "top_probabilities": {
+        "admiration": 0.0028378347944344176,
+        "amusement": 0.0003618459180451318,
+        "anger": 0.0016956552609511885,
+        "annoyance": 0.01822206367312231,
+        "approval": 0.0006504227716337304,
+        "caring": 0.0001522300391610143,
+        "confusion": 0.00027211344886274573,
+        "curiosity": 0.00012826704395155311,
+        "desire": 0.00021678206357178908,
+        "disappointment": 0.3071289495780529,
+        "disapproval": 0.0033897111285123424,
+        "disgust": 0.00030648474351209536,
+        "embarrassment": 0.0006104633971505301,
+        "excitement": 0.00017269182226494842,
+        "fear": 0.00006665892174075411,
+        "gratitude": 0.00028350425987652895,
+        "joy": 0.00087237850908879,
+        "love": 0.00012783936725380607,
+        "neutral": 0.04278178328357303,
+        "optimism": 0.00023141752927512428,
+        "pride": 0.00014104034433237166,
+        "realization": 0.006547964487583826,
+        "relief": 0.002173512334356351,
+        "remorse": 0.0015207739791020364,
+        "sadness": 0.5960139103256583,
+        "surprise": 0.01309370097493241
+      }
+    },
+    {
+      "id": "emo-02",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "easy",
+      "expected": "gratitude",
+      "actual": "gratitude",
+      "accurate": true,
+      "confidence": 0.9962320685855028,
+      "entropy_nats": 0.03238595169888197,
+      "normalized_entropy": 0.00994014490391642,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.000018442827507643172,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "admiration": 0.00014221122256854803,
+        "amusement": 0.00013091794386498637,
+        "anger": 0.00003298879750441875,
+        "annoyance": 0.00005598391212939385,
+        "approval": 0.000030163261936364802,
+        "caring": 0.00006656450738025829,
+        "confusion": 0.00005598391212939385,
+        "curiosity": 0.000027568745267949683,
+        "desire": 0.00005598391212939385,
+        "disappointment": 0.00023168422379227907,
+        "disapproval": 0.00015441741115732136,
+        "disgust": 0.00014221122256854803,
+        "embarrassment": 0.000030163261936364802,
+        "excitement": 0.00010189328706593375,
+        "fear": 0.000005360641881004924,
+        "gratitude": 0.9962320685855028,
+        "joy": 0.0020038766145960177,
+        "love": 0.00005131144323582339,
+        "neutral": 0.00008604120011621586,
+        "optimism": 0.000027568745267949683,
+        "pride": 0.000020999093450988857,
+        "realization": 0.000036064600451303214,
+        "relief": 0.000013235729751135725,
+        "remorse": 0.000020999093450988857,
+        "sadness": 0.000012054407072434055,
+        "surprise": 0.00023168422379227907
+      }
+    },
+    {
+      "id": "emo-03",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "ambiguous",
+      "expected": "confusion",
+      "actual": "confusion",
+      "accurate": true,
+      "confidence": 0.8892869722989839,
+      "entropy_nats": 0.4291681099559232,
+      "normalized_entropy": 0.13172357078668412,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.021067937593140056,
+      "tvd_gold": 0.1392869722989839,
+      "stderr": 0.03435112567276805,
+      "wall_time_ms": 253,
+      "top_probabilities": {
+        "admiration": 0.0008639488530907038,
+        "amusement": 0.00022523829816866966,
+        "anger": 0.0003033518615356719,
+        "annoyance": 0.0006535132700854957,
+        "approval": 0.00009104384277522901,
+        "caring": 0.000026803416725892184,
+        "confusion": 0.8892869722989839,
+        "curiosity": 0.0934465141809194,
+        "desire": 0.00007485679437560771,
+        "disappointment": 0.00020660608591554353,
+        "disapproval": 0.0004210685398043206,
+        "disgust": 0.00010515074049614515,
+        "embarrassment": 0.00014545234284490645,
+        "excitement": 0.0003261077714554938,
+        "fear": 0.00004147017563840904,
+        "gratitude": 0.00012825557419773507,
+        "joy": 0.0009256653177737719,
+        "love": 0.00004347550693630596,
+        "neutral": 0.0016164039333243228,
+        "optimism": 0.00001610762525611453,
+        "pride": 0.000041329875411768605,
+        "realization": 0.002682405288254463,
+        "relief": 0.00001610539829585939,
+        "remorse": 0.000043322835504477875,
+        "sadness": 0.00015026992261123824,
+        "surprise": 0.008118560249619001
+      }
+    },
+    {
+      "id": "emo-04",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "ambiguous",
+      "expected": "neutral",
+      "actual": "neutral",
+      "accurate": true,
+      "confidence": 0.9970093242602114,
+      "entropy_nats": 0.026207300040523768,
+      "normalized_entropy": 0.008043745706945339,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.000011043002332911488,
+      "tvd_gold": 0.24700932426021138,
+      "wall_time_ms": 131,
+      "top_probabilities": {
+        "admiration": 0.00030396978557262227,
+        "amusement": 0.00007607518171188136,
+        "anger": 0.00000896236369019224,
+        "annoyance": 0.00015453525599108542,
+        "approval": 0.000050602623659201384,
+        "caring": 0.000014023727407477503,
+        "confusion": 0.000010732909573040638,
+        "curiosity": 0.000004709632892219741,
+        "desire": 0.000014023727407477503,
+        "disappointment": 0.000039431393133094644,
+        "disapproval": 0.000039431393133094644,
+        "disgust": 0.000011738312032871143,
+        "embarrassment": 0.00003625640484393636,
+        "excitement": 0.00002172596745254236,
+        "fear": 0.000003232681608174195,
+        "gratitude": 0.00004286740635655554,
+        "joy": 0.0006655479279823336,
+        "love": 0.000012832766437804158,
+        "neutral": 0.9970093242602114,
+        "optimism": 0.00000164853231157561,
+        "pride": 0.0000014235259270220791,
+        "realization": 0.00007607518171188136,
+        "relief": 0.000008184977556311786,
+        "remorse": 0.000005169146818951976,
+        "sadness": 0.00016686314299337873,
+        "surprise": 0.0012206117715838202
+      }
+    },
+    {
+      "id": "clinc-01",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "easy",
+      "expected": "bill_balance",
+      "actual": "bill_balance",
+      "accurate": true,
+      "confidence": 0.9902509018744544,
+      "entropy_nats": 0.07035048634387354,
+      "normalized_entropy": 0.028311118387435396,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.0001199146467711555,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "account_blocked": 0.00047919223423824423,
+        "balance": 0.0015423086703241146,
+        "bill_balance": 0.9902509018744544,
+        "bill_due": 0.0015423086703241146,
+        "freeze_account": 0.00029420321182839363,
+        "interest_rate": 0.000008800347440347128,
+        "min_payment": 0.000040604397025496964,
+        "out_of_scope": 0.004172430448509115,
+        "pay_bill": 0.0015423086703241146,
+        "pin_change": 0.000014863402695596138,
+        "routing": 0.00005603903641798807,
+        "transfer": 0.00005603903641798807
+      }
+    },
+    {
+      "id": "clinc-02",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "easy",
+      "expected": "freeze_account",
+      "actual": "freeze_account",
+      "accurate": true,
+      "confidence": 0.9966944292430922,
+      "entropy_nats": 0.026382103060531834,
+      "normalized_entropy": 0.01061693929741088,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.000014747681280887302,
+      "wall_time_ms": 112,
+      "top_probabilities": {
+        "account_blocked": 0.0013624713173290375,
+        "balance": 0.00016834690893196912,
+        "bill_balance": 0.00000837747144945541,
+        "bill_due": 0.00011720988746274387,
+        "freeze_account": 0.9966944292430922,
+        "interest_rate": 0.000001993208357160913,
+        "min_payment": 0.000005907005010618217,
+        "out_of_scope": 0.0013624713173290375,
+        "pay_bill": 0.0002566332716109701,
+        "pin_change": 0.00000452625983790975,
+        "routing": 0.000009952782829379132,
+        "transfer": 0.000007681326759679617
+      }
+    },
+    {
+      "id": "clinc-03",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "out-of-scope",
+      "expected": "out_of_scope",
+      "actual": "out_of_scope",
+      "accurate": true,
+      "confidence": 0.9997565929831389,
+      "entropy_nats": 0.002738953979592599,
+      "normalized_entropy": 0.0011022361664275286,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 7.003346036760498e-8,
+      "wall_time_ms": 117,
+      "top_probabilities": {
+        "account_blocked": 0.0000643347817489966,
+        "balance": 0.000027174894335716328,
+        "bill_balance": 0.000004193054580941234,
+        "bill_due": 0.000034540123713254665,
+        "freeze_account": 0.000016640378727868412,
+        "interest_rate": 0.0000013948864724191134,
+        "min_payment": 0.000008460708898573911,
+        "out_of_scope": 0.9997565929831389,
+        "pay_bill": 0.0000643347817489966,
+        "pin_change": 0.0000026690983500834736,
+        "routing": 0.000007760755302564626,
+        "transfer": 0.000011903552981721317
+      }
+    },
+    {
+      "id": "clinc-04",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "out-of-scope",
+      "expected": "out_of_scope",
+      "actual": "out_of_scope",
+      "accurate": true,
+      "confidence": 0.9960922134731545,
+      "entropy_nats": 0.032843360127055986,
+      "normalized_entropy": 0.013217140422501591,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.000018158943592905896,
+      "wall_time_ms": 168,
+      "top_probabilities": {
+        "account_blocked": 0.0009658789526271179,
+        "balance": 0.0005033664690503423,
+        "bill_balance": 0.00007568043827436337,
+        "bill_due": 0.00017139408075518458,
+        "freeze_account": 0.00015822574318062003,
+        "interest_rate": 0.0008998140638038455,
+        "min_payment": 0.00003504248630643669,
+        "out_of_scope": 0.9960922134731545,
+        "pay_bill": 0.0008998140638038455,
+        "pin_change": 0.000009921112739850957,
+        "routing": 0.000053964261735323905,
+        "transfer": 0.000134684854568743
+      }
+    },
+    {
+      "id": "anli-01",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "adversarial",
+      "expected": "contradiction",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.9422887834180014,
+      "entropy_nats": 0.25312009893543086,
+      "normalized_entropy": 0.23039984309869518,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.803501228094789,
+      "stderr": 0.015918437774794354,
+      "wall_time_ms": 294,
+      "top_probabilities": {
+        "contradiction": 0.04324310790528882,
+        "entailment": 0.9422887834180014,
+        "neutral": 0.014468108676709779
+      }
+    },
+    {
+      "id": "anli-02",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "adversarial",
+      "expected": "contradiction",
+      "actual": "contradiction",
+      "accurate": true,
+      "confidence": 0.5607577910815562,
+      "entropy_nats": 0.880503313623951,
+      "normalized_entropy": 0.8014686552354329,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.3334165192235027,
+      "stderr": 0.10983251153335577,
+      "wall_time_ms": 296,
+      "top_probabilities": {
+        "contradiction": 0.5607577910815562,
+        "entailment": 0.36797194213830603,
+        "neutral": 0.07127026678013773
+      }
+    },
+    {
+      "id": "anli-03",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "ambiguous",
+      "expected": "neutral",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.4931040534821584,
+      "entropy_nats": 0.8488686156365087,
+      "normalized_entropy": 0.7726735122047699,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.3637399560970698,
+      "tvd_gold": 0.24310405348215847,
+      "stderr": 0.12680233232887542,
+      "wall_time_ms": 288,
+      "top_probabilities": {
+        "contradiction": 0.46020555006877245,
+        "entailment": 0.4931040534821584,
+        "neutral": 0.0466903964490691
+      }
+    },
+    {
+      "id": "chaos-01",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "entailment",
+      "actual": "entailment",
+      "accurate": true,
+      "confidence": 0.9285277975297197,
+      "entropy_nats": 0.2900982731965097,
+      "normalized_entropy": 0.26405882784017204,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.008769963062816181,
+      "tvd_gold": 0.00852779752971966,
+      "stderr": 0.02699124458946807,
+      "wall_time_ms": 237,
+      "top_probabilities": {
+        "contradiction": 0.01220368238489159,
+        "entailment": 0.9285277975297197,
+        "neutral": 0.05926852008538872
+      }
+    },
+    {
+      "id": "chaos-02",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "neutral",
+      "actual": "contradiction",
+      "accurate": false,
+      "confidence": 0.8104725103463601,
+      "entropy_nats": 0.5631900896073412,
+      "normalized_entropy": 0.5126377116080856,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.3590125402087354,
+      "tvd_gold": 0.21047251034636016,
+      "stderr": 0.03045054672687464,
+      "wall_time_ms": 235,
+      "top_probabilities": {
+        "contradiction": 0.8104725103463601,
+        "entailment": 0.027033339621572543,
+        "neutral": 0.16249415003206732
+      }
+    },
+    {
+      "id": "chaos-03",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "contradiction",
+      "actual": "contradiction",
+      "accurate": true,
+      "confidence": 0.9802666901148194,
+      "entropy_nats": 0.10383696804219612,
+      "normalized_entropy": 0.09451648148600424,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.0007025663555332981,
+      "tvd_gold": 0.06026669011481933,
+      "stderr": 0.007655564600635086,
+      "wall_time_ms": 231,
+      "top_probabilities": {
+        "contradiction": 0.9802666901148194,
+        "entailment": 0.0021705170476313245,
+        "neutral": 0.017562792837549284
+      }
+    },
+    {
+      "id": "chaos-04",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "contradiction",
+      "actual": "contradiction",
+      "accurate": true,
+      "confidence": 0.5214172309586742,
+      "entropy_nats": 0.71015903238838,
+      "normalized_entropy": 0.6464146084232623,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.45528116009355657,
+      "tvd_gold": 0.12141723095867418,
+      "stderr": 0.04225973044781855,
+      "wall_time_ms": 224,
+      "top_probabilities": {
+        "contradiction": 0.5214172309586742,
+        "entailment": 0.0029452825233355824,
+        "neutral": 0.4756374865179902
+      }
+    },
+    {
+      "id": "chaos-05",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "entailment",
+      "actual": "entailment",
+      "accurate": true,
+      "confidence": 0.9957777247214509,
+      "entropy_nats": 0.030044259643613843,
+      "normalized_entropy": 0.02734746366257897,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.000027492756061972568,
+      "tvd_gold": 0.07577772472145083,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "contradiction": 0.0014982170611399536,
+        "entailment": 0.9957777247214509,
+        "neutral": 0.002724058217409208
+      }
+    },
+    {
+      "id": "chaos-06",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "neutral",
+      "actual": "neutral",
+      "accurate": true,
+      "confidence": 0.4919984466658392,
+      "entropy_nats": 1.0387990611602294,
+      "normalized_entropy": 0.945555654051172,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.3903056301107437,
+      "tvd_gold": 0.09199844666583917,
+      "stderr": 0.07733883677937524,
+      "wall_time_ms": 260,
+      "top_probabilities": {
+        "contradiction": 0.29404614360174175,
+        "entailment": 0.21395540973241903,
+        "neutral": 0.4919984466658392
+      }
+    },
+    {
+      "id": "adlab-01",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "benign",
+      "actual": "benign",
+      "accurate": true,
+      "confidence": 0.9991846174854317,
+      "entropy_nats": 0.007242435772868818,
+      "normalized_entropy": 0.005224313086737336,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.0000010429727528165665,
+      "wall_time_ms": 138,
+      "top_probabilities": {
+        "benign": 0.9991846174854317,
+        "failed_injection": 0.0001186149565662165,
+        "hijacked": 0.00010209930436891081,
+        "injection_point": 0.0005946682536331489
+      }
+    },
+    {
+      "id": "adlab-02",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "injection_point",
+      "actual": "injection_point",
+      "accurate": true,
+      "confidence": 0.9997377057061719,
+      "entropy_nats": 0.002549901408679189,
+      "normalized_entropy": 0.001839365058528624,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 1.2138348095555189e-7,
+      "wall_time_ms": 134,
+      "top_probabilities": {
+        "benign": 0.00022796397209085874,
+        "failed_injection": 0.000020928557875064575,
+        "hijacked": 0.000013401763862067968,
+        "injection_point": 0.9997377057061719
+      }
+    },
+    {
+      "id": "adlab-03",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "hijacked",
+      "actual": "hijacked",
+      "accurate": true,
+      "confidence": 0.9983417729357298,
+      "entropy_nats": 0.013684117974035253,
+      "normalized_entropy": 0.009871004570040094,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.000004148263148471669,
+      "wall_time_ms": 136,
+      "top_probabilities": {
+        "benign": 0.0011152448548011336,
+        "failed_injection": 0.00021082717091456868,
+        "hijacked": 0.9983417729357298,
+        "injection_point": 0.000332155038554618
+      }
+    },
+    {
+      "id": "adlab-04",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "benign",
+      "actual": "benign",
+      "accurate": true,
+      "confidence": 0.9972867224254285,
+      "entropy_nats": 0.019592009884953146,
+      "normalized_entropy": 0.014132647751034727,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.000013698507006463637,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "benign": 0.9972867224254285,
+        "failed_injection": 0.00007739340812919457,
+        "hijacked": 0.00012280623305212533,
+        "injection_point": 0.002513077933390178
+      }
+    },
+    {
+      "id": "b77-01",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "card_arrival",
+      "actual": "card_arrival",
+      "accurate": true,
+      "confidence": 0.8736953221097539,
+      "entropy_nats": 0.5518035104134884,
+      "normalized_entropy": 0.1693637692972037,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.023350707918393762,
+      "stderr": 0.030639699953488714,
+      "wall_time_ms": 257,
+      "top_probabilities": {
+        "activate_my_card": 0.00032295898607307604,
+        "apple_pay_or_google_pay": 0.0037543511172345324,
+        "atm_support": 0.0011428121145570683,
+        "automatic_top_up": 0.0004944711194385835,
+        "balance_not_updated_after_bank_transfer": 0.0003276482062639332,
+        "beneficiary_not_allowed": 0.00011472517367747631,
+        "cancel_transfer": 0.0000963937435687235,
+        "card_about_to_expire": 0.0004242478896395012,
+        "card_acceptance": 0.002386413331705436,
+        "card_arrival": 0.8736953221097539,
+        "card_delivery_estimate": 0.08439755962208637,
+        "card_linking": 0.0025035521356647304,
+        "card_not_working": 0.00011578786822848621,
+        "card_payment_fee_charged": 0.00017415064716260056,
+        "card_payment_not_recognised": 0.00004004384916769633,
+        "card_swallowed": 0.0003107111379212576,
+        "change_pin": 0.012414074752075347,
+        "compromised_card": 0.00025607849209894603,
+        "contactless_not_working": 0.00004215694986507805,
+        "declined_card_payment": 0.00007000775136120715,
+        "declined_transfer": 0.00017050210672329297,
+        "failed_transfer": 0.00006332308211416936,
+        "lost_or_stolen_card": 0.005880600963091758,
+        "pending_top_up": 0.0003485436677039346,
+        "transaction_charged_twice": 0.003968256520233136,
+        "transfer_not_received_by_recipient": 0.006485306662589762
+      }
+    },
+    {
+      "id": "b77-02",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "pending_top_up",
+      "actual": "balance_not_updated_after_bank_transfer",
+      "accurate": false,
+      "confidence": 0.8599195372733732,
+      "entropy_nats": 0.4886186877528481,
+      "normalized_entropy": 0.14997059849232325,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 1.5092580707087384,
+      "stderr": 0.024376508845475132,
+      "wall_time_ms": 252,
+      "top_probabilities": {
+        "activate_my_card": 0.0018922493222307166,
+        "apple_pay_or_google_pay": 0.0037512125783997173,
+        "atm_support": 0.000036693663507637324,
+        "automatic_top_up": 0.006858948806642546,
+        "balance_not_updated_after_bank_transfer": 0.8599195372733732,
+        "beneficiary_not_allowed": 0.000040537536208380474,
+        "cancel_transfer": 0.000027367689985167848,
+        "card_about_to_expire": 0.0001654702233806942,
+        "card_acceptance": 0.00004706005756655239,
+        "card_arrival": 0.00005130497013326386,
+        "card_delivery_estimate": 0.000004691910322390233,
+        "card_linking": 0.00004959171290936638,
+        "card_not_working": 0.000020761534843735262,
+        "card_payment_fee_charged": 0.00007049221042057134,
+        "card_payment_not_recognised": 0.00000604555713398153,
+        "card_swallowed": 0.00003329635723311442,
+        "change_pin": 0.0005094667786354135,
+        "compromised_card": 0.00023457451838838426,
+        "contactless_not_working": 0.000010504516362187708,
+        "declined_card_payment": 0.0000016269648565097472,
+        "declined_transfer": 0.00010845110349515247,
+        "failed_transfer": 0.000019638767285914506,
+        "lost_or_stolen_card": 0.000015277155680929743,
+        "pending_top_up": 0.12266052901930716,
+        "transaction_charged_twice": 0.0010837343729311452,
+        "transfer_not_received_by_recipient": 0.002380935398765968
+      }
+    },
+    {
+      "id": "b77-03",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "transaction_charged_twice",
+      "actual": "transaction_charged_twice",
+      "accurate": true,
+      "confidence": 0.9982775770478407,
+      "entropy_nats": 0.01585881441416879,
+      "normalized_entropy": 0.004867509059077556,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.000003843891706091908,
+      "wall_time_ms": 133,
+      "top_probabilities": {
+        "activate_my_card": 0.00015919230098774962,
+        "apple_pay_or_google_pay": 0.00006963463135278581,
+        "atm_support": 0.0000027604716967697453,
+        "automatic_top_up": 0.000026744733675634982,
+        "balance_not_updated_after_bank_transfer": 0.000029028586736362675,
+        "beneficiary_not_allowed": 3.284658090702455e-7,
+        "cancel_transfer": 0.000012563583819524437,
+        "card_about_to_expire": 0.000022674681020675698,
+        "card_acceptance": 0.000003027430931745911,
+        "card_arrival": 0.000005713582405783651,
+        "card_delivery_estimate": 0.00003702932913425739,
+        "card_linking": 0.000014902134189361473,
+        "card_not_working": 0.000017647678493567437,
+        "card_payment_fee_charged": 0.000024630650421974373,
+        "card_payment_not_recognised": 0.000001300591316249895,
+        "card_swallowed": 0.000005713582405783651,
+        "change_pin": 0.000262836450886059,
+        "compromised_card": 0.000043466747811478574,
+        "contactless_not_working": 0.000004362399240292775,
+        "declined_card_payment": 0.000011528816658641674,
+        "declined_transfer": 0.000009696282360901369,
+        "failed_transfer": 0.000001300591316249895,
+        "lost_or_stolen_card": 0.000011528816658641674,
+        "pending_top_up": 0.00006963463135278581,
+        "transaction_charged_twice": 0.9982775770478407,
+        "transfer_not_received_by_recipient": 0.0008751757814770506
+      }
+    },
+    {
+      "id": "tox-s-01",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "0.0",
+      "actual": "0.00",
+      "accurate": true,
+      "confidence": 0.9985959894313647,
+      "entropy_nats": 0.011928395176139247,
+      "normalized_entropy": 0.007411528636167669,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0000029459468027323898,
+      "wall_time_ms": 137,
+      "top_probabilities": {
+        "0.0": 0.9985959894313647,
+        "0.2": 0.0009300614250906563,
+        "0.5": 0.00004067671556487727,
+        "0.7": 0.00013246443178724352,
+        "0.9": 0.00030080799619251264
+      }
+    },
+    {
+      "id": "tox-s-02",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "ambiguous",
+      "expected": "0.2",
+      "actual": "0.20",
+      "accurate": true,
+      "confidence": 0.9369662094480434,
+      "entropy_nats": 0.30302707598046325,
+      "normalized_entropy": 0.18828130842411167,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0056447633945082,
+      "tvd_gold": 0.18696620944804343,
+      "stderr": 0.014364952187141367,
+      "wall_time_ms": 251,
+      "top_probabilities": {
+        "0.0": 0.0374997435501812,
+        "0.2": 0.9369662094480434,
+        "0.5": 0.013825830657050085,
+        "0.7": 0.0075243055388499215,
+        "0.9": 0.004183910805875336
+      }
+    },
+    {
+      "id": "tox-s-03",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "0.9",
+      "actual": "0.70",
+      "accurate": true,
+      "confidence": 0.49568518861570665,
+      "entropy_nats": 1.0251289436333515,
+      "normalized_entropy": 0.6369484251075924,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.41136014924910425,
+      "stderr": 0.06949748231606431,
+      "wall_time_ms": 250,
+      "top_probabilities": {
+        "0.0": 0.0103455492775123,
+        "0.2": 0.003437779144088106,
+        "0.5": 0.1099932319617797,
+        "0.7": 0.49568518861570665,
+        "0.9": 0.38053825100091326
+      }
+    },
+    {
+      "id": "yelp-01",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "easy",
+      "expected": "1",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.9871827314836613,
+      "entropy_nats": 0.08438874219230262,
+      "normalized_entropy": 0.05243367360762231,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.00021752747126404368,
+      "wall_time_ms": 122,
+      "top_probabilities": {
+        "1": 0.9871827314836613,
+        "2": 0.0037318050596264647,
+        "3": 0.0022368878759789212,
+        "4": 0.0010936688887814715,
+        "5": 0.0057549066919519295
+      }
+    },
+    {
+      "id": "yelp-02",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "ambiguous",
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9756716556049359,
+      "entropy_nats": 0.14657005104213977,
+      "normalized_entropy": 0.09106909307266688,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.000758715222463328,
+      "tvd_gold": 0.22567165560493585,
+      "stderr": 0.004906733220681784,
+      "wall_time_ms": 245,
+      "top_probabilities": {
+        "1": 0.008833460941675482,
+        "2": 0.007483850095982176,
+        "3": 0.9756716556049359,
+        "4": 0.0034053045226700813,
+        "5": 0.004605728834736438
+      }
+    },
+    {
+      "id": "yelp-03",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "easy",
+      "expected": "5",
+      "actual": "5",
+      "accurate": true,
+      "confidence": 0.974839126867512,
+      "entropy_nats": 0.13712520445446091,
+      "normalized_entropy": 0.08520067993618587,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0010212939645501328,
+      "stderr": 0.004502099826746304,
+      "wall_time_ms": 242,
+      "top_probabilities": {
+        "1": 0.01939385912499479,
+        "2": 0.0014924241936632163,
+        "3": 0.0027452698195343703,
+        "4": 0.0015293199942957264,
+        "5": 0.974839126867512
+      }
+    },
+    {
+      "id": "yelp-04",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "ambiguous",
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9218178968043876,
+      "entropy_nats": 0.3580828434687871,
+      "normalized_entropy": 0.22248938011359856,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.008674068238682693,
+      "tvd_gold": 0.17181789680438764,
+      "stderr": 0.016065650714407707,
+      "wall_time_ms": 240,
+      "top_probabilities": {
+        "1": 0.018459749634411465,
+        "2": 0.004849266194313592,
+        "3": 0.9218178968043876,
+        "4": 0.04603521810956634,
+        "5": 0.008837869257321081
+      }
+    },
+    {
+      "id": "sst5-01",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "1",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.8766318353803655,
+      "entropy_nats": 0.45489724729587305,
+      "normalized_entropy": 0.28264355137992886,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.02499565885824131,
+      "stderr": 0.029633292786368275,
+      "wall_time_ms": 238,
+      "top_probabilities": {
+        "0": 0.8766318353803655,
+        "1": 0.09643490062885539,
+        "2": 0.02132770444354119,
+        "3": 0.004488400071460817,
+        "4": 0.0011171594757771468
+      }
+    },
+    {
+      "id": "sst5-02",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "2",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.6117165983098558,
+      "entropy_nats": 0.9107037613444349,
+      "normalized_entropy": 0.5658520619581368,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.2531306898346025,
+      "stderr": 0.034955061710961625,
+      "wall_time_ms": 237,
+      "top_probabilities": {
+        "0": 0.02858716593597386,
+        "1": 0.6117165983098558,
+        "2": 0.31587416571568605,
+        "3": 0.042070254652850844,
+        "4": 0.0017518153856333296
+      }
+    },
+    {
+      "id": "sst5-03",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.7567541484249153,
+      "entropy_nats": 0.8026013366925023,
+      "normalized_entropy": 0.49868424901129293,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.08474339284798754,
+      "stderr": 0.02916462730755183,
+      "wall_time_ms": 203,
+      "top_probabilities": {
+        "0": 0.7567541484249153,
+        "1": 0.144005798707834,
+        "2": 0.062475782529443365,
+        "3": 0.029743237417721717,
+        "4": 0.007021032920085615
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/calibration_rollback_cloudrun.json b/benchmarks/runs/20260928-v010-verification/calibration_rollback_cloudrun.json
new file mode 100644
index 0000000..d2b61b0
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/calibration_rollback_cloudrun.json
@@ -0,0 +1,1722 @@
+{
+  "timestamp": "2026-09-28T22:44:31Z",
+  "idc_config": "none",
+  "target_url": "https://rb00032---dgemma-<hash>-uc.a.run.app/v1",
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "workers": 4,
+  "total_cases": 50,
+  "total_correct": 43,
+  "overall_accuracy_pct": 86,
+  "chance_baseline_pct": 29.205128205128204,
+  "chance_corrected_acc_pct": 80.22455632017385,
+  "avg_confidence": 0.8996163430147146,
+  "avg_entropy_nats": 0.25578431918403216,
+  "avg_normalized_entropy": 0.1879061504699149,
+  "avg_brier_score": 0.18423647760231088,
+  "ece_10bin": 0.06240700112182909,
+  "p50_latency_ms": 256.5,
+  "p95_latency_ms": 1583.5999999999997,
+  "avg_wall_time_ms": 350.5,
+  "total_elapsed_sec": 4.400891627,
+  "temperature_scale": 1,
+  "jev_parity": {
+    "protocol_version": "jevbench::v1.4",
+    "temperature_applied": 1,
+    "optimal_temperature": 1.4,
+    "chance_baseline_pct": 29.205128205128204,
+    "chance_corrected_acc_pct": 80.22455632017385,
+    "intelligence_score": 80.65411368068337,
+    "calibration_score": 86.18694383923875,
+    "speed_score": 76.57714814884943,
+    "cost_score": 57.55079956087546,
+    "composite_jevbench_score": 74.39686556997238,
+    "ece_10bin": 0.06240700112182909,
+    "brier_mean": 0.18423647760231088,
+    "soft_tvd_mean": 0.1514471209715668,
+    "probability_fidelity": 84.85528790284333,
+    "p50_latency_sec": 0.2565,
+    "p95_latency_sec": 1.5835999999999997,
+    "usd_per_1k_decisions": 0.026,
+    "presets": {
+      "Balanced 33:33:33 (no calibration)": 70.8368147440007,
+      "Emphasis on Accuracy 60:20:20": 74.61154340932886,
+      "Emphasis on Speed 20:60:20": 73.07941969298423,
+      "JevBench Score (25:25:25:25)": 74.39686556997238
+    },
+    "raw_t1_ece_10bin": 0.06240700112182909,
+    "raw_t1_brier_mean": 0.1842364776023109,
+    "raw_t1_calibration_score": 86.18694383923875,
+    "raw_t1_composite_score": 74.39686556997238,
+    "reliability_bins": [
+      {
+        "bin_index": 0,
+        "range_label": "[0.0, 0.1)",
+        "lower": 0,
+        "upper": 0.1,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 1,
+        "range_label": "[0.1, 0.2)",
+        "lower": 0.1,
+        "upper": 0.2,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 2,
+        "range_label": "[0.2, 0.3)",
+        "lower": 0.2,
+        "upper": 0.30000000000000004,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 3,
+        "range_label": "[0.3, 0.4)",
+        "lower": 0.30000000000000004,
+        "upper": 0.4,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 4,
+        "range_label": "[0.4, 0.5)",
+        "lower": 0.4,
+        "upper": 0.5,
+        "count": 3,
+        "avg_confidence": 0.4839098572054685,
+        "empirical_accuracy": 0.3333333333333333,
+        "abs_gap": 0.15057652387213516
+      },
+      {
+        "bin_index": 5,
+        "range_label": "[0.5, 0.6)",
+        "lower": 0.5,
+        "upper": 0.6000000000000001,
+        "count": 3,
+        "avg_confidence": 0.5494605973297064,
+        "empirical_accuracy": 0.6666666666666666,
+        "abs_gap": 0.1172060693369602
+      },
+      {
+        "bin_index": 6,
+        "range_label": "[0.6, 0.7)",
+        "lower": 0.6000000000000001,
+        "upper": 0.7000000000000001,
+        "count": 2,
+        "avg_confidence": 0.6462708297789808,
+        "empirical_accuracy": 0,
+        "abs_gap": 0.6462708297789808
+      },
+      {
+        "bin_index": 7,
+        "range_label": "[0.7, 0.8)",
+        "lower": 0.7000000000000001,
+        "upper": 0.8,
+        "count": 1,
+        "avg_confidence": 0.7818517553330173,
+        "empirical_accuracy": 1,
+        "abs_gap": 0.21814824466698268
+      },
+      {
+        "bin_index": 8,
+        "range_label": "[0.8, 0.9)",
+        "lower": 0.8,
+        "upper": 0.9,
+        "count": 3,
+        "avg_confidence": 0.819945283840691,
+        "empirical_accuracy": 0.6666666666666666,
+        "abs_gap": 0.15327861717402436
+      },
+      {
+        "bin_index": 9,
+        "range_label": "[0.9, 1.0]",
+        "lower": 0.9,
+        "upper": 1,
+        "count": 38,
+        "avg_confidence": 0.982802013703083,
+        "empirical_accuracy": 0.9736842105263158,
+        "abs_gap": 0.009117803176767136
+      }
+    ],
+    "endpoint_kind": "gpu",
+    "raw_speed_score": 83.91267137471998,
+    "adjusted_speed_score": 76.57714814884943,
+    "v14_intelligence_score": 80.65411368068337,
+    "v14_calibration_score": 86.18694383923875,
+    "v14_composite_score": 73.48080049564528,
+    "v14_public_only_estimate": true
+  },
+  "categories": [
+    {
+      "name": "agent-trajectory",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 35.714285714285715,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9983821049462025,
+      "avg_entropy_nats": 0.011947456412756666,
+      "avg_normalized_entropy": 0.009955031855163837,
+      "avg_brier_score": 0.0000084492638230801,
+      "ece_10bin": 0.0016178950537975023,
+      "avg_latency_ms": 139.71428571428572
+    },
+    {
+      "name": "emotion",
+      "total": 4,
+      "correct": 4,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 3.8461538461538463,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.8412931074806923,
+      "avg_entropy_nats": 0.4925468066108808,
+      "avg_normalized_entropy": 0.15117624688616063,
+      "avg_brier_score": 0.08222979279604789,
+      "ece_10bin": 0.15870689251930772,
+      "avg_latency_ms": 225.25
+    },
+    {
+      "name": "grounding",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.992956632149754,
+      "avg_entropy_nats": 0.04115855590086716,
+      "avg_normalized_entropy": 0.059379244488332227,
+      "avg_brier_score": 0.00011983114629954005,
+      "ece_10bin": 0.007043367850245952,
+      "avg_latency_ms": 144.5
+    },
+    {
+      "name": "guardrail",
+      "total": 4,
+      "correct": 4,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9880539760977142,
+      "avg_entropy_nats": 0.05874075562173359,
+      "avg_normalized_entropy": 0.08474499683354554,
+      "avg_brier_score": 0.0005665199508879864,
+      "ece_10bin": 0.01194602390228583,
+      "avg_latency_ms": 164.75
+    },
+    {
+      "name": "intent",
+      "total": 7,
+      "correct": 6,
+      "accuracy_pct": 85.71428571428571,
+      "chance_baseline_pct": 6.41025641025641,
+      "chance_corrected_pct": 84.7358121330724,
+      "avg_confidence": 0.9579373108216476,
+      "avg_entropy_nats": 0.16162972183388527,
+      "avg_normalized_entropy": 0.05104685063712149,
+      "avg_brier_score": 0.1869081339010492,
+      "ece_10bin": 0.12847350248772538,
+      "avg_latency_ms": 588
+    },
+    {
+      "name": "nli",
+      "total": 3,
+      "correct": 0,
+      "accuracy_pct": 0,
+      "chance_baseline_pct": 33.33333333333333,
+      "avg_confidence": 0.6941222404294631,
+      "avg_entropy_nats": 0.5863576800349173,
+      "avg_normalized_entropy": 0.5337257612016898,
+      "avg_brier_score": 1.3800935562180168,
+      "ece_10bin": 0.6941222404294631,
+      "avg_latency_ms": 258
+    },
+    {
+      "name": "nli-soft",
+      "total": 6,
+      "correct": 3,
+      "accuracy_pct": 50,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 25,
+      "avg_confidence": 0.7742619437957153,
+      "avg_entropy_nats": 0.466066443802557,
+      "avg_normalized_entropy": 0.4242319593635599,
+      "avg_brier_score": 0.39699064895992225,
+      "ece_10bin": 0.2984824701885328,
+      "avg_latency_ms": 222.5
+    },
+    {
+      "name": "ordinal",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 20,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.857839422689125,
+      "avg_entropy_nats": 0.408018030538385,
+      "avg_normalized_entropy": 0.25351585630370915,
+      "avg_brier_score": 0.0827327749362394,
+      "ece_10bin": 0.14216057731087492,
+      "avg_latency_ms": 250
+    },
+    {
+      "name": "reading-comprehension",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.976809581564192,
+      "avg_entropy_nats": 0.1082265541852433,
+      "avg_normalized_entropy": 0.1561379130155512,
+      "avg_brier_score": 0.0012502330270751703,
+      "ece_10bin": 0.023190418435808047,
+      "avg_latency_ms": 233.5
+    },
+    {
+      "name": "retrieval",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9974346115174637,
+      "avg_entropy_nats": 0.017591043945036453,
+      "avg_normalized_entropy": 0.025378511863563918,
+      "avg_brier_score": 0.000015877883382358973,
+      "ece_10bin": 0.0025653884825362905,
+      "avg_latency_ms": 386
+    },
+    {
+      "name": "toxicity",
+      "total": 6,
+      "correct": 6,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 35,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.8836628187829637,
+      "avg_entropy_nats": 0.2705792062989288,
+      "avg_normalized_entropy": 0.2702729997539394,
+      "avg_brier_score": 0.07801611264882063,
+      "ece_10bin": 0.1163371812170363,
+      "avg_latency_ms": 914
+    }
+  ],
+  "tiers": [
+    {
+      "name": "adversarial",
+      "total": 5,
+      "correct": 3,
+      "accuracy_pct": 60,
+      "chance_baseline_pct": 43.33333333333333,
+      "chance_corrected_pct": 29.41176470588236,
+      "avg_confidence": 0.9120929116345009,
+      "avg_entropy_nats": 0.2143044308732991,
+      "avg_normalized_entropy": 0.20813623529171502,
+      "avg_brier_score": 0.5465555359776595,
+      "ece_10bin": 0.3120929116345008,
+      "avg_latency_ms": 183.6
+    },
+    {
+      "name": "ambiguous",
+      "total": 9,
+      "correct": 8,
+      "accuracy_pct": 88.88888888888889,
+      "chance_baseline_pct": 27.891737891737893,
+      "chance_corrected_pct": 84.59107072303436,
+      "avg_confidence": 0.853515393527588,
+      "avg_entropy_nats": 0.32771626059979275,
+      "avg_normalized_entropy": 0.272530541605062,
+      "avg_brier_score": 0.2112829229289998,
+      "ece_10bin": 0.14647160310069363,
+      "avg_latency_ms": 436.1111111111111
+    },
+    {
+      "name": "easy",
+      "total": 17,
+      "correct": 17,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 31.138763197586727,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9574448568893885,
+      "avg_entropy_nats": 0.14933878175897672,
+      "avg_normalized_entropy": 0.08042977173212734,
+      "avg_brier_score": 0.018764259225777357,
+      "ece_10bin": 0.04255514311061137,
+      "avg_latency_ms": 345.47058823529414
+    },
+    {
+      "name": "held-out",
+      "total": 8,
+      "correct": 7,
+      "accuracy_pct": 87.5,
+      "chance_baseline_pct": 21.442307692307693,
+      "chance_corrected_pct": 84.08812729498165,
+      "avg_confidence": 0.8534413688476314,
+      "avg_entropy_nats": 0.41882841759900685,
+      "avg_normalized_entropy": 0.24213358472194257,
+      "avg_brier_score": 0.23456280668092314,
+      "ece_10bin": 0.22216809279806976,
+      "avg_latency_ms": 594.25
+    },
+    {
+      "name": "high-entropy",
+      "total": 3,
+      "correct": 0,
+      "accuracy_pct": 0,
+      "chance_baseline_pct": 33.33333333333333,
+      "avg_confidence": 0.5727444139842481,
+      "avg_entropy_nats": 0.8185706725987929,
+      "avg_normalized_entropy": 0.7450951359657355,
+      "avg_brier_score": 0.7919485248832993,
+      "ece_10bin": 0.572744413984248,
+      "avg_latency_ms": 264.6666666666667
+    },
+    {
+      "name": "localization",
+      "total": 3,
+      "correct": 3,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 25,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9972236214047028,
+      "avg_entropy_nats": 0.019753988331774473,
+      "avg_normalized_entropy": 0.01424949050201474,
+      "avg_brier_score": 0.00001774694725933654,
+      "ece_10bin": 0.002776378595297224,
+      "avg_latency_ms": 142.33333333333334
+    },
+    {
+      "name": "low-entropy",
+      "total": 3,
+      "correct": 3,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9757794736071824,
+      "avg_entropy_nats": 0.11356221500632102,
+      "avg_normalized_entropy": 0.10336878276138432,
+      "avg_brier_score": 0.0020327730365452133,
+      "ece_10bin": 0.024220526392817576,
+      "avg_latency_ms": 180.33333333333334
+    },
+    {
+      "name": "out-of-scope",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 8.333333333333332,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9986890029629409,
+      "avg_entropy_nats": 0.0116001004658256,
+      "avg_normalized_entropy": 0.0046682238412518486,
+      "avg_brier_score": 0.0000039494896696894295,
+      "ece_10bin": 0.0013109970370590585,
+      "avg_latency_ms": 146.5
+    }
+  ],
+  "cases": [
+    {
+      "id": "tox-01",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.999603530904076,
+      "entropy_nats": 0.003501898213968182,
+      "normalized_entropy": 0.0050521711869898145,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 3.1437548804550195e-7,
+      "wall_time_ms": 637,
+      "top_probabilities": {
+        "no": 0.999603530904076,
+        "yes": 0.00039646909592399324
+      }
+    },
+    {
+      "id": "tox-02",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9905867683480455,
+      "entropy_nats": 0.05328752781877272,
+      "normalized_entropy": 0.07687765212537609,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00017721786026671655,
+      "wall_time_ms": 641,
+      "top_probabilities": {
+        "no": 0.009413231651954544,
+        "yes": 0.9905867683480455
+      }
+    },
+    {
+      "id": "tox-03",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "ambiguous",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.5430758251422115,
+      "entropy_nats": 0.6894315228140827,
+      "normalized_entropy": 0.9946394389964034,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.4175594031389417,
+      "tvd_gold": 0.20692417485778847,
+      "stderr": 0.05686187625344647,
+      "wall_time_ms": 773,
+      "top_probabilities": {
+        "no": 0.5430758251422115,
+        "yes": 0.45692417485778847
+      }
+    },
+    {
+      "id": "marco-01",
+      "metric": "calibration/ms-marco-passage-relevance-check",
+      "category": "retrieval",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.996269397535034,
+      "entropy_nats": 0.024582125654191964,
+      "normalized_entropy": 0.035464510775812114,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00002783478950322135,
+      "wall_time_ms": 638,
+      "top_probabilities": {
+        "no": 0.996269397535034,
+        "yes": 0.0037306024649659936
+      }
+    },
+    {
+      "id": "marco-02",
+      "metric": "calibration/ms-marco-passage-relevance-check",
+      "category": "retrieval",
+      "tier": "ambiguous",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9985998254998936,
+      "entropy_nats": 0.01059996223588094,
+      "normalized_entropy": 0.015292512951315722,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.000003920977261496597,
+      "tvd_gold": 0.24859982549989357,
+      "wall_time_ms": 134,
+      "top_probabilities": {
+        "no": 0.001400174500106488,
+        "yes": 0.9985998254998936
+      }
+    },
+    {
+      "id": "pi-01",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9952820796705182,
+      "entropy_nats": 0.029977781390436845,
+      "normalized_entropy": 0.04324879654883669,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00004451754447067564,
+      "wall_time_ms": 135,
+      "top_probabilities": {
+        "no": 0.9952820796705182,
+        "yes": 0.004717920329481862
+      }
+    },
+    {
+      "id": "pi-02",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9686407547332957,
+      "entropy_nats": 0.13943574841243897,
+      "normalized_entropy": 0.20116326275726687,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0019668045273946327,
+      "stderr": 0.006317284294785691,
+      "wall_time_ms": 266,
+      "top_probabilities": {
+        "no": 0.031359245266704326,
+        "yes": 0.9686407547332957
+      }
+    },
+    {
+      "id": "pi-03",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "adversarial",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9887219262205692,
+      "entropy_nats": 0.06179521044193594,
+      "normalized_entropy": 0.08915164365527088,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00025438989634856893,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "no": 0.011278073779430872,
+        "yes": 0.9887219262205692
+      }
+    },
+    {
+      "id": "pi-04",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "adversarial",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9995711437664737,
+      "entropy_nats": 0.0037542822421225744,
+      "normalized_entropy": 0.005416284372807737,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 3.678353380687062e-7,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "no": 0.9995711437664737,
+        "yes": 0.00042885623352627663
+      }
+    },
+    {
+      "id": "ad-01",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9997502379630008,
+      "entropy_nats": 0.00232150742666158,
+      "normalized_entropy": 0.0033492272518315606,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 1.247621502520032e-7,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "no": 0.9997502379630008,
+        "yes": 0.000249762036999275
+      }
+    },
+    {
+      "id": "ad-02",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9994521372509538,
+      "entropy_nats": 0.004661880157412946,
+      "normalized_entropy": 0.0067256713843183175,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 6.0030718358485e-7,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "no": 0.0005478627490460652,
+        "yes": 0.9994521372509538
+      }
+    },
+    {
+      "id": "ad-03",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "ambiguous",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9992718668761021,
+      "entropy_nats": 0.005988649205706392,
+      "normalized_entropy": 0.008639794510696241,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0000010603556922345013,
+      "tvd_gold": 0.24927186687610214,
+      "wall_time_ms": 144,
+      "top_probabilities": {
+        "no": 0.9992718668761021,
+        "yes": 0.0007281331238978986
+      }
+    },
+    {
+      "id": "ground-01",
+      "metric": "calibration/grounding-claim-support-check",
+      "category": "grounding",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9961670125751695,
+      "entropy_nats": 0.02515279874955398,
+      "normalized_entropy": 0.03628781802045965,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00002938358519781771,
+      "wall_time_ms": 146,
+      "top_probabilities": {
+        "no": 0.003832987424830589,
+        "yes": 0.9961670125751695
+      }
+    },
+    {
+      "id": "ground-02",
+      "metric": "calibration/grounding-claim-support-check",
+      "category": "grounding",
+      "tier": "adversarial",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9897462517243385,
+      "entropy_nats": 0.05716431305218033,
+      "normalized_entropy": 0.0824706709562048,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0002102787074012624,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "no": 0.9897462517243385,
+        "yes": 0.010253748275661544
+      }
+    },
+    {
+      "id": "boolq-01",
+      "metric": "calibration/boolq-passage-answer-check",
+      "category": "reading-comprehension",
+      "tier": "held-out",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9674650106104392,
+      "entropy_nats": 0.14344651865397712,
+      "normalized_entropy": 0.206949581094879,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.002117051069157666,
+      "stderr": 0.00881078874493946,
+      "wall_time_ms": 306,
+      "top_probabilities": {
+        "no": 0.03253498938956083,
+        "yes": 0.9674650106104392
+      }
+    },
+    {
+      "id": "boolq-02",
+      "metric": "calibration/boolq-passage-answer-check",
+      "category": "reading-comprehension",
+      "tier": "held-out",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9861541525179447,
+      "entropy_nats": 0.07300658971650946,
+      "normalized_entropy": 0.10532624493622339,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00038341498499267455,
+      "wall_time_ms": 161,
+      "top_probabilities": {
+        "no": 0.9861541525179447,
+        "yes": 0.013845847482055326
+      }
+    },
+    {
+      "id": "emo-01",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "easy",
+      "expected": "sadness",
+      "actual": "sadness",
+      "accurate": true,
+      "confidence": 0.5567140154591986,
+      "entropy_nats": 1.2997987695393374,
+      "normalized_entropy": 0.39894421616145714,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.2712318874218151,
+      "stderr": 0.04829114340483047,
+      "wall_time_ms": 312,
+      "top_probabilities": {
+        "admiration": 0.003863648547346874,
+        "amusement": 0.0009298680994586277,
+        "anger": 0.004757255338525102,
+        "annoyance": 0.02455389264537816,
+        "approval": 0.0005034308410147159,
+        "caring": 0.00012083933819327354,
+        "confusion": 0.0005403647346845219,
+        "curiosity": 0.0002422603881778873,
+        "desire": 0.00226518126472781,
+        "disappointment": 0.24464226257031496,
+        "disapproval": 0.005102268138510665,
+        "disgust": 0.0005746069584692752,
+        "embarrassment": 0.002629733847789108,
+        "excitement": 0.0003079981571561152,
+        "fear": 0.00005894765354049132,
+        "gratitude": 0.0002889022584828125,
+        "joy": 0.0016203546103288088,
+        "love": 0.0002434959100901411,
+        "neutral": 0.11681029880442587,
+        "optimism": 0.0002744200760443097,
+        "pride": 0.00012000876278848971,
+        "realization": 0.02204640613907582,
+        "relief": 0.0009305327713848863,
+        "remorse": 0.0020897522758352768,
+        "sadness": 0.5567140154591986,
+        "surprise": 0.007769254409057319
+      }
+    },
+    {
+      "id": "emo-02",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "easy",
+      "expected": "gratitude",
+      "actual": "gratitude",
+      "accurate": true,
+      "confidence": 0.9935808468349622,
+      "entropy_nats": 0.0539748972017073,
+      "normalized_entropy": 0.016566389783675407,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.00004902327696199549,
+      "wall_time_ms": 158,
+      "top_probabilities": {
+        "admiration": 0.0003259291260444824,
+        "amusement": 0.00014256942325992084,
+        "anger": 0.00005475696763666317,
+        "annoyance": 0.00010429746385619359,
+        "approval": 0.000059432911306541246,
+        "caring": 0.0005381296397323062,
+        "confusion": 0.000059432911306541246,
+        "curiosity": 0.000039295794933173,
+        "desire": 0.00017948366280982696,
+        "disappointment": 0.0003505431914940118,
+        "disapproval": 0.0003505431914940118,
+        "disgust": 0.00024263072570846083,
+        "embarrassment": 0.00008899349427257374,
+        "excitement": 0.00026136444982524524,
+        "fear": 0.000008931543567268075,
+        "gratitude": 0.9935808468349622,
+        "joy": 0.002604406051781115,
+        "love": 0.00016628932756294754,
+        "neutral": 0.00012203846467382124,
+        "optimism": 0.00002802003544408255,
+        "pride": 0.000039295794933173,
+        "realization": 0.00016628932756294754,
+        "relief": 0.00002802003544408255,
+        "remorse": 0.00013193113489807219,
+        "sadness": 0.000023604012974015047,
+        "surprise": 0.00030292448251637244
+      }
+    },
+    {
+      "id": "emo-03",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "ambiguous",
+      "expected": "confusion",
+      "actual": "confusion",
+      "accurate": true,
+      "confidence": 0.8183265308443377,
+      "entropy_nats": 0.587367423189482,
+      "normalized_entropy": 0.1802793184103034,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.05762296227249292,
+      "tvd_gold": 0.06832653084433771,
+      "stderr": 0.0226065558375463,
+      "wall_time_ms": 280,
+      "top_probabilities": {
+        "admiration": 0.00031201336645658256,
+        "amusement": 0.00016819427793963058,
+        "anger": 0.00016612994737731337,
+        "annoyance": 0.0008506233036001925,
+        "approval": 0.0000636436397053849,
+        "caring": 0.000017860983748246846,
+        "confusion": 0.8183265308443377,
+        "curiosity": 0.15624983036286413,
+        "desire": 0.0000628822755671801,
+        "disappointment": 0.00021259422077172485,
+        "disapproval": 0.00029720037515610915,
+        "disgust": 0.00013447596155840032,
+        "embarrassment": 0.000122507347924854,
+        "excitement": 0.0002909703835432198,
+        "fear": 0.00001655731680226958,
+        "gratitude": 0.00004587489752681778,
+        "joy": 0.000872607965821522,
+        "love": 0.00004423037358397411,
+        "neutral": 0.0014902140769218859,
+        "optimism": 0.000006097212649010586,
+        "pride": 0.000023836102749464502,
+        "realization": 0.009711718862712666,
+        "relief": 0.000011226420161284383,
+        "remorse": 0.000038390434167253185,
+        "sadness": 0.00020619368529415543,
+        "surprise": 0.010257595361059182
+      }
+    },
+    {
+      "id": "emo-04",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "ambiguous",
+      "expected": "neutral",
+      "actual": "neutral",
+      "accurate": true,
+      "confidence": 0.9965510367842706,
+      "entropy_nats": 0.029046136512996496,
+      "normalized_entropy": 0.008915063189206514,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.000015298212921548033,
+      "tvd_gold": 0.2465510367842706,
+      "wall_time_ms": 151,
+      "top_probabilities": {
+        "admiration": 0.00016233877433478804,
+        "amusement": 0.0000604728040204754,
+        "anger": 0.000017796868405033847,
+        "annoyance": 0.00015082192572535982,
+        "approval": 0.000022893861617611083,
+        "caring": 0.000011603018111945162,
+        "confusion": 0.000003990538973487187,
+        "curiosity": 0.0000025152369333336036,
+        "desire": 0.000008933495959438576,
+        "disappointment": 0.0001300270427146568,
+        "disapproval": 0.00005164055113311996,
+        "disgust": 0.000008933495959438576,
+        "embarrassment": 0.000017796868405033847,
+        "excitement": 0.000037477368587355875,
+        "fear": 0.0000013603266625392822,
+        "gratitude": 0.000012649510873974782,
+        "joy": 0.0009010597875624436,
+        "love": 0.000010638839422380298,
+        "neutral": 0.9965510367842706,
+        "optimism": 0.0000010187035168439836,
+        "pride": 0.0000027606459548748397,
+        "realization": 0.00010377698796430396,
+        "relief": 0.000006268988822641805,
+        "remorse": 0.000005732041366714246,
+        "sadness": 0.00014006654096379706,
+        "surprise": 0.0015763889917379148
+      }
+    },
+    {
+      "id": "clinc-01",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "easy",
+      "expected": "bill_balance",
+      "actual": "bill_balance",
+      "accurate": true,
+      "confidence": 0.9921758063597531,
+      "entropy_nats": 0.05747690049061435,
+      "normalized_entropy": 0.023130406325532586,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.0000793056451751561,
+      "wall_time_ms": 166,
+      "top_probabilities": {
+        "account_blocked": 0.00025598775792707017,
+        "balance": 0.002934413228139043,
+        "bill_balance": 0.9921758063597531,
+        "bill_due": 0.00042265200910695687,
+        "freeze_account": 0.00036695367506345516,
+        "interest_rate": 0.0000506450134068994,
+        "min_payment": 0.00003086328172932912,
+        "out_of_scope": 0.0006847024699036769,
+        "pay_bill": 0.002934413228139043,
+        "pin_change": 0.0000070149171396877865,
+        "routing": 0.000015592020257223046,
+        "transfer": 0.000120956039434576
+      }
+    },
+    {
+      "id": "clinc-02",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "easy",
+      "expected": "freeze_account",
+      "actual": "freeze_account",
+      "accurate": true,
+      "confidence": 0.9970411625224643,
+      "entropy_nats": 0.024739720249407935,
+      "normalized_entropy": 0.009955995832486747,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.000011785829319950281,
+      "wall_time_ms": 157,
+      "top_probabilities": {
+        "account_blocked": 0.0016559373671697361,
+        "balance": 0.0001248774974476478,
+        "bill_balance": 0.00003739104124191147,
+        "bill_due": 0.00035318391318167,
+        "freeze_account": 0.9970411625224643,
+        "interest_rate": 0.0001912558966412881,
+        "min_payment": 0.000015382879533291836,
+        "out_of_scope": 0.0002197750699270813,
+        "pay_bill": 0.0002354557133513482,
+        "pin_change": 0.00005947305729214067,
+        "routing": 0.000011004011659260586,
+        "transfer": 0.00005510103009041807
+      }
+    },
+    {
+      "id": "clinc-03",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "out-of-scope",
+      "expected": "out_of_scope",
+      "actual": "out_of_scope",
+      "accurate": true,
+      "confidence": 0.99988153312181,
+      "entropy_nats": 0.0014002857703297616,
+      "normalized_entropy": 0.0005635164485753326,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 1.7510553610582403e-8,
+      "wall_time_ms": 152,
+      "top_probabilities": {
+        "account_blocked": 0.000010876403938896142,
+        "balance": 0.000009162257492381226,
+        "bill_balance": 0.0000018218292127771747,
+        "bill_due": 0.0000041518157881399695,
+        "freeze_account": 0.000023069790611954582,
+        "interest_rate": 7.744863741019137e-7,
+        "min_payment": 0.0000034669148020643304,
+        "out_of_scope": 0.99988153312181,
+        "pay_bill": 0.00005121108137539739,
+        "pin_change": 0.000002637617980714948,
+        "routing": 0.0000028904157617426906,
+        "transfer": 0.000008404264851783911
+      }
+    },
+    {
+      "id": "clinc-04",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "out-of-scope",
+      "expected": "out_of_scope",
+      "actual": "out_of_scope",
+      "accurate": true,
+      "confidence": 0.9974964728040717,
+      "entropy_nats": 0.02179991516132144,
+      "normalized_entropy": 0.008772931233928364,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.000007881468785768277,
+      "wall_time_ms": 141,
+      "top_probabilities": {
+        "account_blocked": 0.000341318301373438,
+        "balance": 0.0002947224401148965,
+        "bill_balance": 0.00006752697211655357,
+        "bill_due": 0.00004861596479553374,
+        "freeze_account": 0.000034777015725727086,
+        "interest_rate": 0.0011161998938467208,
+        "min_payment": 0.00003783775020181543,
+        "out_of_scope": 0.9974964728040717,
+        "pay_bill": 0.00036709437747641183,
+        "pin_change": 0.000005394602331199548,
+        "routing": 0.00006223914460151692,
+        "transfer": 0.00012780073334460896
+      }
+    },
+    {
+      "id": "anli-01",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "adversarial",
+      "expected": "contradiction",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.9784409490235777,
+      "entropy_nats": 0.11828351190703601,
+      "normalized_entropy": 0.10766629240096678,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.9305470700640996,
+      "stderr": 0.011241960853727778,
+      "wall_time_ms": 261,
+      "top_probabilities": {
+        "contradiction": 0.013523538106027588,
+        "entailment": 0.9784409490235777,
+        "neutral": 0.00803551287039472
+      }
+    },
+    {
+      "id": "anli-02",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "adversarial",
+      "expected": "contradiction",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.6039842874375452,
+      "entropy_nats": 0.8305248367232205,
+      "normalized_entropy": 0.7559762850733247,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.8017655733851096,
+      "stderr": 0.07467930355369624,
+      "wall_time_ms": 256,
+      "top_probabilities": {
+        "contradiction": 0.34123759113995283,
+        "entailment": 0.6039842874375452,
+        "neutral": 0.05477812142250202
+      }
+    },
+    {
+      "id": "anli-03",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "ambiguous",
+      "expected": "neutral",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.4999414848272666,
+      "entropy_nats": 0.8102646914744953,
+      "normalized_entropy": 0.7375347061307778,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.4079680252048405,
+      "tvd_gold": 0.24994148482726652,
+      "stderr": 0.07575376379514209,
+      "wall_time_ms": 257,
+      "top_probabilities": {
+        "contradiction": 0.4687280097864463,
+        "entailment": 0.4999414848272666,
+        "neutral": 0.031330505386287184
+      }
+    },
+    {
+      "id": "chaos-01",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "entailment",
+      "actual": "entailment",
+      "accurate": true,
+      "confidence": 0.9404383791903109,
+      "entropy_nats": 0.2558656682329654,
+      "normalized_entropy": 0.23289896797273343,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.005944378581700041,
+      "tvd_gold": 0.020438379190310885,
+      "stderr": 0.020649209708185477,
+      "wall_time_ms": 279,
+      "top_probabilities": {
+        "contradiction": 0.012131468044797995,
+        "entailment": 0.9404383791903109,
+        "neutral": 0.04743015276489102
+      }
+    },
+    {
+      "id": "chaos-02",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "neutral",
+      "actual": "contradiction",
+      "accurate": false,
+      "confidence": 0.6885573721204162,
+      "entropy_nats": 0.6932911268385317,
+      "normalized_entropy": 0.6310607791207538,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.9759088711227821,
+      "tvd_gold": 0.08855737212041626,
+      "stderr": 0.0673335605650376,
+      "wall_time_ms": 264,
+      "top_probabilities": {
+        "contradiction": 0.6885573721204162,
+        "entailment": 0.019549562616775557,
+        "neutral": 0.2918930652628082
+      }
+    },
+    {
+      "id": "chaos-03",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "contradiction",
+      "actual": "contradiction",
+      "accurate": true,
+      "confidence": 0.9920306453592204,
+      "entropy_nats": 0.04974942284378782,
+      "normalized_entropy": 0.045283876174460946,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.00011124847412144016,
+      "tvd_gold": 0.07203064535922032,
+      "wall_time_ms": 134,
+      "top_probabilities": {
+        "contradiction": 0.9920306453592204,
+        "entailment": 0.001157792636177653,
+        "neutral": 0.006811562004601959
+      }
+    },
+    {
+      "id": "chaos-04",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "contradiction",
+      "actual": "neutral",
+      "accurate": false,
+      "confidence": 0.548591951387709,
+      "entropy_nats": 0.7132854782429909,
+      "normalized_entropy": 0.649260422080054,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.6068001685851863,
+      "tvd_gold": 0.05140804861229098,
+      "stderr": 0.1338217797948031,
+      "wall_time_ms": 264,
+      "top_probabilities": {
+        "contradiction": 0.446983308450538,
+        "entailment": 0.0044247401617530345,
+        "neutral": 0.548591951387709
+      }
+    },
+    {
+      "id": "chaos-05",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "entailment",
+      "actual": "entailment",
+      "accurate": true,
+      "confidence": 0.9948693962720156,
+      "entropy_nats": 0.035071553942209864,
+      "normalized_entropy": 0.03192350413695852,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.000042692053814158896,
+      "tvd_gold": 0.07486939627201561,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "contradiction": 0.0012989267430764811,
+        "entailment": 0.9948693962720156,
+        "neutral": 0.0038316769849078753
+      }
+    },
+    {
+      "id": "chaos-06",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "neutral",
+      "actual": "contradiction",
+      "accurate": false,
+      "confidence": 0.4810839184446191,
+      "entropy_nats": 1.0491354127148562,
+      "normalized_entropy": 0.9549642066963987,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.7931365349419296,
+      "tvd_gold": 0.1189160815553808,
+      "stderr": 0.07657021269851898,
+      "wall_time_ms": 266,
+      "top_probabilities": {
+        "contradiction": 0.4810839184446191,
+        "entailment": 0.23167299716031398,
+        "neutral": 0.2872430843950669
+      }
+    },
+    {
+      "id": "adlab-01",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "benign",
+      "actual": "benign",
+      "accurate": true,
+      "confidence": 0.9985296283192525,
+      "entropy_nats": 0.011398193104192319,
+      "normalized_entropy": 0.008222058333256519,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.0000041185799574797254,
+      "wall_time_ms": 136,
+      "top_probabilities": {
+        "benign": 0.9985296283192525,
+        "failed_injection": 0.000040601604086735745,
+        "hijacked": 0.000031943843342235255,
+        "injection_point": 0.001397826233318596
+      }
+    },
+    {
+      "id": "adlab-02",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "injection_point",
+      "actual": "injection_point",
+      "accurate": true,
+      "confidence": 0.9958349765453671,
+      "entropy_nats": 0.0296533160886261,
+      "normalized_entropy": 0.021390346033486895,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.00002814782032877084,
+      "wall_time_ms": 136,
+      "top_probabilities": {
+        "benign": 0.0008777645413739519,
+        "failed_injection": 0.00012262752046442288,
+        "hijacked": 0.0031646313927944565,
+        "injection_point": 0.9958349765453671
+      }
+    },
+    {
+      "id": "adlab-03",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "hijacked",
+      "actual": "hijacked",
+      "accurate": true,
+      "confidence": 0.9993610531097705,
+      "entropy_nats": 0.005853979004551287,
+      "normalized_entropy": 0.0042227532396671265,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 6.296110711873936e-7,
+      "wall_time_ms": 136,
+      "top_probabilities": {
+        "benign": 0.00045079909190560096,
+        "failed_injection": 0.0000792697716373108,
+        "hijacked": 0.9993610531097705,
+        "injection_point": 0.00010887802668643065
+      }
+    },
+    {
+      "id": "adlab-04",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "benign",
+      "actual": "benign",
+      "accurate": true,
+      "confidence": 0.9964748345589709,
+      "entropy_nats": 0.023754669902146038,
+      "normalized_entropy": 0.017135372232890203,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.000024463410378051388,
+      "wall_time_ms": 155,
+      "top_probabilities": {
+        "benign": 0.9964748345589709,
+        "failed_injection": 0.000025698560844473802,
+        "hijacked": 0.000030311392914671825,
+        "injection_point": 0.003469155487270007
+      }
+    },
+    {
+      "id": "b77-01",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "card_arrival",
+      "actual": "card_arrival",
+      "accurate": true,
+      "confidence": 0.9170519038125,
+      "entropy_nats": 0.4738202047475712,
+      "normalized_entropy": 0.14542853448882279,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.007811978600389612,
+      "stderr": 0.0202244392554557,
+      "wall_time_ms": 1674,
+      "top_probabilities": {
+        "activate_my_card": 0.006910802776928644,
+        "apple_pay_or_google_pay": 0.013468252598225415,
+        "atm_support": 0.0010498578522876408,
+        "automatic_top_up": 0.0004857381116680427,
+        "balance_not_updated_after_bank_transfer": 0.0003693529440327282,
+        "beneficiary_not_allowed": 0.00009892338316612952,
+        "cancel_transfer": 0.00004839238196187783,
+        "card_about_to_expire": 0.0019020314635402268,
+        "card_acceptance": 0.002338938382223116,
+        "card_arrival": 0.9170519038125,
+        "card_delivery_estimate": 0.011989378939814228,
+        "card_linking": 0.003284284357996513,
+        "card_not_working": 0.00012007678271782777,
+        "card_payment_fee_charged": 0.00010626702964442937,
+        "card_payment_not_recognised": 0.000008473523352016031,
+        "card_swallowed": 0.0005478520507732529,
+        "change_pin": 0.010008709981012902,
+        "compromised_card": 0.0003682137225494589,
+        "contactless_not_working": 0.0000262675105968246,
+        "declined_card_payment": 0.00004958603474700358,
+        "declined_transfer": 0.00008156386031669231,
+        "failed_transfer": 0.000035348377012165364,
+        "lost_or_stolen_card": 0.019835193423351744,
+        "pending_top_up": 0.0005942591310049187,
+        "transaction_charged_twice": 0.005051756455220969,
+        "transfer_not_received_by_recipient": 0.004168575113355222
+      }
+    },
+    {
+      "id": "b77-02",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "pending_top_up",
+      "actual": "balance_not_updated_after_bank_transfer",
+      "accurate": false,
+      "confidence": 0.8024378465828051,
+      "entropy_nats": 0.5465331761281561,
+      "normalized_entropy": 0.16774615784099658,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 1.3004456449002857,
+      "stderr": 0.05813215198824891,
+      "wall_time_ms": 1674,
+      "top_probabilities": {
+        "activate_my_card": 0.0017943892439284176,
+        "apple_pay_or_google_pay": 0.0012295785203637044,
+        "atm_support": 0.00003766146379385546,
+        "automatic_top_up": 0.0014447986329640863,
+        "balance_not_updated_after_bank_transfer": 0.8024378465828051,
+        "beneficiary_not_allowed": 0.00006938867637680768,
+        "cancel_transfer": 0.0000034122272092098287,
+        "card_about_to_expire": 0.00012387691628991016,
+        "card_acceptance": 0.00020052581335962537,
+        "card_arrival": 0.00006145754038809266,
+        "card_delivery_estimate": 0.0000017181622988174517,
+        "card_linking": 0.0000607724476192111,
+        "card_not_working": 0.00000967565395630653,
+        "card_payment_fee_charged": 0.00004671531385644435,
+        "card_payment_not_recognised": 0.000004319836562274193,
+        "card_swallowed": 0.00004254492099127072,
+        "change_pin": 0.0005260000994745347,
+        "compromised_card": 0.00002474540944372492,
+        "contactless_not_working": 0.000008149349386068,
+        "declined_card_payment": 0.000001215203937770363,
+        "declined_transfer": 0.00007265304637905222,
+        "failed_transfer": 0.000022604819389715017,
+        "lost_or_stolen_card": 0.000003733126843527091,
+        "pending_top_up": 0.18973470829454628,
+        "transaction_charged_twice": 0.0011814361236635656,
+        "transfer_not_received_by_recipient": 0.0008560725741725752
+      }
+    },
+    {
+      "id": "b77-03",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "transaction_charged_twice",
+      "actual": "transaction_charged_twice",
+      "accurate": true,
+      "confidence": 0.9994764505481281,
+      "entropy_nats": 0.005637850289795937,
+      "normalized_entropy": 0.0017304122895080293,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 3.2335283471540864e-7,
+      "wall_time_ms": 152,
+      "top_probabilities": {
+        "activate_my_card": 0.00005916282836697445,
+        "apple_pay_or_google_pay": 0.00008753310619198938,
+        "atm_support": 0.000017203413480593647,
+        "automatic_top_up": 0.000020389268662188153,
+        "balance_not_updated_after_bank_transfer": 0.000006002139297350655,
+        "beneficiary_not_allowed": 2.0159326778744708e-8,
+        "cancel_transfer": 0.0000014158046146499655,
+        "card_about_to_expire": 0.000010234803397099067,
+        "card_acceptance": 0.0000017148727088672099,
+        "card_arrival": 0.0000023892626918263637,
+        "card_delivery_estimate": 0.000018732490847493638,
+        "card_linking": 0.000020389268662188153,
+        "card_not_working": 0.0000017148727088672099,
+        "card_payment_fee_charged": 0.000012188557576536062,
+        "card_payment_not_recognised": 1.4778439379678795e-7,
+        "card_swallowed": 0.000006002139297350655,
+        "change_pin": 0.000036489923419344065,
+        "compromised_card": 0.000013293184407081935,
+        "contactless_not_working": 0.0000011670940762911372,
+        "declined_card_payment": 0.0000093731034923204,
+        "declined_transfer": 6.807281354988156e-7,
+        "failed_transfer": 1.1367226658201004e-7,
+        "lost_or_stolen_card": 0.0000026244295738708103,
+        "pending_top_up": 0.000008580515224901193,
+        "transaction_charged_twice": 0.9994764505481281,
+        "transfer_not_received_by_recipient": 0.00018598602905138653
+      }
+    },
+    {
+      "id": "tox-s-01",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "0.0",
+      "actual": "0.00",
+      "accurate": true,
+      "confidence": 0.9999343391586882,
+      "entropy_nats": 0.0007670834206563609,
+      "normalized_entropy": 0.0004766157269752832,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 6.194895110387991e-9,
+      "wall_time_ms": 1511,
+      "top_probabilities": {
+        "0.0": 0.9999343391586882,
+        "0.2": 0.000014229279179689865,
+        "0.5": 0.000005601602326963978,
+        "0.7": 0.000005601602326963978,
+        "0.9": 0.000040228357478111133
+      }
+    },
+    {
+      "id": "tox-s-02",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "ambiguous",
+      "expected": "0.2",
+      "actual": "0.20",
+      "accurate": true,
+      "confidence": 0.929704975049831,
+      "entropy_nats": 0.33523935537596655,
+      "normalized_entropy": 0.20829592293433263,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0066702150375897015,
+      "tvd_gold": 0.17970497504983096,
+      "stderr": 0.01329563163842451,
+      "wall_time_ms": 1643,
+      "top_probabilities": {
+        "0.0": 0.01088323711688546,
+        "0.2": 0.929704975049831,
+        "0.5": 0.02771034297280521,
+        "0.7": 0.028889620848261133,
+        "0.9": 0.0028118240122172724
+      }
+    },
+    {
+      "id": "tox-s-03",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "0.9",
+      "actual": "0.90",
+      "accurate": true,
+      "confidence": 0.8390714740949301,
+      "entropy_nats": 0.5412478501501264,
+      "normalized_entropy": 0.3362961975535594,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.043689519285742565,
+      "stderr": 0.03383439382065462,
+      "wall_time_ms": 279,
+      "top_probabilities": {
+        "0.0": 0.004384514765753097,
+        "0.2": 0.004203644595310875,
+        "0.5": 0.020713711157523986,
+        "0.7": 0.13162665538648197,
+        "0.9": 0.8390714740949301
+      }
+    },
+    {
+      "id": "yelp-01",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "easy",
+      "expected": "1",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.9681749382627072,
+      "entropy_nats": 0.17999781673029747,
+      "normalized_entropy": 0.11183893167899238,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0013440096519159474,
+      "stderr": 0.0032925313369453557,
+      "wall_time_ms": 269,
+      "top_probabilities": {
+        "1": 0.9681749382627072,
+        "2": 0.00950404208945825,
+        "3": 0.005569543515736342,
+        "4": 0.0024798712926457516,
+        "5": 0.014271604839452515
+      }
+    },
+    {
+      "id": "yelp-02",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "ambiguous",
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9810789285238869,
+      "entropy_nats": 0.11673278996411096,
+      "normalized_entropy": 0.0725301604133118,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.00047619257684378255,
+      "tvd_gold": 0.2310789285238869,
+      "stderr": 0.0038191514526292308,
+      "wall_time_ms": 275,
+      "top_probabilities": {
+        "1": 0.004749957820679615,
+        "2": 0.008685513537670074,
+        "3": 0.9810789285238869,
+        "4": 0.004345841231514676,
+        "5": 0.0011397588862487684
+      }
+    },
+    {
+      "id": "yelp-03",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "easy",
+      "expected": "5",
+      "actual": "5",
+      "accurate": true,
+      "confidence": 0.9855884371275552,
+      "entropy_nats": 0.08643679119282786,
+      "normalized_entropy": 0.05370619799933853,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.00033595320077688297,
+      "wall_time_ms": 151,
+      "top_probabilities": {
+        "1": 0.011134987494039517,
+        "2": 0.0007522258334161645,
+        "3": 0.00177212371157294,
+        "4": 0.0007522258334161645,
+        "5": 0.9855884371275552
+      }
+    },
+    {
+      "id": "yelp-04",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "ambiguous",
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9150880682004906,
+      "entropy_nats": 0.3647758146254135,
+      "normalized_entropy": 0.22664795690921039,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.011229228584414518,
+      "tvd_gold": 0.1650880682004906,
+      "stderr": 0.022946644733461342,
+      "wall_time_ms": 268,
+      "top_probabilities": {
+        "1": 0.010187979433042501,
+        "2": 0.005499886442643184,
+        "3": 0.9150880682004906,
+        "4": 0.06189905455796525,
+        "5": 0.007325011365858444
+      }
+    },
+    {
+      "id": "sst5-01",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "1",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.7818517553330173,
+      "entropy_nats": 0.7152676315197738,
+      "normalized_entropy": 0.4444207670229472,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.06925888884564488,
+      "stderr": 0.03821193547244103,
+      "wall_time_ms": 269,
+      "top_probabilities": {
+        "0": 0.7818517553330173,
+        "1": 0.11040516107676127,
+        "2": 0.09691071580593012,
+        "3": 0.009326149879766859,
+        "4": 0.0015062179045242392
+      }
+    },
+    {
+      "id": "sst5-02",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "2",
+      "actual": "2",
+      "accurate": true,
+      "confidence": 0.47070416834451967,
+      "entropy_nats": 0.970219003636212,
+      "normalized_entropy": 0.6028309611327977,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.48329468957245697,
+      "stderr": 0.05639679843327608,
+      "wall_time_ms": 259,
+      "top_probabilities": {
+        "0": 0.01734400728346235,
+        "1": 0.44560483704100856,
+        "2": 0.47070416834451967,
+        "3": 0.06538502434666987,
+        "4": 0.0009619629843396245
+      }
+    },
+    {
+      "id": "sst5-03",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.9023896630316983,
+      "entropy_nats": 0.42269636610005945,
+      "normalized_entropy": 0.26263601896936617,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.013190462121622809,
+      "stderr": 0.019384184289031495,
+      "wall_time_ms": 259,
+      "top_probabilities": {
+        "0": 0.9023896630316983,
+        "1": 0.04300232324471003,
+        "2": 0.040761509492907425,
+        "3": 0.012220465452055974,
+        "4": 0.0016260387786282754
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/calibration_rollback_vertex.json b/benchmarks/runs/20260928-v010-verification/calibration_rollback_vertex.json
new file mode 100644
index 0000000..cc0e3a3
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/calibration_rollback_vertex.json
@@ -0,0 +1,1722 @@
+{
+  "timestamp": "2026-09-28T23:02:24Z",
+  "idc_config": "none",
+  "target_url": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "workers": 4,
+  "total_cases": 50,
+  "total_correct": 44,
+  "overall_accuracy_pct": 88,
+  "chance_baseline_pct": 29.205128205128204,
+  "chance_corrected_acc_pct": 83.04961970300616,
+  "avg_confidence": 0.9012386289534349,
+  "avg_entropy_nats": 0.2610803713068672,
+  "avg_normalized_entropy": 0.19780956946065176,
+  "avg_brier_score": 0.16646681142921985,
+  "ece_10bin": 0.042632924441280036,
+  "p50_latency_ms": 217.5,
+  "p95_latency_ms": 1272.2999999999997,
+  "avg_wall_time_ms": 281.74,
+  "total_elapsed_sec": 3.541350543,
+  "temperature_scale": 1,
+  "jev_parity": {
+    "protocol_version": "jevbench::v1.4",
+    "temperature_applied": 1,
+    "optimal_temperature": 1.05,
+    "chance_baseline_pct": 29.205128205128204,
+    "chance_corrected_acc_pct": 83.04961970300616,
+    "intelligence_score": 84.10326086956523,
+    "calibration_score": 88.06244546559911,
+    "speed_score": 78.02349828471462,
+    "cost_score": 57.55079956087546,
+    "composite_jevbench_score": 75.93991192665607,
+    "ece_10bin": 0.042632924441280036,
+    "brier_mean": 0.16646681142921985,
+    "soft_tvd_mean": 0.1534852418054576,
+    "probability_fidelity": 84.65147581945423,
+    "p50_latency_sec": 0.2175,
+    "p95_latency_sec": 1.2722999999999998,
+    "usd_per_1k_decisions": 0.026,
+    "presets": {
+      "Balanced 33:33:33 (no calibration)": 72.28195188869739,
+      "Emphasis on Accuracy 60:20:20": 76.79678811975292,
+      "Emphasis on Speed 20:60:20": 74.52604774928582,
+      "JevBench Score (25:25:25:25)": 75.93991192665607
+    },
+    "raw_t1_ece_10bin": 0.042632924441279856,
+    "raw_t1_brier_mean": 0.16646681142921987,
+    "raw_t1_calibration_score": 88.06244546559913,
+    "raw_t1_composite_score": 75.93991192665607,
+    "reliability_bins": [
+      {
+        "bin_index": 0,
+        "range_label": "[0.0, 0.1)",
+        "lower": 0,
+        "upper": 0.1,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 1,
+        "range_label": "[0.1, 0.2)",
+        "lower": 0.1,
+        "upper": 0.2,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 2,
+        "range_label": "[0.2, 0.3)",
+        "lower": 0.2,
+        "upper": 0.30000000000000004,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 3,
+        "range_label": "[0.3, 0.4)",
+        "lower": 0.30000000000000004,
+        "upper": 0.4,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 4,
+        "range_label": "[0.4, 0.5)",
+        "lower": 0.4,
+        "upper": 0.5,
+        "count": 2,
+        "avg_confidence": 0.4546656732409018,
+        "empirical_accuracy": 0.5,
+        "abs_gap": 0.04533432675909821
+      },
+      {
+        "bin_index": 5,
+        "range_label": "[0.5, 0.6)",
+        "lower": 0.5,
+        "upper": 0.6000000000000001,
+        "count": 2,
+        "avg_confidence": 0.5250592571743169,
+        "empirical_accuracy": 0,
+        "abs_gap": 0.5250592571743169
+      },
+      {
+        "bin_index": 6,
+        "range_label": "[0.6, 0.7)",
+        "lower": 0.6000000000000001,
+        "upper": 0.7000000000000001,
+        "count": 4,
+        "avg_confidence": 0.6389528165805164,
+        "empirical_accuracy": 0.75,
+        "abs_gap": 0.11104718341948361
+      },
+      {
+        "bin_index": 7,
+        "range_label": "[0.7, 0.8)",
+        "lower": 0.7000000000000001,
+        "upper": 0.8,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 8,
+        "range_label": "[0.8, 0.9)",
+        "lower": 0.8,
+        "upper": 0.9,
+        "count": 7,
+        "avg_confidence": 0.8635524170401269,
+        "empirical_accuracy": 0.8571428571428571,
+        "abs_gap": 0.006409559897269768
+      },
+      {
+        "bin_index": 9,
+        "range_label": "[0.9, 1.0]",
+        "lower": 0.9,
+        "upper": 1,
+        "count": 35,
+        "avg_confidence": 0.9857658114639528,
+        "empirical_accuracy": 0.9714285714285714,
+        "abs_gap": 0.014337240035381393
+      }
+    ],
+    "endpoint_kind": "gpu",
+    "raw_speed_score": 85.57951211529972,
+    "adjusted_speed_score": 78.02349828471462,
+    "v14_intelligence_score": 84.10326086956523,
+    "v14_calibration_score": 88.06244546559911,
+    "v14_composite_score": 74.85264045954096,
+    "v14_public_only_estimate": true
+  },
+  "categories": [
+    {
+      "name": "agent-trajectory",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 35.714285714285715,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9983233010994851,
+      "avg_entropy_nats": 0.011631151279087525,
+      "avg_normalized_entropy": 0.00977585141156707,
+      "avg_brier_score": 0.000016479841831062998,
+      "ece_10bin": 0.0016766989005149169,
+      "avg_latency_ms": 132.85714285714286
+    },
+    {
+      "name": "emotion",
+      "total": 4,
+      "correct": 4,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 3.8461538461538463,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.8755425792166093,
+      "avg_entropy_nats": 0.39911207503507323,
+      "avg_normalized_entropy": 0.12249854182572467,
+      "avg_brier_score": 0.06196654901893351,
+      "ece_10bin": 0.12445742078339067,
+      "avg_latency_ms": 188
+    },
+    {
+      "name": "grounding",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9957368771082851,
+      "avg_entropy_nats": 0.027398915533923474,
+      "avg_normalized_entropy": 0.039528279566526985,
+      "avg_brier_score": 0.00003840330340598425,
+      "ece_10bin": 0.004263122891714932,
+      "avg_latency_ms": 118
+    },
+    {
+      "name": "guardrail",
+      "total": 4,
+      "correct": 4,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9915797593126893,
+      "avg_entropy_nats": 0.04759192752610088,
+      "avg_normalized_entropy": 0.0686606378282527,
+      "avg_brier_score": 0.00017896404164000328,
+      "ece_10bin": 0.008420240687310732,
+      "avg_latency_ms": 131.75
+    },
+    {
+      "name": "intent",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 6.41025641025641,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9340748364621153,
+      "avg_entropy_nats": 0.1827513732235672,
+      "avg_normalized_entropy": 0.058208235750029404,
+      "avg_brier_score": 0.03761355299746211,
+      "ece_10bin": 0.06592516353788481,
+      "avg_latency_ms": 496.42857142857144
+    },
+    {
+      "name": "nli",
+      "total": 3,
+      "correct": 0,
+      "accuracy_pct": 0,
+      "chance_baseline_pct": 33.33333333333333,
+      "avg_confidence": 0.7619640439216643,
+      "avg_entropy_nats": 0.5432337130736384,
+      "avg_normalized_entropy": 0.49447263486577403,
+      "avg_brier_score": 1.5477611471757635,
+      "ece_10bin": 0.7619640439216642,
+      "avg_latency_ms": 373.3333333333333
+    },
+    {
+      "name": "nli-soft",
+      "total": 6,
+      "correct": 4,
+      "accuracy_pct": 66.66666666666666,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 49.99999999999999,
+      "avg_confidence": 0.7659075478126708,
+      "avg_entropy_nats": 0.4859144345782284,
+      "avg_normalized_entropy": 0.4422983791373037,
+      "avg_brier_score": 0.36251976174148454,
+      "ece_10bin": 0.30395523686661624,
+      "avg_latency_ms": 196.5
+    },
+    {
+      "name": "ordinal",
+      "total": 7,
+      "correct": 7,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 20,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.867590944862126,
+      "avg_entropy_nats": 0.42816384851212214,
+      "avg_normalized_entropy": 0.266033156796071,
+      "avg_brier_score": 0.053417108825566835,
+      "ece_10bin": 0.13240905513787407,
+      "avg_latency_ms": 215.42857142857142
+    },
+    {
+      "name": "reading-comprehension",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9170355522039488,
+      "avg_entropy_nats": 0.2807972032803834,
+      "avg_normalized_entropy": 0.4051047326680993,
+      "avg_brier_score": 0.01530686239809184,
+      "ece_10bin": 0.08296444779605117,
+      "avg_latency_ms": 243.5
+    },
+    {
+      "name": "retrieval",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9923122196138539,
+      "avg_entropy_nats": 0.0448630268637043,
+      "avg_normalized_entropy": 0.06472366637553455,
+      "avg_brier_score": 0.00012485950235544463,
+      "ece_10bin": 0.007687780386146148,
+      "avg_latency_ms": 176.5
+    },
+    {
+      "name": "toxicity",
+      "total": 6,
+      "correct": 5,
+      "accuracy_pct": 83.33333333333334,
+      "chance_baseline_pct": 35,
+      "chance_corrected_pct": 74.35897435897438,
+      "avg_confidence": 0.8836686333492109,
+      "avg_entropy_nats": 0.2763453176581527,
+      "avg_normalized_entropy": 0.2719665850119228,
+      "avg_brier_score": 0.09801437753012927,
+      "ece_10bin": 0.1250777323817323,
+      "avg_latency_ms": 586.6666666666666
+    }
+  ],
+  "tiers": [
+    {
+      "name": "adversarial",
+      "total": 5,
+      "correct": 3,
+      "accuracy_pct": 60,
+      "chance_baseline_pct": 43.33333333333333,
+      "chance_corrected_pct": 29.41176470588236,
+      "avg_confidence": 0.9551692336288375,
+      "avg_entropy_nats": 0.17571077683559025,
+      "avg_normalized_entropy": 0.17378619185286923,
+      "avg_brier_score": 0.6644765302926308,
+      "ece_10bin": 0.3551692336288375,
+      "avg_latency_ms": 226.8
+    },
+    {
+      "name": "ambiguous",
+      "total": 9,
+      "correct": 7,
+      "accuracy_pct": 77.77777777777779,
+      "chance_baseline_pct": 27.891737891737893,
+      "chance_corrected_pct": 69.18214144606874,
+      "avg_confidence": 0.8516213527241607,
+      "avg_entropy_nats": 0.3434531006242799,
+      "avg_normalized_entropy": 0.2943661765893796,
+      "avg_brier_score": 0.21428550206363783,
+      "ece_10bin": 0.15129268942472834,
+      "avg_latency_ms": 347.44444444444446
+    },
+    {
+      "name": "easy",
+      "total": 17,
+      "correct": 17,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 31.138763197586727,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9641309575066733,
+      "avg_entropy_nats": 0.1309783630245505,
+      "avg_normalized_entropy": 0.06869786772153201,
+      "avg_brier_score": 0.015061467445066992,
+      "ece_10bin": 0.03586904249332657,
+      "avg_latency_ms": 225.94117647058823
+    },
+    {
+      "name": "held-out",
+      "total": 8,
+      "correct": 8,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 21.442307692307693,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.8316051396060368,
+      "avg_entropy_nats": 0.4778898422487655,
+      "avg_normalized_entropy": 0.3104101736651608,
+      "avg_brier_score": 0.08013000800605301,
+      "ece_10bin": 0.16839486039396317,
+      "avg_latency_ms": 501.875
+    },
+    {
+      "name": "high-entropy",
+      "total": 3,
+      "correct": 1,
+      "accuracy_pct": 33.33333333333333,
+      "chance_baseline_pct": 33.33333333333333,
+      "avg_confidence": 0.544015202680831,
+      "avg_entropy_nats": 0.9056731375852346,
+      "avg_normalized_entropy": 0.8243792163322855,
+      "avg_brier_score": 0.724585105234821,
+      "ece_10bin": 0.5957103666777432,
+      "avg_latency_ms": 237.33333333333334
+    },
+    {
+      "name": "localization",
+      "total": 3,
+      "correct": 3,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 25,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9968871493047816,
+      "avg_entropy_nats": 0.0202427427751157,
+      "avg_normalized_entropy": 0.014602052307825157,
+      "avg_brier_score": 0.000037488115975763685,
+      "ece_10bin": 0.003112850695218383,
+      "avg_latency_ms": 132.66666666666666
+    },
+    {
+      "name": "low-entropy",
+      "total": 3,
+      "correct": 3,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9877998929445108,
+      "avg_entropy_nats": 0.06615573157122205,
+      "avg_normalized_entropy": 0.060217541942321824,
+      "avg_brier_score": 0.0004544182481480536,
+      "ece_10bin": 0.01220010705548924,
+      "avg_latency_ms": 155.66666666666666
+    },
+    {
+      "name": "out-of-scope",
+      "total": 2,
+      "correct": 2,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 8.333333333333332,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9961594878789946,
+      "avg_entropy_nats": 0.02921051517234423,
+      "avg_normalized_entropy": 0.011755176064596359,
+      "avg_brier_score": 0.00003617800685114195,
+      "ece_10bin": 0.003840512121005446,
+      "avg_latency_ms": 196.5
+    }
+  ],
+  "cases": [
+    {
+      "id": "tox-01",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9995253818210019,
+      "entropy_nats": 0.004106758411534803,
+      "normalized_entropy": 0.005924799994450297,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 4.5052483167090006e-7,
+      "wall_time_ms": 208,
+      "top_probabilities": {
+        "no": 0.9995253818210019,
+        "yes": 0.00047461817899813993
+      }
+    },
+    {
+      "id": "tox-02",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9940260394431708,
+      "entropy_nats": 0.03654482084786397,
+      "normalized_entropy": 0.05272303180738896,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00007137640946910195,
+      "wall_time_ms": 218,
+      "top_probabilities": {
+        "no": 0.005973960556829179,
+        "yes": 0.9940260394431708
+      }
+    },
+    {
+      "id": "tox-03",
+      "metric": "calibration/civil-comments-toxicity-check",
+      "category": "toxicity",
+      "tier": "ambiguous",
+      "expected": "false",
+      "actual": "true",
+      "accurate": false,
+      "confidence": 0.5262390971928295,
+      "entropy_nats": 0.6917695673946511,
+      "normalized_entropy": 0.9980125243181668,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.5538551748286484,
+      "tvd_gold": 0.27623909719282946,
+      "stderr": 0.04141516426599375,
+      "wall_time_ms": 315,
+      "top_probabilities": {
+        "no": 0.47376090280717054,
+        "yes": 0.5262390971928295
+      }
+    },
+    {
+      "id": "marco-01",
+      "metric": "calibration/ms-marco-passage-relevance-check",
+      "category": "retrieval",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9941364410675455,
+      "entropy_nats": 0.03597915532673065,
+      "normalized_entropy": 0.05190694896524804,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00006876264670873376,
+      "wall_time_ms": 217,
+      "top_probabilities": {
+        "no": 0.9941364410675455,
+        "yes": 0.005863558932454567
+      }
+    },
+    {
+      "id": "marco-02",
+      "metric": "calibration/ms-marco-passage-relevance-check",
+      "category": "retrieval",
+      "tier": "ambiguous",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9904879981601622,
+      "entropy_nats": 0.05374689840067796,
+      "normalized_entropy": 0.07754038378582105,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00018095635800215547,
+      "tvd_gold": 0.2404879981601622,
+      "wall_time_ms": 136,
+      "top_probabilities": {
+        "no": 0.009512001839837712,
+        "yes": 0.9904879981601622
+      }
+    },
+    {
+      "id": "pi-01",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.993510021320329,
+      "entropy_nats": 0.0391621149157013,
+      "normalized_entropy": 0.056498988979605966,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00008423964652516955,
+      "wall_time_ms": 127,
+      "top_probabilities": {
+        "no": 0.993510021320329,
+        "yes": 0.00648997867967109
+      }
+    },
+    {
+      "id": "pi-02",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9927313888234878,
+      "entropy_nats": 0.04303415334930483,
+      "normalized_entropy": 0.06208515962589725,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.000105665416870637,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "no": 0.0072686111765122895,
+        "yes": 0.9927313888234878
+      }
+    },
+    {
+      "id": "pi-03",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "adversarial",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9843588011970996,
+      "entropy_nats": 0.08055194236360685,
+      "normalized_entropy": 0.11621188778194921,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0004892941999837045,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "no": 0.01564119880290038,
+        "yes": 0.9843588011970996
+      }
+    },
+    {
+      "id": "pi-04",
+      "metric": "calibration/prompt-injection-check",
+      "category": "guardrail",
+      "tier": "adversarial",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9957188259098407,
+      "entropy_nats": 0.027619499475790548,
+      "normalized_entropy": 0.03984651492555835,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00003665690318050206,
+      "wall_time_ms": 150,
+      "top_probabilities": {
+        "no": 0.9957188259098407,
+        "yes": 0.004281174090159298
+      }
+    },
+    {
+      "id": "ad-01",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9996260776152701,
+      "entropy_nats": 0.003324646872831339,
+      "normalized_entropy": 0.004796451556140773,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 2.7963589960416915e-7,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "no": 0.9996260776152701,
+        "yes": 0.00037392238472985144
+      }
+    },
+    {
+      "id": "ad-02",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9992295231318423,
+      "entropy_nats": 0.006293344117071385,
+      "normalized_entropy": 0.00907937634830662,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.0000011872692087321242,
+      "wall_time_ms": 149,
+      "top_probabilities": {
+        "no": 0.0007704768681577319,
+        "yes": 0.9992295231318423
+      }
+    },
+    {
+      "id": "ad-03",
+      "metric": "calibration/agent-step-drift-check",
+      "category": "agent-trajectory",
+      "tier": "ambiguous",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9995614415495073,
+      "entropy_nats": 0.0038294038654940444,
+      "normalized_entropy": 0.005524661966309285,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 3.8466702899707476e-7,
+      "tvd_gold": 0.24956144154950732,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "no": 0.9995614415495073,
+        "yes": 0.00043855845049265335
+      }
+    },
+    {
+      "id": "ground-01",
+      "metric": "calibration/grounding-claim-support-check",
+      "category": "grounding",
+      "tier": "easy",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9947232524671439,
+      "entropy_nats": 0.032936415108233204,
+      "normalized_entropy": 0.04751720274130838,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.00005568812905100627,
+      "wall_time_ms": 131,
+      "top_probabilities": {
+        "no": 0.0052767475328561525,
+        "yes": 0.9947232524671439
+      }
+    },
+    {
+      "id": "ground-02",
+      "metric": "calibration/grounding-claim-support-check",
+      "category": "grounding",
+      "tier": "adversarial",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.9967505017494264,
+      "entropy_nats": 0.021861415959613745,
+      "normalized_entropy": 0.03153935639174559,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.000021118477760962224,
+      "wall_time_ms": 105,
+      "top_probabilities": {
+        "no": 0.9967505017494264,
+        "yes": 0.0032494982505736902
+      }
+    },
+    {
+      "id": "boolq-01",
+      "metric": "calibration/boolq-passage-answer-check",
+      "category": "reading-comprehension",
+      "tier": "held-out",
+      "expected": "true",
+      "actual": "true",
+      "accurate": true,
+      "confidence": 0.9447904004460528,
+      "entropy_nats": 0.21357784524808265,
+      "normalized_entropy": 0.3081276981831593,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.006096199765814418,
+      "stderr": 0.023746262243989253,
+      "wall_time_ms": 218,
+      "top_probabilities": {
+        "no": 0.05520959955394728,
+        "yes": 0.9447904004460528
+      }
+    },
+    {
+      "id": "boolq-02",
+      "metric": "calibration/boolq-passage-answer-check",
+      "category": "reading-comprehension",
+      "tier": "held-out",
+      "expected": "false",
+      "actual": "false",
+      "accurate": true,
+      "confidence": 0.8892807039618449,
+      "entropy_nats": 0.34801656131268416,
+      "normalized_entropy": 0.5020817671530393,
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "brier_score": 0.02451752503036926,
+      "stderr": 0.019378595564169755,
+      "wall_time_ms": 269,
+      "top_probabilities": {
+        "no": 0.8892807039618449,
+        "yes": 0.11071929603815511
+      }
+    },
+    {
+      "id": "emo-01",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "easy",
+      "expected": "sadness",
+      "actual": "sadness",
+      "accurate": true,
+      "confidence": 0.6139673646770131,
+      "entropy_nats": 1.1144588004536038,
+      "normalized_entropy": 0.3420582501003399,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.22934030708008157,
+      "stderr": 0.020576329160432755,
+      "wall_time_ms": 245,
+      "top_probabilities": {
+        "admiration": 0.0038495046112992813,
+        "amusement": 0.0004624809684721372,
+        "anger": 0.0023874723396776986,
+        "annoyance": 0.02968847642811872,
+        "approval": 0.0013033406513016188,
+        "caring": 0.00018350294078705394,
+        "confusion": 0.00039749421880871347,
+        "curiosity": 0.0002112010974641474,
+        "desire": 0.0009379402055579616,
+        "disappointment": 0.2795913282306054,
+        "disapproval": 0.005032138118979996,
+        "disgust": 0.00035583490443750007,
+        "embarrassment": 0.0007746659477020428,
+        "excitement": 0.00020594873036214697,
+        "fear": 0.00013131148010320608,
+        "gratitude": 0.0005517268603396997,
+        "joy": 0.0009343175607713725,
+        "love": 0.00026721881888317037,
+        "neutral": 0.030829569848888853,
+        "optimism": 0.00041912055661803794,
+        "pride": 0.00019902724381097098,
+        "realization": 0.00963199684023234,
+        "relief": 0.0022731911244359376,
+        "remorse": 0.0028040153752885498,
+        "sadness": 0.6139673646770131,
+        "surprise": 0.012609810220040533
+      }
+    },
+    {
+      "id": "emo-02",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "easy",
+      "expected": "gratitude",
+      "actual": "gratitude",
+      "accurate": true,
+      "confidence": 0.9962320685855027,
+      "entropy_nats": 0.03238595169888208,
+      "normalized_entropy": 0.009940144903916455,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.00001844282750764401,
+      "wall_time_ms": 116,
+      "top_probabilities": {
+        "admiration": 0.00014221122256854803,
+        "amusement": 0.00013091794386498637,
+        "anger": 0.00003298879750441875,
+        "annoyance": 0.00005598391212939384,
+        "approval": 0.0000301632619363648,
+        "caring": 0.00006656450738025829,
+        "confusion": 0.00005598391212939384,
+        "curiosity": 0.00002756874526794968,
+        "desire": 0.00005598391212939384,
+        "disappointment": 0.00023168422379227905,
+        "disapproval": 0.00015441741115732136,
+        "disgust": 0.00014221122256854803,
+        "embarrassment": 0.0000301632619363648,
+        "excitement": 0.00010189328706593374,
+        "fear": 0.000005360641881004924,
+        "gratitude": 0.9962320685855027,
+        "joy": 0.0020038766145960177,
+        "love": 0.00005131144323582338,
+        "neutral": 0.00008604120011621586,
+        "optimism": 0.00002756874526794968,
+        "pride": 0.000020999093450988857,
+        "realization": 0.000036064600451303214,
+        "relief": 0.000013235729751135723,
+        "remorse": 0.000020999093450988857,
+        "sadness": 0.000012054407072434053,
+        "surprise": 0.00023168422379227905
+      }
+    },
+    {
+      "id": "emo-03",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "ambiguous",
+      "expected": "confusion",
+      "actual": "confusion",
+      "accurate": true,
+      "confidence": 0.89496155934371,
+      "entropy_nats": 0.4233962479472836,
+      "normalized_entropy": 0.129952026591697,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.018496403165811918,
+      "tvd_gold": 0.14496155934371002,
+      "stderr": 0.02927380725117882,
+      "wall_time_ms": 263,
+      "top_probabilities": {
+        "admiration": 0.0004983319679122382,
+        "amusement": 0.0002461008866821308,
+        "anger": 0.00048358315342307047,
+        "annoyance": 0.0007726251681304246,
+        "approval": 0.00007964442370470922,
+        "caring": 0.00003219675523228632,
+        "confusion": 0.89496155934371,
+        "curiosity": 0.08582442026540257,
+        "desire": 0.00010761079138763832,
+        "disappointment": 0.0001902384186847547,
+        "disapproval": 0.0004276298591431129,
+        "disgust": 0.00017727615340570852,
+        "embarrassment": 0.00020348586123494894,
+        "excitement": 0.0003259389355790639,
+        "fear": 0.00003374322877568429,
+        "gratitude": 0.0001657800066051002,
+        "joy": 0.0008711073739913534,
+        "love": 0.00016472178699021578,
+        "neutral": 0.0017038299812063922,
+        "optimism": 0.00002549114887436558,
+        "pride": 0.000060217019710739425,
+        "realization": 0.0031855837156315736,
+        "relief": 0.00001863101839666032,
+        "remorse": 0.000052993889419832535,
+        "sadness": 0.00033427526001658463,
+        "surprise": 0.009052983586748855
+      }
+    },
+    {
+      "id": "emo-04",
+      "metric": "calibration/go-emotions-emotion-select",
+      "category": "emotion",
+      "tier": "ambiguous",
+      "expected": "neutral",
+      "actual": "neutral",
+      "accurate": true,
+      "confidence": 0.9970093242602115,
+      "entropy_nats": 0.02620730004052366,
+      "normalized_entropy": 0.008043745706945306,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.000011043002332910824,
+      "tvd_gold": 0.2470093242602115,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "admiration": 0.0003039697855726223,
+        "amusement": 0.00007607518171188137,
+        "anger": 0.000008962363690192242,
+        "annoyance": 0.00015453525599108545,
+        "approval": 0.00005060262365920139,
+        "caring": 0.000014023727407477505,
+        "confusion": 0.00001073290957304064,
+        "curiosity": 0.000004709632892219742,
+        "desire": 0.000014023727407477505,
+        "disappointment": 0.00003943139313309465,
+        "disapproval": 0.00003943139313309465,
+        "disgust": 0.000011738312032871145,
+        "embarrassment": 0.00003625640484393637,
+        "excitement": 0.000021725967452542363,
+        "fear": 0.0000032326816081741955,
+        "gratitude": 0.00004286740635655555,
+        "joy": 0.0006655479279823337,
+        "love": 0.00001283276643780416,
+        "neutral": 0.9970093242602115,
+        "optimism": 0.0000016485323115756103,
+        "pride": 0.0000014235259270220793,
+        "realization": 0.00007607518171188137,
+        "relief": 0.000008184977556311788,
+        "remorse": 0.0000051691468189519765,
+        "sadness": 0.00016686314299337875,
+        "surprise": 0.0012206117715838204
+      }
+    },
+    {
+      "id": "clinc-01",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "easy",
+      "expected": "bill_balance",
+      "actual": "bill_balance",
+      "accurate": true,
+      "confidence": 0.9902509018744544,
+      "entropy_nats": 0.07035048634387353,
+      "normalized_entropy": 0.02831111838743539,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.0001199146467711555,
+      "wall_time_ms": 115,
+      "top_probabilities": {
+        "account_blocked": 0.00047919223423824423,
+        "balance": 0.0015423086703241146,
+        "bill_balance": 0.9902509018744544,
+        "bill_due": 0.0015423086703241146,
+        "freeze_account": 0.00029420321182839363,
+        "interest_rate": 0.000008800347440347128,
+        "min_payment": 0.000040604397025496964,
+        "out_of_scope": 0.004172430448509115,
+        "pay_bill": 0.0015423086703241146,
+        "pin_change": 0.000014863402695596138,
+        "routing": 0.00005603903641798807,
+        "transfer": 0.00005603903641798807
+      }
+    },
+    {
+      "id": "clinc-02",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "easy",
+      "expected": "freeze_account",
+      "actual": "freeze_account",
+      "accurate": true,
+      "confidence": 0.9966944292430922,
+      "entropy_nats": 0.026382103060531837,
+      "normalized_entropy": 0.01061693929741088,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.000014747681280887302,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "account_blocked": 0.0013624713173290375,
+        "balance": 0.00016834690893196912,
+        "bill_balance": 0.00000837747144945541,
+        "bill_due": 0.00011720988746274387,
+        "freeze_account": 0.9966944292430922,
+        "interest_rate": 0.000001993208357160913,
+        "min_payment": 0.000005907005010618217,
+        "out_of_scope": 0.0013624713173290375,
+        "pay_bill": 0.0002566332716109701,
+        "pin_change": 0.00000452625983790975,
+        "routing": 0.000009952782829379132,
+        "transfer": 0.000007681326759679617
+      }
+    },
+    {
+      "id": "clinc-03",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "out-of-scope",
+      "expected": "out_of_scope",
+      "actual": "out_of_scope",
+      "accurate": true,
+      "confidence": 0.9997565929831389,
+      "entropy_nats": 0.002738953979592599,
+      "normalized_entropy": 0.0011022361664275286,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 7.003346036760498e-8,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "account_blocked": 0.0000643347817489966,
+        "balance": 0.000027174894335716328,
+        "bill_balance": 0.000004193054580941234,
+        "bill_due": 0.000034540123713254665,
+        "freeze_account": 0.000016640378727868412,
+        "interest_rate": 0.0000013948864724191134,
+        "min_payment": 0.000008460708898573911,
+        "out_of_scope": 0.9997565929831389,
+        "pay_bill": 0.0000643347817489966,
+        "pin_change": 0.0000026690983500834736,
+        "routing": 0.000007760755302564626,
+        "transfer": 0.000011903552981721317
+      }
+    },
+    {
+      "id": "clinc-04",
+      "metric": "calibration/clinc-oos-intent-select",
+      "category": "intent",
+      "tier": "out-of-scope",
+      "expected": "out_of_scope",
+      "actual": "out_of_scope",
+      "accurate": true,
+      "confidence": 0.9925623827748503,
+      "entropy_nats": 0.05568207636509586,
+      "normalized_entropy": 0.02240811596276519,
+      "vocab_cardinality": 12,
+      "chance_baseline": 0.08333333333333333,
+      "brier_score": 0.0000722859802419163,
+      "wall_time_ms": 273,
+      "top_probabilities": {
+        "account_blocked": 0.0008644911452409701,
+        "balance": 0.0003197126139270134,
+        "bill_balance": 0.00027563158014042244,
+        "bill_due": 0.00012819036855482422,
+        "freeze_account": 0.00010946774156342239,
+        "interest_rate": 0.003768739789999622,
+        "min_payment": 0.00012819036855482422,
+        "out_of_scope": 0.9925623827748503,
+        "pay_bill": 0.0012108455526947184,
+        "pin_change": 0.000005331804183870316,
+        "routing": 0.00005728753954191565,
+        "transfer": 0.0005697287207480293
+      }
+    },
+    {
+      "id": "anli-01",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "adversarial",
+      "expected": "contradiction",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.9138947736912604,
+      "entropy_nats": 0.3187714819216458,
+      "normalized_entropy": 0.29015830717504976,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.6841257330329102,
+      "stderr": 0.053685671815687244,
+      "wall_time_ms": 377,
+      "top_probabilities": {
+        "contradiction": 0.07866040447585942,
+        "entailment": 0.9138947736912604,
+        "neutral": 0.007444821832880181
+      }
+    },
+    {
+      "id": "anli-02",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "adversarial",
+      "expected": "contradiction",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.8851232655965604,
+      "entropy_nats": 0.42974954445729435,
+      "normalized_entropy": 0.3911748929900433,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.6377098488493187,
+      "stderr": 0.04834077135036923,
+      "wall_time_ms": 377,
+      "top_probabilities": {
+        "contradiction": 0.07653034278393167,
+        "entailment": 0.8851232655965604,
+        "neutral": 0.038346391619508
+      }
+    },
+    {
+      "id": "anli-03",
+      "metric": "calibration/anli-entailment-select",
+      "category": "nli",
+      "tier": "ambiguous",
+      "expected": "neutral",
+      "actual": "entailment",
+      "accurate": false,
+      "confidence": 0.4868740924771719,
+      "entropy_nats": 0.8811801128419752,
+      "normalized_entropy": 0.8020847044322289,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 1.3214478596450618,
+      "tvd_gold": 0.23687409247717195,
+      "stderr": 0.11774282772442646,
+      "wall_time_ms": 366,
+      "top_probabilities": {
+        "contradiction": 0.4515011836627691,
+        "entailment": 0.4868740924771719,
+        "neutral": 0.06162472386005898
+      }
+    },
+    {
+      "id": "chaos-01",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "entailment",
+      "actual": "entailment",
+      "accurate": true,
+      "confidence": 0.9738249349854984,
+      "entropy_nats": 0.12956878961887508,
+      "normalized_entropy": 0.11793859485766027,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.0012493037352389426,
+      "tvd_gold": 0.05382493498549834,
+      "stderr": 0.0037707498653369582,
+      "wall_time_ms": 220,
+      "top_probabilities": {
+        "contradiction": 0.0025613106667977193,
+        "entailment": 0.9738249349854984,
+        "neutral": 0.023613754347703884
+      }
+    },
+    {
+      "id": "chaos-02",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "neutral",
+      "actual": "contradiction",
+      "accurate": false,
+      "confidence": 0.5238794171558044,
+      "entropy_nats": 0.9638899642543727,
+      "normalized_entropy": 0.8773704556162703,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.707842892101104,
+      "tvd_gold": 0.07612058284419554,
+      "stderr": 0.07183489835951937,
+      "wall_time_ms": 240,
+      "top_probabilities": {
+        "contradiction": 0.5238794171558044,
+        "entailment": 0.12287740806443596,
+        "neutral": 0.35324317477975953
+      }
+    },
+    {
+      "id": "chaos-03",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "contradiction",
+      "actual": "contradiction",
+      "accurate": true,
+      "confidence": 0.9918814737333803,
+      "entropy_nats": 0.05217435579353033,
+      "normalized_entropy": 0.04749114526725651,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.00010375314422031651,
+      "tvd_gold": 0.07188147373338027,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "contradiction": 0.9918814737333803,
+        "entailment": 0.0024960227315120407,
+        "neutral": 0.0056225035351076405
+      }
+    },
+    {
+      "id": "chaos-04",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "contradiction",
+      "actual": "neutral",
+      "accurate": false,
+      "confidence": 0.6857089368820569,
+      "entropy_nats": 0.6806904251285222,
+      "normalized_entropy": 0.6195911261412793,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.960433392503027,
+      "tvd_gold": 0.08570893688205694,
+      "stderr": 0.018365624431298997,
+      "wall_time_ms": 235,
+      "top_probabilities": {
+        "contradiction": 0.299977312561854,
+        "entailment": 0.014313750556089069,
+        "neutral": 0.6857089368820569
+      }
+    },
+    {
+      "id": "chaos-05",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "low-entropy",
+      "expected": "entailment",
+      "actual": "entailment",
+      "accurate": true,
+      "confidence": 0.9976932701146538,
+      "entropy_nats": 0.016724049301260743,
+      "normalized_entropy": 0.015222885702048682,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.000010197864984901563,
+      "tvd_gold": 0.07769327011465377,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "contradiction": 0.00010066345626360885,
+        "entailment": 0.9976932701146538,
+        "neutral": 0.0022060664290826262
+      }
+    },
+    {
+      "id": "chaos-06",
+      "metric": "calibration/chaos-nli-entailment-select",
+      "category": "nli-soft",
+      "tier": "high-entropy",
+      "expected": "neutral",
+      "actual": "neutral",
+      "accurate": true,
+      "confidence": 0.42245725400463163,
+      "entropy_nats": 1.0724390233728094,
+      "normalized_entropy": 0.9761760672393069,
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "brier_score": 0.505479031100332,
+      "tvd_gold": 0.02245725400463161,
+      "stderr": 0.08363603900828781,
+      "wall_time_ms": 237,
+      "top_probabilities": {
+        "contradiction": 0.23804861714527578,
+        "entailment": 0.3394941288500926,
+        "neutral": 0.42245725400463163
+      }
+    },
+    {
+      "id": "adlab-01",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "easy",
+      "expected": "benign",
+      "actual": "benign",
+      "accurate": true,
+      "confidence": 0.9991846174854317,
+      "entropy_nats": 0.007242435772868818,
+      "normalized_entropy": 0.005224313086737336,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.0000010429727528165665,
+      "wall_time_ms": 119,
+      "top_probabilities": {
+        "benign": 0.9991846174854317,
+        "failed_injection": 0.0001186149565662165,
+        "hijacked": 0.00010209930436891081,
+        "injection_point": 0.0005946682536331489
+      }
+    },
+    {
+      "id": "adlab-02",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "injection_point",
+      "actual": "injection_point",
+      "accurate": true,
+      "confidence": 0.9997377057061719,
+      "entropy_nats": 0.002549901408679189,
+      "normalized_entropy": 0.001839365058528624,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 1.2138348095555189e-7,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "benign": 0.00022796397209085874,
+        "failed_injection": 0.000020928557875064575,
+        "hijacked": 0.000013401763862067968,
+        "injection_point": 0.9997377057061719
+      }
+    },
+    {
+      "id": "adlab-03",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "hijacked",
+      "actual": "hijacked",
+      "accurate": true,
+      "confidence": 0.9983417729357298,
+      "entropy_nats": 0.013684117974035253,
+      "normalized_entropy": 0.009871004570040094,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.000004148263148471669,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "benign": 0.0011152448548011336,
+        "failed_injection": 0.00021082717091456868,
+        "hijacked": 0.9983417729357298,
+        "injection_point": 0.000332155038554618
+      }
+    },
+    {
+      "id": "adlab-04",
+      "metric": "calibration/agent-step-label-select",
+      "category": "agent-trajectory",
+      "tier": "localization",
+      "expected": "benign",
+      "actual": "benign",
+      "accurate": true,
+      "confidence": 0.9925819692724434,
+      "entropy_nats": 0.04449420894263265,
+      "normalized_entropy": 0.03209578729490675,
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "brier_score": 0.00010819470129786384,
+      "wall_time_ms": 154,
+      "top_probabilities": {
+        "benign": 0.9925819692724434,
+        "failed_injection": 0.000048379851771345966,
+        "hijacked": 0.00007862911785199827,
+        "injection_point": 0.00729102175793336
+      }
+    },
+    {
+      "id": "b77-01",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "card_arrival",
+      "actual": "card_arrival",
+      "accurate": true,
+      "confidence": 0.9219869658266583,
+      "entropy_nats": 0.40318827269656515,
+      "normalized_entropy": 0.12374963970263632,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.008323511988969912,
+      "stderr": 0.007891537482948843,
+      "wall_time_ms": 1353,
+      "top_probabilities": {
+        "activate_my_card": 0.005989659757952318,
+        "apple_pay_or_google_pay": 0.003505742393094032,
+        "atm_support": 0.00024001010313010783,
+        "automatic_top_up": 0.0009597467854853942,
+        "balance_not_updated_after_bank_transfer": 0.00045737709793176786,
+        "beneficiary_not_allowed": 0.00011575180816596809,
+        "cancel_transfer": 0.00008958452594447844,
+        "card_about_to_expire": 0.0005162922768956536,
+        "card_acceptance": 0.0010590160941283937,
+        "card_arrival": 0.9219869658266583,
+        "card_delivery_estimate": 0.04603792139656742,
+        "card_linking": 0.0052092899921558355,
+        "card_not_working": 0.00025997560427795994,
+        "card_payment_fee_charged": 0.00014723331949210135,
+        "card_payment_not_recognised": 0.00000361720433893413,
+        "card_swallowed": 0.0004925999521049939,
+        "change_pin": 0.0038240771968184807,
+        "compromised_card": 0.00016038549833998124,
+        "contactless_not_working": 0.000017331862323346874,
+        "declined_card_payment": 0.00003956272587376536,
+        "declined_transfer": 0.00006952908693376283,
+        "failed_transfer": 0.000038269209402649154,
+        "lost_or_stolen_card": 0.0030296418299076916,
+        "pending_top_up": 0.00023205776562690007,
+        "transaction_charged_twice": 0.0021935848733898554,
+        "transfer_not_received_by_recipient": 0.0033247758130600263
+      }
+    },
+    {
+      "id": "b77-02",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "pending_top_up",
+      "actual": "pending_top_up",
+      "accurate": true,
+      "confidence": 0.639853570962882,
+      "entropy_nats": 0.6990882425250634,
+      "normalized_entropy": 0.2145695298978443,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.25475510427283155,
+      "stderr": 0.06026920712606249,
+      "wall_time_ms": 1355,
+      "top_probabilities": {
+        "activate_my_card": 0.0006819906056681555,
+        "apple_pay_or_google_pay": 0.0010434606788243617,
+        "atm_support": 0.000013260213286383106,
+        "automatic_top_up": 0.002041433848096379,
+        "balance_not_updated_after_bank_transfer": 0.3536126110102365,
+        "beneficiary_not_allowed": 0.000019506142075547903,
+        "cancel_transfer": 0.000008099754776760039,
+        "card_about_to_expire": 0.00010468835413248299,
+        "card_acceptance": 0.000016036672646424407,
+        "card_arrival": 0.00002339051490213694,
+        "card_delivery_estimate": 0.0000012990734002239683,
+        "card_linking": 0.00010822101830846979,
+        "card_not_working": 0.000008703965915104863,
+        "card_payment_fee_charged": 0.00002266936034363504,
+        "card_payment_not_recognised": 0.0000018799839305763667,
+        "card_swallowed": 0.000021781211767072682,
+        "change_pin": 0.0004443177293482688,
+        "compromised_card": 0.00002962910802069559,
+        "contactless_not_working": 0.000002890782307766161,
+        "declined_card_payment": 4.1323897145732303e-7,
+        "declined_transfer": 0.00008167738351343836,
+        "failed_transfer": 0.000013329041965707533,
+        "lost_or_stolen_card": 0.000005129785982920055,
+        "pending_top_up": 0.639853570962882,
+        "transaction_charged_twice": 0.0006643475314836486,
+        "transfer_not_received_by_recipient": 0.0011756620272138172
+      }
+    },
+    {
+      "id": "b77-03",
+      "metric": "calibration/banking77-intent-select",
+      "category": "intent",
+      "tier": "held-out",
+      "expected": "transaction_charged_twice",
+      "actual": "transaction_charged_twice",
+      "accurate": true,
+      "confidence": 0.9974190115697303,
+      "entropy_nats": 0.021829477594247793,
+      "normalized_entropy": 0.006700070835686165,
+      "vocab_cardinality": 26,
+      "chance_baseline": 0.038461538461538464,
+      "brier_score": 0.000009236378678951248,
+      "wall_time_ms": 136,
+      "top_probabilities": {
+        "activate_my_card": 0.0003798507843310514,
+        "apple_pay_or_google_pay": 0.00006801882020785168,
+        "atm_support": 0.000007550010265023075,
+        "automatic_top_up": 0.00001977856571088189,
+        "balance_not_updated_after_bank_transfer": 0.0000579916811240748,
+        "beneficiary_not_allowed": 1.306876915231585e-7,
+        "cancel_transfer": 0.000003017276358493615,
+        "card_about_to_expire": 0.000009027143821040945,
+        "card_acceptance": 0.000005256174595626376,
+        "card_arrival": 0.000009027143821040945,
+        "card_delivery_estimate": 0.00005351469691694059,
+        "card_linking": 0.000016661417954974726,
+        "card_not_working": 0.0000036362098533401126,
+        "card_payment_fee_charged": 0.000025504382281588744,
+        "card_payment_not_recognised": 4.992434420231379e-7,
+        "card_swallowed": 0.000009864919215036762,
+        "change_pin": 0.0002858863984676787,
+        "compromised_card": 0.00005351469691694059,
+        "contactless_not_working": 0.000001551214193138916,
+        "declined_card_payment": 0.000003989427020533254,
+        "declined_transfer": 0.000009027143821040945,
+        "failed_transfer": 0.0000022740133194238797,
+        "lost_or_stolen_card": 0.000008257255475703564,
+        "pending_top_up": 0.00001977856571088189,
+        "transaction_charged_twice": 0.9974190115697303,
+        "transfer_not_received_by_recipient": 0.0015273805577537606
+      }
+    },
+    {
+      "id": "tox-s-01",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "0.0",
+      "actual": "0.00",
+      "accurate": true,
+      "confidence": 0.9983553232638073,
+      "entropy_nats": 0.01389333744201951,
+      "normalized_entropy": 0.008632415910351797,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0000038699600013226885,
+      "wall_time_ms": 1214,
+      "top_probabilities": {
+        "0.0": 0.9983553232638073,
+        "0.2": 0.0009756027275894846,
+        "0.5": 0.00008318848815382418,
+        "0.7": 0.00016132492979967602,
+        "0.9": 0.0004245605906497782
+      }
+    },
+    {
+      "id": "tox-s-02",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "ambiguous",
+      "expected": "0.2",
+      "actual": "0.20",
+      "accurate": true,
+      "confidence": 0.9185083328701555,
+      "entropy_nats": 0.36985171550664103,
+      "normalized_entropy": 0.22980179145107898,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.009230662547706917,
+      "tvd_gold": 0.16850833287015554,
+      "stderr": 0.004530264906739705,
+      "wall_time_ms": 1320,
+      "top_probabilities": {
+        "0.0": 0.037710335897437657,
+        "0.2": 0.9185083328701555,
+        "0.5": 0.03335806084867539,
+        "0.7": 0.005767870878357989,
+        "0.9": 0.004655399505373397
+      }
+    },
+    {
+      "id": "tox-s-03",
+      "metric": "calibration/civil-comments-toxicity-rate",
+      "category": "toxicity",
+      "tier": "easy",
+      "expected": "0.9",
+      "actual": "0.90",
+      "accurate": true,
+      "confidence": 0.8653576255043002,
+      "entropy_nats": 0.5419057063462056,
+      "normalized_entropy": 0.3367049465900999,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.024924730910118226,
+      "stderr": 0.03382260870433078,
+      "wall_time_ms": 245,
+      "top_probabilities": {
+        "0.0": 0.016680484038127538,
+        "0.2": 0.0039791398717578435,
+        "0.5": 0.05873150213579946,
+        "0.7": 0.055251248450015056,
+        "0.9": 0.8653576255043002
+      }
+    },
+    {
+      "id": "yelp-01",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "easy",
+      "expected": "1",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.9720056642028677,
+      "entropy_nats": 0.15886882330954616,
+      "normalized_entropy": 0.0987107499345994,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0010911317066652263,
+      "stderr": 0.007935390639697124,
+      "wall_time_ms": 240,
+      "top_probabilities": {
+        "1": 0.9720056642028677,
+        "2": 0.006093495225664612,
+        "3": 0.00473719197650661,
+        "4": 0.001490217233158134,
+        "5": 0.015673431361802895
+      }
+    },
+    {
+      "id": "yelp-02",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "ambiguous",
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9840356340702405,
+      "entropy_nats": 0.10332746736793424,
+      "normalized_entropy": 0.06420096517526586,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0003234704609940504,
+      "tvd_gold": 0.23403563407024053,
+      "stderr": 0.0037091246135593785,
+      "wall_time_ms": 238,
+      "top_probabilities": {
+        "1": 0.004224191604465417,
+        "2": 0.0046344733289595845,
+        "3": 0.9840356340702405,
+        "4": 0.004974442982330179,
+        "5": 0.002131258014004302
+      }
+    },
+    {
+      "id": "yelp-03",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "easy",
+      "expected": "5",
+      "actual": "5",
+      "accurate": true,
+      "confidence": 0.9906701570871883,
+      "entropy_nats": 0.05976311804055543,
+      "normalized_entropy": 0.03713291303680687,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.0001431091023953098,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "1": 0.0074044858460318545,
+        "2": 0.0006285169163875682,
+        "3": 0.0006285169163875682,
+        "4": 0.0006683232340046953,
+        "5": 0.9906701570871883
+      }
+    },
+    {
+      "id": "yelp-04",
+      "metric": "calibration/yelp-review-stars-rate",
+      "category": "ordinal",
+      "tier": "ambiguous",
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.8669146945934588,
+      "entropy_nats": 0.5377691922533382,
+      "normalized_entropy": 0.3341347858769032,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.025023563897153264,
+      "tvd_gold": 0.11691469459345882,
+      "stderr": 0.012429383490494066,
+      "wall_time_ms": 222,
+      "top_probabilities": {
+        "1": 0.02638479204724186,
+        "2": 0.009101983198833222,
+        "3": 0.8669146945934588,
+        "4": 0.07854926917881411,
+        "5": 0.019049260981652008
+      }
+    },
+    {
+      "id": "sst5-01",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "1",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.8271924119184983,
+      "entropy_nats": 0.574892056588922,
+      "normalized_entropy": 0.3572005183595185,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.049001193814710776,
+      "stderr": 0.02244872857904024,
+      "wall_time_ms": 222,
+      "top_probabilities": {
+        "0": 0.8271924119184983,
+        "1": 0.13495112525242783,
+        "2": 0.02967371313700767,
+        "3": 0.006632748654570101,
+        "4": 0.0015500010374961744
+      }
+    },
+    {
+      "id": "sst5-02",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "2",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.6162813938001136,
+      "entropy_nats": 0.8940537901188007,
+      "normalized_entropy": 0.5555068531762379,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.2505472276437561,
+      "stderr": 0.025231766757863115,
+      "wall_time_ms": 231,
+      "top_probabilities": {
+        "0": 0.02625558802514162,
+        "1": 0.6162813938001136,
+        "2": 0.31818752450275667,
+        "3": 0.03700610954589057,
+        "4": 0.0022693841260976017
+      }
+    },
+    {
+      "id": "sst5-03",
+      "metric": "calibration/sst5-sentiment-rate",
+      "category": "ordinal",
+      "tier": "held-out",
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.8160366583625143,
+      "entropy_nats": 0.6684724919057584,
+      "normalized_entropy": 0.41534531201316505,
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "brier_score": 0.04779006515329314,
+      "stderr": 0.04253217414566576,
+      "wall_time_ms": 231,
+      "top_probabilities": {
+        "0": 0.8160366583625143,
+        "1": 0.10149094135784291,
+        "2": 0.05652249098150099,
+        "3": 0.020579296373276375,
+        "4": 0.005370612924865472
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/contract_rollback_cloudrun.json b/benchmarks/runs/20260928-v010-verification/contract_rollback_cloudrun.json
new file mode 100644
index 0000000..3392085
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/contract_rollback_cloudrun.json
@@ -0,0 +1,831 @@
+{
+ "health": {
+  "cr_rollback_00032": {
+   "status": 200,
+   "body": {
+    "status": "ok",
+    "server": "dgem-structured-server",
+    "vllm_ready": false,
+    "phase": "staging_tmpfs",
+    "bytes_staged_gb": 0.0,
+    "boot_ts": 1790635338.935399,
+    "stage_start_ts": 1790635338.999826
+   }
+  }
+ },
+ "cases": {
+  "triage_s1": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9794809554761067,
+      "probabilities": {
+       "yes": 0.9794809554761067,
+       "no": 0.020519044523893235
+      },
+      "noul": 0.9794809554761067
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.9748278349258633,
+      "probabilities": {
+       "billing": 0.024488219890708512,
+       "support": 0.0006839451834283018,
+       "engineering": 0.9748278349258633
+      },
+      "choice": "engineering"
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9893275713532491,
+      "probabilities": {
+       "calm": 0.006691029686497045,
+       "frustrated": 0.9893275713532491,
+       "furious": 0.003981398960253843
+      },
+      "score": 1.9972903692737567,
+      "level": "frustrated"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 91458.71782302856,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.9794809554761067,
+      "no": 0.020519044523893235
+     },
+     "team": {
+      "billing": 0.024488219890708512,
+      "support": 0.0006839451834283018,
+      "engineering": 0.9748278349258633
+     },
+     "sentiment": {
+      "calm": 0.006691029686497045,
+      "frustrated": 0.9893275713532491,
+      "furious": 0.003981398960253843
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2"
+    }
+   }
+  },
+  "triage_s4": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9468836377410987,
+      "probabilities": {
+       "yes": 0.9468836377410987,
+       "no": 0.053116362258901276
+      },
+      "noul": 0.9468836377410987,
+      "stderr": 0.021595451723594384,
+      "agreement": 1.0
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.9413688264194429,
+      "probabilities": {
+       "billing": 0.055938936108213724,
+       "support": 0.0026922374723433932,
+       "engineering": 0.9413688264194429
+      },
+      "choice": "engineering",
+      "stderr": 0.011696036119967753,
+      "agreement": 1.0
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9834006241636751,
+      "probabilities": {
+       "calm": 0.01005942244945491,
+       "frustrated": 0.9834006241636751,
+       "furious": 0.006539953386869948
+      },
+      "score": 1.996480530937415,
+      "level": "frustrated",
+      "stderr": 0.003066445317634698,
+      "agreement": 1.0
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 4,
+    "denoise_ms": 91457.56125450134,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.9468836377410987,
+      "no": 0.053116362258901276
+     },
+     "team": {
+      "billing": 0.055938936108213724,
+      "support": 0.0026922374723433932,
+      "engineering": 0.9413688264194429
+     },
+     "sentiment": {
+      "calm": 0.01005942244945491,
+      "frustrated": 0.9834006241636751,
+      "furious": 0.006539953386869948
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2"
+    }
+   }
+  },
+  "samples_string": {
+   "cr_rollback_00032": {
+    "status": 400,
+    "error": "schema: samples must be a positive count or \"auto\""
+   }
+  },
+  "default_samples": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9794809554761067,
+      "probabilities": {
+       "yes": 0.9794809554761067,
+       "no": 0.020519044523893235
+      },
+      "noul": 0.9794809554761067
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.9748278349258633,
+      "probabilities": {
+       "billing": 0.024488219890708512,
+       "support": 0.0006839451834283018,
+       "engineering": 0.9748278349258633
+      },
+      "choice": "engineering"
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9893275713532491,
+      "probabilities": {
+       "calm": 0.006691029686497045,
+       "frustrated": 0.9893275713532491,
+       "furious": 0.003981398960253843
+      },
+      "score": 1.9972903692737567,
+      "level": "frustrated"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 91454.96201515198,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.9794809554761067,
+      "no": 0.020519044523893235
+     },
+     "team": {
+      "billing": 0.024488219890708512,
+      "support": 0.0006839451834283018,
+      "engineering": 0.9748278349258633
+     },
+     "sentiment": {
+      "calm": 0.006691029686497045,
+      "frustrated": 0.9893275713532491,
+      "furious": 0.003981398960253843
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2"
+    }
+   }
+  },
+  "mirror_s1": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.8518040627724602,
+      "probabilities": {
+       "yes": 0.8518040627724602,
+       "no": 0.14819593722753982
+      },
+      "noul": 0.8518040627724602
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.8624406365940793,
+      "probabilities": {
+       "billing": 0.13629326277679935,
+       "support": 0.0012661006291213497,
+       "engineering": 0.8624406365940793
+      },
+      "choice": "engineering"
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9317971807886221,
+      "probabilities": {
+       "calm": 0.058611639027540276,
+       "frustrated": 0.9317971807886221,
+       "furious": 0.00959118018383762
+      },
+      "score": 1.9509795411562973,
+      "level": "frustrated"
+     },
+     "urgent__rev": {
+      "type": "choice",
+      "label": "A",
+      "confidence": 0.8138439633001073,
+      "probabilities": {
+       "no": 0.8138439633001073,
+       "yes": 0.1861560366998927
+      },
+      "choice": "no"
+     },
+     "team__rev": {
+      "type": "choice",
+      "label": "A",
+      "confidence": 0.9522892857470563,
+      "probabilities": {
+       "engineering": 0.9522892857470563,
+       "support": 0.014868217827145348,
+       "billing": 0.03284249642579831
+      },
+      "choice": "engineering"
+     },
+     "sentiment__rev": {
+      "type": "score",
+      "label": "3",
+      "confidence": 0.771446635697799,
+      "probabilities": {
+       "furious": 0.09793673701972952,
+       "frustrated": 0.1306166272824715,
+       "calm": 0.771446635697799
+      },
+      "score": 2.67350989867807,
+      "level": "calm"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 91434.8247051239,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.8518040627724602,
+      "no": 0.14819593722753982
+     },
+     "team": {
+      "billing": 0.13629326277679935,
+      "support": 0.0012661006291213497,
+      "engineering": 0.8624406365940793
+     },
+     "sentiment": {
+      "calm": 0.058611639027540276,
+      "frustrated": 0.9317971807886221,
+      "furious": 0.00959118018383762
+     },
+     "urgent__rev": {
+      "no": 0.8138439633001073,
+      "yes": 0.1861560366998927
+     },
+     "team__rev": {
+      "engineering": 0.9522892857470563,
+      "support": 0.014868217827145348,
+      "billing": 0.03284249642579831
+     },
+     "sentiment__rev": {
+      "furious": 0.09793673701972952,
+      "frustrated": 0.1306166272824715,
+      "calm": 0.771446635697799
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2",
+     "urgent__rev": "A",
+     "team__rev": "A",
+     "sentiment__rev": "3"
+    }
+   }
+  },
+  "think120": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "department": {
+      "type": "choice",
+      "label": "D",
+      "confidence": 0.9900434260009825,
+      "probabilities": {
+       "it_helpdesk": 6.957419350486225e-05,
+       "hr_people_ops": 0.009861242814499804,
+       "facilities": 2.575699101288641e-05,
+       "other_unclassified": 0.9900434260009825
+      },
+      "choice": "other_unclassified"
+     },
+     "requires_new_category": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9915395208030443,
+      "probabilities": {
+       "yes": 0.9915395208030443,
+       "no": 0.008460479196955669
+      },
+      "noul": 0.9915395208030443
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 91851.73058509827,
+    "probabilities": {
+     "department": {
+      "it_helpdesk": 6.957419350486225e-05,
+      "hr_people_ops": 0.009861242814499804,
+      "facilities": 2.575699101288641e-05,
+      "other_unclassified": 0.9900434260009825
+     },
+     "requires_new_category": {
+      "yes": 0.9915395208030443,
+      "no": 0.008460479196955669
+     }
+    },
+    "labels": {
+     "department": "D",
+     "requires_new_category": "yes"
+    }
+   }
+  },
+  "image_s1": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "has_button": {
+      "type": "noul",
+      "label": "no",
+      "confidence": 0.9964630034693642,
+      "probabilities": {
+       "yes": 0.003536996530635852,
+       "no": 0.9964630034693642
+      },
+      "noul": 0.003536996530635852
+     },
+     "dominant_color": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.999207934384256,
+      "probabilities": {
+       "red": 0.0003869810272619841,
+       "green": 0.00036023157772876335,
+       "blue": 0.999207934384256,
+       "white": 4.485301075332818e-05
+      },
+      "choice": "blue"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 91398.11396598816,
+    "probabilities": {
+     "has_button": {
+      "yes": 0.003536996530635852,
+      "no": 0.9964630034693642
+     },
+     "dominant_color": {
+      "red": 0.0003869810272619841,
+      "green": 0.00036023157772876335,
+      "blue": 0.999207934384256,
+      "white": 4.485301075332818e-05
+     }
+    },
+    "labels": {
+     "has_button": "no",
+     "dominant_color": "C"
+    }
+   }
+  },
+  "choice_26": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "pick": {
+      "type": "choice",
+      "label": "B",
+      "confidence": 0.5586765802008992,
+      "probabilities": {
+       "opt_00": 0.327110534227298,
+       "opt_01": 0.5586765802008992,
+       "opt_02": 0.048895170660441675,
+       "opt_03": 0.008852346583868822,
+       "opt_04": 0.009892548705644412,
+       "opt_05": 0.0014085504985519963,
+       "opt_06": 0.01103976721698335,
+       "opt_07": 0.0005323410959614957,
+       "opt_08": 0.00020138519552982166,
+       "opt_09": 0.0015055900002954566,
+       "opt_10": 0.0006145771661568181,
+       "opt_11": 0.0018345330513766546,
+       "opt_12": 0.024087010865204747,
+       "opt_13": 0.0006145771661568181,
+       "opt_14": 1.6523831427140274e-05,
+       "opt_15": 0.0007601299673131074,
+       "opt_16": 0.0032524559017676254,
+       "opt_17": 0.00015939451286814337,
+       "opt_18": 9.108394119220128e-05,
+       "opt_19": 2.804190502954912e-05,
+       "opt_20": 0.00013611441747134973,
+       "opt_21": 1.1521860127477082e-05,
+       "opt_22": 2.3546952381802993e-05,
+       "opt_23": 1.3808969526832655e-05,
+       "opt_24": 0.00018635817407692794,
+       "opt_25": 5.550693244854763e-05
+      },
+      "choice": "opt_01"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 91454.20956611633,
+    "probabilities": {
+     "pick": {
+      "opt_00": 0.327110534227298,
+      "opt_01": 0.5586765802008992,
+      "opt_02": 0.048895170660441675,
+      "opt_03": 0.008852346583868822,
+      "opt_04": 0.009892548705644412,
+      "opt_05": 0.0014085504985519963,
+      "opt_06": 0.01103976721698335,
+      "opt_07": 0.0005323410959614957,
+      "opt_08": 0.00020138519552982166,
+      "opt_09": 0.0015055900002954566,
+      "opt_10": 0.0006145771661568181,
+      "opt_11": 0.0018345330513766546,
+      "opt_12": 0.024087010865204747,
+      "opt_13": 0.0006145771661568181,
+      "opt_14": 1.6523831427140274e-05,
+      "opt_15": 0.0007601299673131074,
+      "opt_16": 0.0032524559017676254,
+      "opt_17": 0.00015939451286814337,
+      "opt_18": 9.108394119220128e-05,
+      "opt_19": 2.804190502954912e-05,
+      "opt_20": 0.00013611441747134973,
+      "opt_21": 1.1521860127477082e-05,
+      "opt_22": 2.3546952381802993e-05,
+      "opt_23": 1.3808969526832655e-05,
+      "opt_24": 0.00018635817407692794,
+      "opt_25": 5.550693244854763e-05
+     }
+    },
+    "labels": {
+     "pick": "B"
+    }
+   }
+  },
+  "choice_27_error": {
+   "cr_rollback_00032": {
+    "status": 400,
+    "error": "question 'pick': at most 26 alternatives"
+   }
+  },
+  "depends_on_2reads": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "billing_issue": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9973924015054095,
+      "probabilities": {
+       "yes": 0.9973924015054095,
+       "no": 0.002607598494590423
+      },
+      "noul": 0.9973924015054095
+     },
+     "refund": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.998079526882602,
+      "probabilities": {
+       "yes": 0.998079526882602,
+       "no": 0.0019204731173980004
+      },
+      "noul": 0.998079526882602
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 2,
+    "denoise_ms": 91500.15568733215,
+    "probabilities": {
+     "billing_issue": {
+      "yes": 0.9973924015054095,
+      "no": 0.002607598494590423
+     },
+     "refund": {
+      "yes": 0.998079526882602,
+      "no": 0.0019204731173980004
+     }
+    },
+    "labels": {
+     "billing_issue": "yes",
+     "refund": "yes"
+    }
+   }
+  },
+  "unknown_type_error": {
+   "cr_rollback_00032": {
+    "status": 400,
+    "error": "question 'a': unknown type 'banana'"
+   }
+  },
+  "empty_questions_error": {
+   "cr_rollback_00032": {
+    "status": 400,
+    "error": "schema: needs a non-empty questions array"
+   }
+  },
+  "systemone": {
+   "cr_rollback_00032": {
+    "status": 200,
+    "envelope_keys": [
+     "answers",
+     "diagnostics",
+     "model",
+     "usage"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "noul": 0.09116820634923525
+     },
+     "billing": {
+      "type": "noul",
+      "noul": 0.9977163913940218
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 251.50132179260254,
+    "probabilities": {
+     "urgent": null,
+     "billing": null
+    },
+    "labels": {
+     "urgent": null,
+     "billing": null
+    }
+   }
+  },
+  "invalid_json_error": {
+   "cr_rollback_00032": {
+    "status": 400,
+    "error": "invalid body: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)"
+   }
+  }
+ }
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/contract_rollback_vertex.json b/benchmarks/runs/20260928-v010-verification/contract_rollback_vertex.json
new file mode 100644
index 0000000..ea68000
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/contract_rollback_vertex.json
@@ -0,0 +1,834 @@
+{
+ "health": {
+  "vertex_rollback": {
+   "status": 200,
+   "body": {
+    "status": "ok",
+    "server": "dgem-structured-server",
+    "vllm_ready": true,
+    "phase": "ready",
+    "bytes_staged_gb": 17.53,
+    "boot_ts": 1790635379.2526886,
+    "stage_start_ts": 1790635379.2526886,
+    "stage_end_ts": 1790635379.2526886,
+    "warmed": true,
+    "warmup_s": 16.0
+   }
+  }
+ },
+ "cases": {
+  "triage_s1": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.927646972863927,
+      "probabilities": {
+       "yes": 0.927646972863927,
+       "no": 0.07235302713607297
+      },
+      "noul": 0.927646972863927
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.76175306565893,
+      "probabilities": {
+       "billing": 0.2372346751555079,
+       "support": 0.0010122591855622307,
+       "engineering": 0.76175306565893
+      },
+      "choice": "engineering"
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9701460967937148,
+      "probabilities": {
+       "calm": 0.01056589192655398,
+       "frustrated": 0.9701460967937148,
+       "furious": 0.019288011279731156
+      },
+      "score": 2.008722119353177,
+      "level": "frustrated"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 1619.3983554840088,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.927646972863927,
+      "no": 0.07235302713607297
+     },
+     "team": {
+      "billing": 0.2372346751555079,
+      "support": 0.0010122591855622307,
+      "engineering": 0.76175306565893
+     },
+     "sentiment": {
+      "calm": 0.01056589192655398,
+      "frustrated": 0.9701460967937148,
+      "furious": 0.019288011279731156
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2"
+    }
+   }
+  },
+  "triage_s4": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9292395598519316,
+      "probabilities": {
+       "yes": 0.9292395598519316,
+       "no": 0.07076044014806838
+      },
+      "noul": 0.9292395598519316,
+      "stderr": 0.014425799901086873,
+      "agreement": 1.0
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.9145352694825613,
+      "probabilities": {
+       "billing": 0.08411738622278779,
+       "support": 0.001347344294650914,
+       "engineering": 0.9145352694825613
+      },
+      "choice": "engineering",
+      "stderr": 0.051473485601623147,
+      "agreement": 1.0
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9750351206273226,
+      "probabilities": {
+       "calm": 0.016719193025549507,
+       "frustrated": 0.9750351206273226,
+       "furious": 0.008245686347127912
+      },
+      "score": 1.9915264933215784,
+      "level": "frustrated",
+      "stderr": 0.005185585961473277,
+      "agreement": 1.0
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 4,
+    "denoise_ms": 1629.1697025299072,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.9292395598519316,
+      "no": 0.07076044014806838
+     },
+     "team": {
+      "billing": 0.08411738622278779,
+      "support": 0.001347344294650914,
+      "engineering": 0.9145352694825613
+     },
+     "sentiment": {
+      "calm": 0.016719193025549507,
+      "frustrated": 0.9750351206273226,
+      "furious": 0.008245686347127912
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2"
+    }
+   }
+  },
+  "samples_string": {
+   "vertex_rollback": {
+    "status": 400,
+    "error": "schema: samples must be a positive count or \"auto\""
+   }
+  },
+  "default_samples": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.927646972863927,
+      "probabilities": {
+       "yes": 0.927646972863927,
+       "no": 0.07235302713607297
+      },
+      "noul": 0.927646972863927
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.76175306565893,
+      "probabilities": {
+       "billing": 0.2372346751555079,
+       "support": 0.0010122591855622307,
+       "engineering": 0.76175306565893
+      },
+      "choice": "engineering"
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9701460967937148,
+      "probabilities": {
+       "calm": 0.01056589192655398,
+       "frustrated": 0.9701460967937148,
+       "furious": 0.019288011279731156
+      },
+      "score": 2.008722119353177,
+      "level": "frustrated"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 1628.1278133392334,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.927646972863927,
+      "no": 0.07235302713607297
+     },
+     "team": {
+      "billing": 0.2372346751555079,
+      "support": 0.0010122591855622307,
+      "engineering": 0.76175306565893
+     },
+     "sentiment": {
+      "calm": 0.01056589192655398,
+      "frustrated": 0.9701460967937148,
+      "furious": 0.019288011279731156
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2"
+    }
+   }
+  },
+  "mirror_s1": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9294909143977973,
+      "probabilities": {
+       "yes": 0.9294909143977973,
+       "no": 0.0705090856022027
+      },
+      "noul": 0.9294909143977973
+     },
+     "team": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.862708161001292,
+      "probabilities": {
+       "billing": 0.13536361536593802,
+       "support": 0.0019282236327700505,
+       "engineering": 0.862708161001292
+      },
+      "choice": "engineering"
+     },
+     "sentiment": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.9790651912238408,
+      "probabilities": {
+       "calm": 0.013772408016632215,
+       "frustrated": 0.9790651912238408,
+       "furious": 0.0071624007595268244
+      },
+      "score": 1.9933899927428944,
+      "level": "frustrated"
+     },
+     "urgent__rev": {
+      "type": "choice",
+      "label": "B",
+      "confidence": 0.8615366238386888,
+      "probabilities": {
+       "no": 0.1384633761613112,
+       "yes": 0.8615366238386888
+      },
+      "choice": "yes"
+     },
+     "team__rev": {
+      "type": "choice",
+      "label": "A",
+      "confidence": 0.9820320222438539,
+      "probabilities": {
+       "engineering": 0.9820320222438539,
+       "support": 0.0075364193091134405,
+       "billing": 0.010431558447032733
+      },
+      "choice": "engineering"
+     },
+     "sentiment__rev": {
+      "type": "score",
+      "label": "2",
+      "confidence": 0.8058103012294772,
+      "probabilities": {
+       "furious": 0.09962254289982532,
+       "frustrated": 0.8058103012294772,
+       "calm": 0.09456715587069747
+      },
+      "score": 1.994944612970872,
+      "level": "frustrated"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 1617.6409721374512,
+    "probabilities": {
+     "urgent": {
+      "yes": 0.9294909143977973,
+      "no": 0.0705090856022027
+     },
+     "team": {
+      "billing": 0.13536361536593802,
+      "support": 0.0019282236327700505,
+      "engineering": 0.862708161001292
+     },
+     "sentiment": {
+      "calm": 0.013772408016632215,
+      "frustrated": 0.9790651912238408,
+      "furious": 0.0071624007595268244
+     },
+     "urgent__rev": {
+      "no": 0.1384633761613112,
+      "yes": 0.8615366238386888
+     },
+     "team__rev": {
+      "engineering": 0.9820320222438539,
+      "support": 0.0075364193091134405,
+      "billing": 0.010431558447032733
+     },
+     "sentiment__rev": {
+      "furious": 0.09962254289982532,
+      "frustrated": 0.8058103012294772,
+      "calm": 0.09456715587069747
+     }
+    },
+    "labels": {
+     "urgent": "yes",
+     "team": "C",
+     "sentiment": "2",
+     "urgent__rev": "B",
+     "team__rev": "A",
+     "sentiment__rev": "2"
+    }
+   }
+  },
+  "think120": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "department": {
+      "type": "choice",
+      "label": "D",
+      "confidence": 0.9807583413346004,
+      "probabilities": {
+       "it_helpdesk": 0.0007180913961278903,
+       "hr_people_ops": 0.01838467788912123,
+       "facilities": 0.0001388893801503393,
+       "other_unclassified": 0.9807583413346004
+      },
+      "choice": "other_unclassified"
+     },
+     "requires_new_category": {
+      "type": "noul",
+      "label": "no",
+      "confidence": 0.6154987056069142,
+      "probabilities": {
+       "yes": 0.3845012943930857,
+       "no": 0.6154987056069142
+      },
+      "noul": 0.3845012943930857
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 1910.7089042663574,
+    "probabilities": {
+     "department": {
+      "it_helpdesk": 0.0007180913961278903,
+      "hr_people_ops": 0.01838467788912123,
+      "facilities": 0.0001388893801503393,
+      "other_unclassified": 0.9807583413346004
+     },
+     "requires_new_category": {
+      "yes": 0.3845012943930857,
+      "no": 0.6154987056069142
+     }
+    },
+    "labels": {
+     "department": "D",
+     "requires_new_category": "no"
+    }
+   }
+  },
+  "image_s1": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "has_button": {
+      "type": "noul",
+      "label": "no",
+      "confidence": 0.9825308905651104,
+      "probabilities": {
+       "yes": 0.017469109434889663,
+       "no": 0.9825308905651104
+      },
+      "noul": 0.017469109434889663
+     },
+     "dominant_color": {
+      "type": "choice",
+      "label": "C",
+      "confidence": 0.9987875503795025,
+      "probabilities": {
+       "red": 0.0006868687764550382,
+       "green": 0.0004800983001549823,
+       "blue": 0.9987875503795025,
+       "white": 4.548254388738905e-05
+      },
+      "choice": "blue"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 1615.7300472259521,
+    "probabilities": {
+     "has_button": {
+      "yes": 0.017469109434889663,
+      "no": 0.9825308905651104
+     },
+     "dominant_color": {
+      "red": 0.0006868687764550382,
+      "green": 0.0004800983001549823,
+      "blue": 0.9987875503795025,
+      "white": 4.548254388738905e-05
+     }
+    },
+    "labels": {
+     "has_button": "no",
+     "dominant_color": "C"
+    }
+   }
+  },
+  "choice_26": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "pick": {
+      "type": "choice",
+      "label": "B",
+      "confidence": 0.5368288441971905,
+      "probabilities": {
+       "opt_00": 0.29997741714000264,
+       "opt_01": 0.5368288441971905,
+       "opt_02": 0.02522306219997332,
+       "opt_03": 0.017849704493855048,
+       "opt_04": 0.013808502560774082,
+       "opt_05": 0.0013852100316049606,
+       "opt_06": 0.011178115917111658,
+       "opt_07": 0.00029511167680616805,
+       "opt_08": 0.0004583839921725969,
+       "opt_09": 0.0020450612369436612,
+       "opt_10": 0.00023553408808387122,
+       "opt_11": 0.002179399226743793,
+       "opt_12": 0.08319194619493558,
+       "opt_13": 0.0006099939564941258,
+       "opt_14": 2.804774644875688e-06,
+       "opt_15": 0.0013852100316049606,
+       "opt_16": 0.0019182902217256838,
+       "opt_17": 0.00020225765779985187,
+       "opt_18": 8.505929885961104e-05,
+       "opt_19": 6.133630205109804e-05,
+       "opt_20": 7.228823168183779e-05,
+       "opt_21": 6.87424501163473e-06,
+       "opt_22": 2.4146098193589926e-05,
+       "opt_23": 5.6465463159497865e-05,
+       "opt_24": 0.00042636537233369524,
+       "opt_25": 0.0004926153902416718
+      },
+      "choice": "opt_01"
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 1628.5290718078613,
+    "probabilities": {
+     "pick": {
+      "opt_00": 0.29997741714000264,
+      "opt_01": 0.5368288441971905,
+      "opt_02": 0.02522306219997332,
+      "opt_03": 0.017849704493855048,
+      "opt_04": 0.013808502560774082,
+      "opt_05": 0.0013852100316049606,
+      "opt_06": 0.011178115917111658,
+      "opt_07": 0.00029511167680616805,
+      "opt_08": 0.0004583839921725969,
+      "opt_09": 0.0020450612369436612,
+      "opt_10": 0.00023553408808387122,
+      "opt_11": 0.002179399226743793,
+      "opt_12": 0.08319194619493558,
+      "opt_13": 0.0006099939564941258,
+      "opt_14": 2.804774644875688e-06,
+      "opt_15": 0.0013852100316049606,
+      "opt_16": 0.0019182902217256838,
+      "opt_17": 0.00020225765779985187,
+      "opt_18": 8.505929885961104e-05,
+      "opt_19": 6.133630205109804e-05,
+      "opt_20": 7.228823168183779e-05,
+      "opt_21": 6.87424501163473e-06,
+      "opt_22": 2.4146098193589926e-05,
+      "opt_23": 5.6465463159497865e-05,
+      "opt_24": 0.00042636537233369524,
+      "opt_25": 0.0004926153902416718
+     }
+    },
+    "labels": {
+     "pick": "B"
+    }
+   }
+  },
+  "choice_27_error": {
+   "vertex_rollback": {
+    "status": 400,
+    "error": "question 'pick': at most 26 alternatives"
+   }
+  },
+  "depends_on_2reads": {
+   "vertex_rollback": {
+    "status": 200,
+    "top_keys": [
+     "choices",
+     "created",
+     "id",
+     "model",
+     "object",
+     "usage"
+    ],
+    "envelope_keys": [
+     "answers",
+     "diagnostics"
+    ],
+    "answers": {
+     "billing_issue": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9830926307941301,
+      "probabilities": {
+       "yes": 0.9830926307941301,
+       "no": 0.016907369205869847
+      },
+      "noul": 0.9830926307941301
+     },
+     "refund": {
+      "type": "noul",
+      "label": "yes",
+      "confidence": 0.9993759598430194,
+      "probabilities": {
+       "yes": 0.9993759598430194,
+       "no": 0.0006240401569805946
+      },
+      "noul": 0.9993759598430194
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 2,
+    "denoise_ms": 1578.8750648498535,
+    "probabilities": {
+     "billing_issue": {
+      "yes": 0.9830926307941301,
+      "no": 0.016907369205869847
+     },
+     "refund": {
+      "yes": 0.9993759598430194,
+      "no": 0.0006240401569805946
+     }
+    },
+    "labels": {
+     "billing_issue": "yes",
+     "refund": "yes"
+    }
+   }
+  },
+  "unknown_type_error": {
+   "vertex_rollback": {
+    "status": 400,
+    "error": "question 'a': unknown type 'banana'"
+   }
+  },
+  "empty_questions_error": {
+   "vertex_rollback": {
+    "status": 400,
+    "error": "schema: needs a non-empty questions array"
+   }
+  },
+  "systemone": {
+   "vertex_rollback": {
+    "status": 200,
+    "envelope_keys": [
+     "answers",
+     "diagnostics",
+     "model",
+     "usage"
+    ],
+    "answers": {
+     "urgent": {
+      "type": "noul",
+      "noul": 0.033254436506512774
+     },
+     "billing": {
+      "type": "noul",
+      "noul": 0.9979322375401428
+     }
+    },
+    "diagnostic_keys": [
+     "chunk_prompt",
+     "chunks",
+     "conditioning",
+     "engine",
+     "prompt_tokens",
+     "questions",
+     "samples",
+     "sequential",
+     "skipped",
+     "stages",
+     "steps",
+     "thought",
+     "timing"
+    ],
+    "reads": 1,
+    "denoise_ms": 94.05994415283203,
+    "probabilities": {
+     "urgent": null,
+     "billing": null
+    },
+    "labels": {
+     "urgent": null,
+     "billing": null
+    }
+   }
+  },
+  "invalid_json_error": {
+   "vertex_rollback": {
+    "status": 400,
+    "error": "invalid body: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)"
+   }
+  }
+ }
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/decision_index.json b/benchmarks/runs/20260928-v010-verification/decision_index.json
new file mode 100644
index 0000000..2c94152
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/decision_index.json
@@ -0,0 +1,2913 @@
+{
+  "timestamp": "2026-09-28T22:40:19Z",
+  "endpoint": "http://127.0.0.1:8080/v1",
+  "model": "diffgemma-26b-a4b-it-q4",
+  "dgem_wide_canvas_report": {
+    "suite_edition": "release-v1-panel",
+    "engine_mode": "dgem-wide-canvas+batching",
+    "temperature_scale": 1.25,
+    "total_requests": 22,
+    "supported_requests": 22,
+    "coverage_pct": 100,
+    "total_fields": 54,
+    "total_forward_passes": 28,
+    "headline_decision_index": 98.88888888888889,
+    "supported_decision_index": 98.88888888888889,
+    "skill_decision_index": 98.33341666250021,
+    "wide_option_accuracy": 100,
+    "multi_slot_canvas_accuracy": 94.28571428571429,
+    "ece_10bin": 0.03419909364691573,
+    "mean_brier": 0.017479579928377155,
+    "p50_latency_ms": 217,
+    "areas": [
+      {
+        "area_id": "knowledge",
+        "label": "Knowledge \u0026 Reasoning",
+        "benchmark_count": 6,
+        "coverage_pct": 100,
+        "plain_score": 100,
+        "supported_score": 100,
+        "skill_score": 100,
+        "benchmarks": [
+          {
+            "catalog_id": 1,
+            "dataset": "MMLU",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.00031808530128879016,
+            "mean_entropy": 0.09366624993609145,
+            "mean_normalized_entropy": 0.06756591714073266,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 2,
+            "dataset": "GPQA Diamond",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.010144805463675468,
+            "mean_entropy": 0.34174040382052917,
+            "mean_normalized_entropy": 0.2465135929316346,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 3,
+            "dataset": "GSM8K",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 20,
+            "skill_score": 100,
+            "mean_brier": 0.00011519615331485815,
+            "mean_entropy": 0.06037123574279207,
+            "mean_normalized_entropy": 0.037510757809530615,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 4,
+            "dataset": "ChessBench",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.0000025870106683533124,
+            "mean_entropy": 0.011206796682705465,
+            "mean_normalized_entropy": 0.008083994999195031,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 5,
+            "dataset": "CRUXEval",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.03585937563335616,
+            "mean_entropy": 0.51875200368341,
+            "mean_normalized_entropy": 0.3742004715826345,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 6,
+            "dataset": "CLadder",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 50,
+            "skill_score": 100,
+            "mean_brier": 0.000005091011376679519,
+            "mean_entropy": 0.011869921763818098,
+            "mean_normalized_entropy": 0.017124677264400347,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "language",
+        "label": "Language Understanding",
+        "benchmark_count": 3,
+        "coverage_pct": 100,
+        "plain_score": 94.44444444444446,
+        "supported_score": 94.44444444444446,
+        "skill_score": 91.66708331250105,
+        "benchmarks": [
+          {
+            "catalog_id": 7,
+            "dataset": "ContractNLI",
+            "area": "language",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 83.33333333333334,
+            "coverage_adj_score": 83.33333333333334,
+            "chance_baseline": 33.33,
+            "skill_score": 75.00124993750315,
+            "mean_brier": 0.22319738456721758,
+            "mean_entropy": 0.08223620580105517,
+            "mean_normalized_entropy": 0.0748546203690779,
+            "mean_passes_per_req": 2
+          },
+          {
+            "catalog_id": 8,
+            "dataset": "iSarcasmEval",
+            "area": "language",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.00003184284214067531,
+            "mean_entropy": 0.03572485030012276,
+            "mean_normalized_entropy": 0.025770032182243854,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 9,
+            "dataset": "VAST",
+            "area": "language",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 33.33,
+            "skill_score": 100,
+            "mean_brier": 0.00001824581387429913,
+            "mean_entropy": 0.025097271675774628,
+            "mean_normalized_entropy": 0.02284452116060073,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "retrieval",
+        "label": "Retrieval \u0026 Classification",
+        "benchmark_count": 2,
+        "coverage_pct": 100,
+        "plain_score": 100,
+        "supported_score": 100,
+        "skill_score": 100,
+        "benchmarks": [
+          {
+            "catalog_id": 10,
+            "dataset": "BRIGHT",
+            "area": "retrieval",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 11.200000000000001,
+            "skill_score": 100,
+            "mean_brier": 3.648203736699759e-7,
+            "mean_entropy": 0.002021309798232538,
+            "mean_normalized_entropy": 0.002916133622010354,
+            "mean_passes_per_req": 2
+          },
+          {
+            "catalog_id": 11,
+            "dataset": "Amazon ESCI",
+            "area": "retrieval",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.000004973925736865692,
+            "mean_entropy": 0.014611664542282533,
+            "mean_normalized_entropy": 0.010540087987142057,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "tools",
+        "label": "Tools \u0026 Automation",
+        "benchmark_count": 3,
+        "coverage_pct": 100,
+        "plain_score": 100,
+        "supported_score": 100,
+        "skill_score": 100,
+        "benchmarks": [
+          {
+            "catalog_id": 12,
+            "dataset": "BFCL",
+            "area": "tools",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25.919999999999998,
+            "skill_score": 100,
+            "mean_brier": 0.000028303132524838522,
+            "mean_entropy": 0.03344297579518745,
+            "mean_normalized_entropy": 0.024124007666143286,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 13,
+            "dataset": "ToolRet",
+            "area": "tools",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 9.370000000000001,
+            "skill_score": 100,
+            "mean_brier": 0.000003682404255355839,
+            "mean_entropy": 0.006292160127400224,
+            "mean_normalized_entropy": 0.009077668212279571,
+            "mean_passes_per_req": 2
+          },
+          {
+            "catalog_id": 14,
+            "dataset": "RouterBench",
+            "area": "tools",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 9.09,
+            "skill_score": 100,
+            "mean_brier": 0.000004425805859090248,
+            "mean_entropy": 0.01402819143826605,
+            "mean_normalized_entropy": 0.010119201110313723,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "arts",
+        "label": "Arts \u0026 Human Judgment",
+        "benchmark_count": 5,
+        "coverage_pct": 100,
+        "plain_score": 100,
+        "supported_score": 100,
+        "skill_score": 100,
+        "benchmarks": [
+          {
+            "catalog_id": 15,
+            "dataset": "BPoMP",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.000025089727133875345,
+            "mean_entropy": 0.03149919820540211,
+            "mean_normalized_entropy": 0.022721868521456082,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 16,
+            "dataset": "Humicroedit",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 50,
+            "skill_score": 100,
+            "mean_brier": 0.001990036227415131,
+            "mean_entropy": 0.14006865139125854,
+            "mean_normalized_entropy": 0.20207634874617372,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 17,
+            "dataset": "POP909-CL",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 12.5,
+            "skill_score": 100,
+            "mean_brier": 0.003170788798504395,
+            "mean_entropy": 0.21810580755465517,
+            "mean_normalized_entropy": 0.1573300834740918,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 18,
+            "dataset": "cfcolor",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 20,
+            "skill_score": 100,
+            "mean_brier": 0.00017122571029852359,
+            "mean_entropy": 0.0694548956320747,
+            "mean_normalized_entropy": 0.05010111674692735,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 19,
+            "dataset": "Habermas Machine",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 4.131786528919932e-7,
+            "mean_entropy": 0.004700562121164799,
+            "mean_normalized_entropy": 0.003390738830797481,
+            "mean_passes_per_req": 1
+          }
+        ]
+      }
+    ],
+    "display_benchmarks": [
+      {
+        "catalog_id": 20,
+        "dataset": "API-Bank",
+        "area": "tools",
+        "in_panel": false,
+        "total_cases": 1,
+        "supported_cases": 1,
+        "coverage_pct": 100,
+        "supported_score": 100,
+        "coverage_adj_score": 100,
+        "chance_baseline": 1.8900000000000001,
+        "skill_score": 100,
+        "mean_brier": 0.034646912648779,
+        "mean_entropy": 0.7739372750938567,
+        "mean_normalized_entropy": 0.2275484743304241,
+        "mean_passes_per_req": 2
+      },
+      {
+        "catalog_id": 21,
+        "dataset": "BANKING77",
+        "area": "retrieval",
+        "in_panel": false,
+        "total_cases": 1,
+        "supported_cases": 1,
+        "coverage_pct": 100,
+        "supported_score": 100,
+        "coverage_adj_score": 100,
+        "chance_baseline": 1.27,
+        "skill_score": 100,
+        "mean_brier": 0.036288909749248824,
+        "mean_entropy": 0.84620946713037,
+        "mean_normalized_entropy": 0.24416444035645543,
+        "mean_passes_per_req": 2
+      },
+      {
+        "catalog_id": 22,
+        "dataset": "CLINC150+OOS",
+        "area": "retrieval",
+        "in_panel": false,
+        "total_cases": 1,
+        "supported_cases": 1,
+        "coverage_pct": 100,
+        "supported_score": 100,
+        "coverage_adj_score": 100,
+        "chance_baseline": 0.66,
+        "skill_score": 100,
+        "mean_brier": 0.03852301849860215,
+        "mean_entropy": 0.8962220901622096,
+        "mean_normalized_entropy": 0.2500955361347496,
+        "mean_passes_per_req": 2
+      }
+    ],
+    "results": [
+      {
+        "id": "di_knowledge_mmlu_01",
+        "catalog_id": 1,
+        "dataset": "MMLU",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.984890336592688,
+        "mean_brier": 0.00031808530128879016,
+        "mean_entropy": 0.09366624993609145,
+        "mean_normalized_entropy": 0.06756591714073266,
+        "forward_passes": 1,
+        "wall_time_ms": 126,
+        "answers": {
+          "answer": {
+            "type": "choice",
+            "choice": "B",
+            "probabilities": {
+              "A": 0.008049626394809576,
+              "B": 0.984890336592688,
+              "C": 0.0037100554490711317,
+              "D": 0.003349981563431216
+            },
+            "entropy": 0.09366624993609145,
+            "normalized_entropy": 0.06756591714073266,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "answer": "B"
+        }
+      },
+      {
+        "id": "di_knowledge_gpqa_01",
+        "catalog_id": 2,
+        "dataset": "GPQA Diamond",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9195671452372348,
+        "mean_brier": 0.010144805463675468,
+        "mean_entropy": 0.34174040382052917,
+        "mean_normalized_entropy": 0.2465135929316346,
+        "forward_passes": 1,
+        "wall_time_ms": 151,
+        "answers": {
+          "answer": {
+            "type": "choice",
+            "choice": "B",
+            "probabilities": {
+              "A": 0.058560882344798254,
+              "B": 0.9195671452372348,
+              "C": 0.012778919829590969,
+              "D": 0.00909305258837602
+            },
+            "entropy": 0.34174040382052917,
+            "normalized_entropy": 0.2465135929316346,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "answer": "B"
+        }
+      },
+      {
+        "id": "di_knowledge_gsm8k_01",
+        "catalog_id": 3,
+        "dataset": "GSM8K",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 5,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9910411390868156,
+        "mean_brier": 0.00011519615331485815,
+        "mean_entropy": 0.06037123574279207,
+        "mean_normalized_entropy": 0.037510757809530615,
+        "forward_passes": 1,
+        "wall_time_ms": 144,
+        "answers": {
+          "answer": {
+            "type": "choice",
+            "choice": "opt_30_00",
+            "probabilities": {
+              "opt_15_00": 0.005325174595336004,
+              "opt_30_00": 0.9910411390868156,
+              "opt_45_00": 0.0005900918579673167,
+              "opt_60_00": 0.0024155359958317924,
+              "opt_7_50": 0.0006280584640493413
+            },
+            "entropy": 0.06037123574279207,
+            "normalized_entropy": 0.037510757809530615,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "answer": "opt_30_00"
+        }
+      },
+      {
+        "id": "di_knowledge_chessbench_01",
+        "catalog_id": 4,
+        "dataset": "ChessBench",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9986826042374531,
+        "mean_brier": 0.0000025870106683533124,
+        "mean_entropy": 0.011206796682705465,
+        "mean_normalized_entropy": 0.008083994999195031,
+        "forward_passes": 1,
+        "wall_time_ms": 224,
+        "answers": {
+          "best_move": {
+            "type": "choice",
+            "choice": "Qxf7#",
+            "probabilities": {
+              "Bxf7+": 0.00028251422567836135,
+              "Qg5": 0.00017380223174887297,
+              "Qh6": 0.0008610793051196632,
+              "Qxf7#": 0.9986826042374531
+            },
+            "entropy": 0.011206796682705465,
+            "normalized_entropy": 0.008083994999195031,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "best_move": "Qxf7#"
+        }
+      },
+      {
+        "id": "di_knowledge_cruxeval_01",
+        "catalog_id": 5,
+        "dataset": "CRUXEval",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.8519781342562239,
+        "mean_brier": 0.03585937563335616,
+        "mean_entropy": 0.51875200368341,
+        "mean_normalized_entropy": 0.3742004715826345,
+        "forward_passes": 1,
+        "wall_time_ms": 168,
+        "answers": {
+          "output": {
+            "type": "choice",
+            "choice": "B",
+            "probabilities": {
+              "A": 0.1158531161231781,
+              "B": 0.8519781342562239,
+              "C": 0.018268883171548234,
+              "D": 0.013899866449049949
+            },
+            "entropy": 0.51875200368341,
+            "normalized_entropy": 0.3742004715826345,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "output": "B"
+        }
+      },
+      {
+        "id": "di_knowledge_cladder_01",
+        "catalog_id": 6,
+        "dataset": "CLadder",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 2,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9984045359018957,
+        "mean_brier": 0.000005091011376679519,
+        "mean_entropy": 0.011869921763818098,
+        "mean_normalized_entropy": 0.017124677264400347,
+        "forward_passes": 1,
+        "wall_time_ms": 217,
+        "answers": {
+          "causal_answer": {
+            "type": "choice",
+            "choice": "yes",
+            "probabilities": {
+              "no": 0.0015954640981043018,
+              "yes": 0.9984045359018957
+            },
+            "entropy": 0.011869921763818098,
+            "normalized_entropy": 0.017124677264400347,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "causal_answer": "yes"
+        }
+      },
+      {
+        "id": "di_language_contractnli_12slots",
+        "catalog_id": 7,
+        "dataset": "ContractNLI",
+        "area": "language",
+        "total_fields": 12,
+        "correct_fields": 10,
+        "max_options": 3,
+        "whole_case_exact": false,
+        "field_accuracy": 0.8333333333333334,
+        "supported": true,
+        "mean_confidence": 0.9691846956043432,
+        "mean_brier": 0.22319738456721758,
+        "mean_entropy": 0.08223620580105517,
+        "mean_normalized_entropy": 0.0748546203690779,
+        "forward_passes": 2,
+        "wall_time_ms": 293,
+        "answers": {
+          "clause_01_confidential_marking": {
+            "type": "choice",
+            "choice": "contradiction",
+            "probabilities": {
+              "contradiction": 0.9999996697144815,
+              "entailment": 8.618604766313014e-8,
+              "not_mentioned": 2.4409947079087e-7
+            },
+            "entropy": 0.0000054488358821123155,
+            "normalized_entropy": 0.000004959744159350476,
+            "passes_used": 1
+          },
+          "clause_02_sharing_employees": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.000041538860322816976,
+              "entailment": 0.9999404281010583,
+              "not_mentioned": 0.000018033038618877185
+            },
+            "entropy": 0.0006756311314425542,
+            "normalized_entropy": 0.0006149859585692858,
+            "passes_used": 1
+          },
+          "clause_03_sharing_affiliates": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.000014963257023359234,
+              "entailment": 0.9999823358467419,
+              "not_mentioned": 0.000002700896234802674
+            },
+            "entropy": 0.000218535173471249,
+            "normalized_entropy": 0.00019891928729123148,
+            "passes_used": 1
+          },
+          "clause_04_return_or_destroy": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.00005096878559474608,
+              "entailment": 0.9999365077131019,
+              "not_mentioned": 0.000012523501303323344
+            },
+            "entropy": 0.0007086449690396565,
+            "normalized_entropy": 0.0006450364485716561,
+            "passes_used": 1
+          },
+          "clause_05_survival_period": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.00003571487426346517,
+              "entailment": 0.9999567078095227,
+              "not_mentioned": 0.000007577316213686362
+            },
+            "entropy": 0.0004983487625891229,
+            "normalized_entropy": 0.0004536165922495647,
+            "passes_used": 1
+          },
+          "clause_06_no_license": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.00006907117585366292,
+              "entailment": 0.9999248034860468,
+              "not_mentioned": 0.000006125338099321152
+            },
+            "entropy": 0.0008104442207947911,
+            "normalized_entropy": 0.0007376981207604406,
+            "passes_used": 1
+          },
+          "clause_07_reverse_engineering": {
+            "type": "choice",
+            "choice": "contradiction",
+            "probabilities": {
+              "contradiction": 0.999940416855388,
+              "entailment": 0.000052558070271802755,
+              "not_mentioned": 0.000007025074340254103
+            },
+            "entropy": 0.0006608268510367453,
+            "normalized_entropy": 0.0006015105218219354,
+            "passes_used": 1
+          },
+          "clause_08_public_domain_exception": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.0001391161015795496,
+              "entailment": 0.999721767796841,
+              "not_mentioned": 0.0001391161015795496
+            },
+            "entropy": 0.002748951579345907,
+            "normalized_entropy": 0.002502203559618442,
+            "passes_used": 1
+          },
+          "clause_09_compelled_disclosure": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.0002377308755064826,
+              "entailment": 0.9997101450195087,
+              "not_mentioned": 0.00005212410498460579
+            },
+            "entropy": 0.002787569490375728,
+            "normalized_entropy": 0.0025373550970881702,
+            "passes_used": 1
+          },
+          "clause_10_non_solicitation": {
+            "type": "choice",
+            "choice": "contradiction",
+            "probabilities": {
+              "contradiction": 0.7758966993768991,
+              "entailment": 0.001690489903369102,
+              "not_mentioned": 0.22241281071973187
+            },
+            "entropy": 0.5419982007055449,
+            "normalized_entropy": 0.4933480230433527,
+            "passes_used": 1
+          },
+          "clause_11_exclusive_dealing": {
+            "type": "choice",
+            "choice": "contradiction",
+            "probabilities": {
+              "contradiction": 0.8561744991646,
+              "entailment": 0.0032793370912873204,
+              "not_mentioned": 0.14054616374411275
+            },
+            "entropy": 0.427488265570396,
+            "normalized_entropy": 0.38911658824484535,
+            "passes_used": 1
+          },
+          "clause_12_injunctive_relief": {
+            "type": "choice",
+            "choice": "entailment",
+            "probabilities": {
+              "contradiction": 0.0007199161256425151,
+              "entailment": 0.9990323663679275,
+              "not_mentioned": 0.0002477175064299172
+            },
+            "entropy": 0.008233602322743233,
+            "normalized_entropy": 0.0074945478106067335,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "clause_01_confidential_marking": "contradiction",
+          "clause_02_sharing_employees": "entailment",
+          "clause_03_sharing_affiliates": "entailment",
+          "clause_04_return_or_destroy": "entailment",
+          "clause_05_survival_period": "entailment",
+          "clause_06_no_license": "entailment",
+          "clause_07_reverse_engineering": "contradiction",
+          "clause_08_public_domain_exception": "entailment",
+          "clause_09_compelled_disclosure": "entailment",
+          "clause_10_non_solicitation": "not_mentioned",
+          "clause_11_exclusive_dealing": "not_mentioned",
+          "clause_12_injunctive_relief": "entailment"
+        }
+      },
+      {
+        "id": "di_language_isarcasm_01",
+        "catalog_id": 8,
+        "dataset": "iSarcasmEval",
+        "area": "language",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9951625133540951,
+        "mean_brier": 0.00003184284214067531,
+        "mean_entropy": 0.03572485030012276,
+        "mean_normalized_entropy": 0.025770032182243854,
+        "forward_passes": 1,
+        "wall_time_ms": 316,
+        "answers": {
+          "tone": {
+            "type": "choice",
+            "choice": "sarcasm",
+            "probabilities": {
+              "literal_happy": 0.002156793640348561,
+              "neutral_fact": 0.0010266995760436232,
+              "sarcasm": 0.9951625133540951,
+              "understatement": 0.0016539934295126573
+            },
+            "entropy": 0.03572485030012276,
+            "normalized_entropy": 0.025770032182243854,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "tone": "sarcasm"
+        }
+      },
+      {
+        "id": "di_language_vast_01",
+        "catalog_id": 9,
+        "dataset": "VAST",
+        "area": "language",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 3,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.996569232361649,
+        "mean_brier": 0.00001824581387429913,
+        "mean_entropy": 0.025097271675774628,
+        "mean_normalized_entropy": 0.02284452116060073,
+        "forward_passes": 1,
+        "wall_time_ms": 183,
+        "answers": {
+          "stance": {
+            "type": "choice",
+            "choice": "pro",
+            "probabilities": {
+              "con": 0.002258782379977801,
+              "neutral": 0.0011719852583732353,
+              "pro": 0.996569232361649
+            },
+            "entropy": 0.025097271675774628,
+            "normalized_entropy": 0.02284452116060073,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "stance": "pro"
+        }
+      },
+      {
+        "id": "di_retrieval_bright_11slots",
+        "catalog_id": 10,
+        "dataset": "BRIGHT",
+        "area": "retrieval",
+        "total_fields": 11,
+        "correct_fields": 11,
+        "max_options": 2,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9997581912472026,
+        "mean_brier": 3.648203736699759e-7,
+        "mean_entropy": 0.002021309798232538,
+        "mean_normalized_entropy": 0.002916133622010354,
+        "forward_passes": 2,
+        "wall_time_ms": 437,
+        "answers": {
+          "passage_01": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9999990652834234,
+              "relevant": 9.347165767063085e-7
+            },
+            "entropy": 0.000013911407386697357,
+            "normalized_entropy": 0.000020069918448574373,
+            "passes_used": 1
+          },
+          "passage_02": {
+            "type": "choice",
+            "choice": "relevant",
+            "probabilities": {
+              "not_relevant": 0.00011683087681758423,
+              "relevant": 0.9998831691231823
+            },
+            "entropy": 0.0011747023085242202,
+            "normalized_entropy": 0.0016947371950287095,
+            "passes_used": 1
+          },
+          "passage_03": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9999989130578817,
+              "relevant": 0.0000010869421183576737
+            },
+            "entropy": 0.000016012985260289724,
+            "normalized_entropy": 0.000023101854424848054,
+            "passes_used": 1
+          },
+          "passage_04": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9999967477721418,
+              "relevant": 0.0000032522278581837736
+            },
+            "entropy": 0.00004434792764549435,
+            "normalized_entropy": 0.00006398053528785726,
+            "passes_used": 1
+          },
+          "passage_05": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9999987568422682,
+              "relevant": 0.000001243157731815019
+            },
+            "entropy": 0.000018147436604288555,
+            "normalized_entropy": 0.00002618121679385395,
+            "passes_used": 1
+          },
+          "passage_06": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.999998817309308,
+              "relevant": 0.0000011826906920004034
+            },
+            "entropy": 0.000017323719591634076,
+            "normalized_entropy": 0.00002499284434460146,
+            "passes_used": 1
+          },
+          "passage_07": {
+            "type": "choice",
+            "choice": "relevant",
+            "probabilities": {
+              "not_relevant": 0.0006268492319647771,
+              "relevant": 0.9993731507680352
+            },
+            "entropy": 0.00524954326116595,
+            "normalized_entropy": 0.0075734900298161925,
+            "passes_used": 1
+          },
+          "passage_08": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9999898159464822,
+              "relevant": 0.000010184053517730875
+            },
+            "entropy": 0.0001272465137354613,
+            "normalized_entropy": 0.0001835779143365594,
+            "passes_used": 1
+          },
+          "passage_09": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9988675412923451,
+              "relevant": 0.0011324587076547885
+            },
+            "entropy": 0.00881369704712029,
+            "normalized_entropy": 0.012715477021778145,
+            "passes_used": 1
+          },
+          "passage_10": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9997267431108126,
+              "relevant": 0.0002732568891874791
+            },
+            "entropy": 0.0025153191659773753,
+            "normalized_entropy": 0.003628838487008523,
+            "passes_used": 1
+          },
+          "passage_11": {
+            "type": "choice",
+            "choice": "not_relevant",
+            "probabilities": {
+              "not_relevant": 0.9995073832133478,
+              "relevant": 0.000492616786652353
+            },
+            "entropy": 0.004244156007546215,
+            "normalized_entropy": 0.006123022824846027,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "passage_01": "not_relevant",
+          "passage_02": "relevant",
+          "passage_03": "not_relevant",
+          "passage_04": "not_relevant",
+          "passage_05": "not_relevant",
+          "passage_06": "not_relevant",
+          "passage_07": "relevant",
+          "passage_08": "not_relevant",
+          "passage_09": "not_relevant",
+          "passage_10": "not_relevant",
+          "passage_11": "not_relevant"
+        }
+      },
+      {
+        "id": "di_retrieval_esci_01",
+        "catalog_id": 11,
+        "dataset": "Amazon ESCI",
+        "area": "retrieval",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9982022627113681,
+        "mean_brier": 0.000004973925736865692,
+        "mean_entropy": 0.014611664542282533,
+        "mean_normalized_entropy": 0.010540087987142057,
+        "forward_passes": 1,
+        "wall_time_ms": 123,
+        "answers": {
+          "esci_label": {
+            "type": "choice",
+            "choice": "Exact",
+            "probabilities": {
+              "Complement": 0.0012603098124193191,
+              "Exact": 0.9982022627113681,
+              "Irrelevant": 0.00020062773163739565,
+              "Substitute": 0.0003367997445751219
+            },
+            "entropy": 0.014611664542282533,
+            "normalized_entropy": 0.010540087987142057,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "esci_label": "Exact"
+        }
+      },
+      {
+        "id": "di_tools_bfcl_01",
+        "catalog_id": 12,
+        "dataset": "BFCL",
+        "area": "tools",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.995484492546379,
+        "mean_brier": 0.000028303132524838522,
+        "mean_entropy": 0.03344297579518745,
+        "mean_normalized_entropy": 0.024124007666143286,
+        "forward_passes": 1,
+        "wall_time_ms": 183,
+        "answers": {
+          "function_call": {
+            "type": "choice",
+            "choice": "get_equity_quote",
+            "probabilities": {
+              "cancel_subscription": 0.0006957056223265837,
+              "execute_wire_transfer": 0.0021686243519798967,
+              "get_equity_quote": 0.995484492546379,
+              "get_weather_forecast": 0.001651177479314425
+            },
+            "entropy": 0.03344297579518745,
+            "normalized_entropy": 0.024124007666143286,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "function_call": "get_equity_quote"
+        }
+      },
+      {
+        "id": "di_tools_toolret_12slots",
+        "catalog_id": 13,
+        "dataset": "ToolRet",
+        "area": "tools",
+        "total_fields": 12,
+        "correct_fields": 12,
+        "max_options": 2,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9991485698750305,
+        "mean_brier": 0.000003682404255355839,
+        "mean_entropy": 0.006292160127400224,
+        "mean_normalized_entropy": 0.009077668212279571,
+        "forward_passes": 2,
+        "wall_time_ms": 420,
+        "answers": {
+          "tool_candidate_01": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.00010803509425130201,
+              "skip": 0.9998919649057488
+            },
+            "entropy": 0.0010947196551378384,
+            "normalized_entropy": 0.0015793466176310356,
+            "passes_used": 1
+          },
+          "tool_candidate_02": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.0003008561984064008,
+              "skip": 0.9996991438015936
+            },
+            "entropy": 0.00274041719125291,
+            "normalized_entropy": 0.003953586291787436,
+            "passes_used": 1
+          },
+          "tool_candidate_03": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.00009907571225545651,
+              "skip": 0.9999009242877446
+            },
+            "entropy": 0.0010125118395474494,
+            "normalized_entropy": 0.001460745809756467,
+            "passes_used": 1
+          },
+          "tool_candidate_04": {
+            "type": "choice",
+            "choice": "invoke",
+            "probabilities": {
+              "invoke": 0.999938510286779,
+              "skip": 0.00006148971322090415
+            },
+            "entropy": 0.0006577314761926615,
+            "normalized_entropy": 0.0009489059389397301,
+            "passes_used": 1
+          },
+          "tool_candidate_05": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.00008757531332327649,
+              "skip": 0.9999124246866768
+            },
+            "entropy": 0.0009057886302049136,
+            "normalized_entropy": 0.001306776764890236,
+            "passes_used": 1
+          },
+          "tool_candidate_06": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.00020379856326541108,
+              "skip": 0.9997962014367346
+            },
+            "entropy": 0.00193573512064899,
+            "normalized_entropy": 0.0027926754590348974,
+            "passes_used": 1
+          },
+          "tool_candidate_07": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.00026784514524786436,
+              "skip": 0.9997321548547522
+            },
+            "entropy": 0.0024708627937360932,
+            "normalized_entropy": 0.0035647014992401115,
+            "passes_used": 1
+          },
+          "tool_candidate_08": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.00023721094970427783,
+              "skip": 0.9997627890502958
+            },
+            "entropy": 0.00221707841038958,
+            "normalized_entropy": 0.003198568027931033,
+            "passes_used": 1
+          },
+          "tool_candidate_09": {
+            "type": "choice",
+            "choice": "invoke",
+            "probabilities": {
+              "invoke": 0.9986223348900516,
+              "skip": 0.0013776651099484847
+            },
+            "entropy": 0.010451898843577407,
+            "normalized_entropy": 0.015078902629502218,
+            "passes_used": 1
+          },
+          "tool_candidate_10": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.001878958101334173,
+              "skip": 0.9981210418986658
+            },
+            "entropy": 0.013671482887054323,
+            "normalized_entropy": 0.019723780562751602,
+            "passes_used": 1
+          },
+          "tool_candidate_11": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.0021961098591722007,
+              "skip": 0.9978038901408278
+            },
+            "entropy": 0.01563623383243009,
+            "normalized_entropy": 0.02255831700822712,
+            "passes_used": 1
+          },
+          "tool_candidate_12": {
+            "type": "choice",
+            "choice": "skip",
+            "probabilities": {
+              "invoke": 0.003398541739504947,
+              "skip": 0.996601458260495
+            },
+            "entropy": 0.022711460848630426,
+            "normalized_entropy": 0.032765711937662964,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "tool_candidate_01": "skip",
+          "tool_candidate_02": "skip",
+          "tool_candidate_03": "skip",
+          "tool_candidate_04": "invoke",
+          "tool_candidate_05": "skip",
+          "tool_candidate_06": "skip",
+          "tool_candidate_07": "skip",
+          "tool_candidate_08": "skip",
+          "tool_candidate_09": "invoke",
+          "tool_candidate_10": "skip",
+          "tool_candidate_11": "skip",
+          "tool_candidate_12": "skip"
+        }
+      },
+      {
+        "id": "di_tools_routerbench_01",
+        "catalog_id": 14,
+        "dataset": "RouterBench",
+        "area": "tools",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9982915615380495,
+        "mean_brier": 0.000004425805859090248,
+        "mean_entropy": 0.01402819143826605,
+        "mean_normalized_entropy": 0.010119201110313723,
+        "forward_passes": 1,
+        "wall_time_ms": 222,
+        "answers": {
+          "target_model": {
+            "type": "choice",
+            "choice": "dgemma-vllm-l4",
+            "probabilities": {
+              "claude-3-7-opus": 0.0011614866068420742,
+              "dgemma-vllm-l4": 0.9982915615380495,
+              "gemini-2.5-pro": 0.00020861231278486073,
+              "gpt-4.5-preview": 0.0003383395423234517
+            },
+            "entropy": 0.01402819143826605,
+            "normalized_entropy": 0.010119201110313723,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "target_model": "dgemma-vllm-l4"
+        }
+      },
+      {
+        "id": "di_arts_bpomp_01",
+        "catalog_id": 15,
+        "dataset": "BPoMP",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9957772169079427,
+        "mean_brier": 0.000025089727133875345,
+        "mean_entropy": 0.03149919820540211,
+        "mean_normalized_entropy": 0.022721868521456082,
+        "forward_passes": 1,
+        "wall_time_ms": 204,
+        "answers": {
+          "mood": {
+            "type": "choice",
+            "choice": "melancholic_somber",
+            "probabilities": {
+              "euphoric_festive": 0.0023334052348670735,
+              "melancholic_somber": 0.9957772169079427,
+              "playful_whimsical": 0.0010633812990020395,
+              "triumphant_march": 0.0008259965581879597
+            },
+            "entropy": 0.03149919820540211,
+            "normalized_entropy": 0.022721868521456082,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "mood": "melancholic_somber"
+        }
+      },
+      {
+        "id": "di_arts_humicroedit_01",
+        "catalog_id": 16,
+        "dataset": "Humicroedit",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 2,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9684560922885644,
+        "mean_brier": 0.001990036227415131,
+        "mean_entropy": 0.14006865139125854,
+        "mean_normalized_entropy": 0.20207634874617372,
+        "forward_passes": 1,
+        "wall_time_ms": 105,
+        "answers": {
+          "funnier_edit": {
+            "type": "choice",
+            "choice": "Edit_A",
+            "probabilities": {
+              "Edit_A": 0.9684560922885644,
+              "Edit_B": 0.031543907711435615
+            },
+            "entropy": 0.14006865139125854,
+            "normalized_entropy": 0.20207634874617372,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "funnier_edit": "Edit_A"
+        }
+      },
+      {
+        "id": "di_arts_pop909_01",
+        "catalog_id": 17,
+        "dataset": "POP909-CL",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9544621455187181,
+        "mean_brier": 0.003170788798504395,
+        "mean_entropy": 0.21810580755465517,
+        "mean_normalized_entropy": 0.1573300834740918,
+        "forward_passes": 1,
+        "wall_time_ms": 106,
+        "answers": {
+          "section_type": {
+            "type": "choice",
+            "choice": "chorus",
+            "probabilities": {
+              "chorus": 0.9544621455187181,
+              "intro": 0.02932567685548618,
+              "outro_fade": 0.0008369783780365701,
+              "verse": 0.01537519924775907
+            },
+            "entropy": 0.21810580755465517,
+            "normalized_entropy": 0.1573300834740918,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "section_type": "chorus"
+        }
+      },
+      {
+        "id": "di_arts_cfcolor_01",
+        "catalog_id": 18,
+        "dataset": "cfcolor",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.989185035990081,
+        "mean_brier": 0.00017122571029852359,
+        "mean_entropy": 0.0694548956320747,
+        "mean_normalized_entropy": 0.05010111674692735,
+        "forward_passes": 1,
+        "wall_time_ms": 195,
+        "answers": {
+          "palette": {
+            "type": "choice",
+            "choice": "crimson_alert_red",
+            "probabilities": {
+              "baby_sky_blue": 0.006634292201524307,
+              "crimson_alert_red": 0.989185035990081,
+              "muted_slate_gray": 0.001221594756696922,
+              "pastel_mint_green": 0.002959077051697847
+            },
+            "entropy": 0.0694548956320747,
+            "normalized_entropy": 0.05010111674692735,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "palette": "crimson_alert_red"
+        }
+      },
+      {
+        "id": "di_arts_habermas_01",
+        "catalog_id": 19,
+        "dataset": "Habermas Machine",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9994936026099215,
+        "mean_brier": 4.131786528919932e-7,
+        "mean_entropy": 0.004700562121164799,
+        "mean_normalized_entropy": 0.003390738830797481,
+        "forward_passes": 1,
+        "wall_time_ms": 256,
+        "answers": {
+          "consensus_statement": {
+            "type": "choice",
+            "choice": "Statement_1",
+            "probabilities": {
+              "Statement_1": 0.9994936026099215,
+              "Statement_2": 0.0003845470694486725,
+              "Statement_3": 0.00008775925498287552,
+              "Statement_4": 0.00003409106564700237
+            },
+            "entropy": 0.004700562121164799,
+            "normalized_entropy": 0.003390738830797481,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "consensus_statement": "Statement_1"
+        }
+      },
+      {
+        "id": "di_wide_apibank_30opts",
+        "catalog_id": 20,
+        "dataset": "API-Bank",
+        "area": "tools",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 30,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.8311449079470515,
+        "mean_brier": 0.034646912648779,
+        "mean_entropy": 0.7739372750938567,
+        "mean_normalized_entropy": 0.2275484743304241,
+        "forward_passes": 2,
+        "wall_time_ms": 496,
+        "answers": {
+          "selected_api": {
+            "type": "choice",
+            "choice": "ConvertCurrencyFX_28",
+            "probabilities": {
+              "ConvertCurrencyFX_28": 0.8311449079470515,
+              "api_tool_01": 0.0017061001966229116,
+              "api_tool_02": 0.0001894593007438493,
+              "api_tool_03": 0.0003368927885236905,
+              "api_tool_04": 0.0002301928824493389,
+              "api_tool_05": 0.00011113580300163345,
+              "api_tool_06": 0.00014547864879905093,
+              "api_tool_07": 0.00011113580300163345,
+              "api_tool_08": 0.00003868222856435114,
+              "api_tool_09": 0.0011040884254357752,
+              "api_tool_10": 0.00012723461573476498,
+              "api_tool_11": 0.00008446577906581872,
+              "api_tool_12": 0.000023067764391002488,
+              "api_tool_13": 0.00007349540856103,
+              "api_tool_14": 0.000005927877202876577,
+              "api_tool_15": 0.000024859199358687114,
+              "api_tool_16": 0.00020223059478397475,
+              "api_tool_17": 0.000018397035158879572,
+              "api_tool_18": 0.000023067764391002488,
+              "api_tool_19": 0.000023067764391002488,
+              "api_tool_20": 0.025350330246865015,
+              "api_tool_21": 0.0001490093745538199,
+              "api_tool_22": 0.008040392123920537,
+              "api_tool_23": 0.005571796869810214,
+              "api_tool_24": 0.007251386096726746,
+              "api_tool_25": 0.025350330246865015,
+              "api_tool_26": 0.008904616353012507,
+              "api_tool_27": 0.017608063367164507,
+              "api_tool_29": 0.06563025403955443,
+              "api_tool_30": 0.0004199334542943012
+            },
+            "entropy": 0.7739372750938567,
+            "normalized_entropy": 0.2275484743304241,
+            "passes_used": 2
+          }
+        },
+        "expected": {
+          "selected_api": "ConvertCurrencyFX_28"
+        }
+      },
+      {
+        "id": "di_wide_banking77_32opts",
+        "catalog_id": 21,
+        "dataset": "BANKING77",
+        "area": "retrieval",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 32,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.8243953404951396,
+        "mean_brier": 0.036288909749248824,
+        "mean_entropy": 0.84620946713037,
+        "mean_normalized_entropy": 0.24416444035645543,
+        "forward_passes": 2,
+        "wall_time_ms": 460,
+        "answers": {
+          "intent": {
+            "type": "choice",
+            "choice": "lost_or_stolen_card",
+            "probabilities": {
+              "banking_intent_01": 0.00046070049283893457,
+              "banking_intent_02": 0.012809611945265916,
+              "banking_intent_03": 0.000390192058010809,
+              "banking_intent_04": 0.0058220037636351595,
+              "banking_intent_05": 0.04814405714277091,
+              "banking_intent_06": 0.0008961033717843241,
+              "banking_intent_07": 0.004979686941726523,
+              "banking_intent_08": 0.0009531521328235439,
+              "banking_intent_09": 0.00027934531528036005,
+              "banking_intent_10": 0.0067885020579019845,
+              "banking_intent_11": 0.0011448472256890302,
+              "banking_intent_12": 0.04814405714277091,
+              "banking_intent_13": 0.014728862054474294,
+              "banking_intent_14": 0.017672658923490306,
+              "banking_intent_15": 0.00024384383267362253,
+              "banking_intent_16": 0.0005765655172005936,
+              "banking_intent_17": 0.0017363430575718347,
+              "banking_intent_18": 0.00009085607932030132,
+              "banking_intent_19": 0.00013977540738608518,
+              "banking_intent_20": 0.0017359767790745876,
+              "banking_intent_21": 0.00199569323439095,
+              "banking_intent_22": 0.00012938917727573423,
+              "banking_intent_23": 0.00034409331756265907,
+              "banking_intent_24": 0.0004390503348369017,
+              "banking_intent_25": 0.0008848689588367163,
+              "banking_intent_26": 0.000195185960656731,
+              "banking_intent_27": 0.0008848689588367163,
+              "banking_intent_28": 0.0007040477074381194,
+              "banking_intent_30": 0.0006644221086587563,
+              "banking_intent_31": 0.0007897892707995008,
+              "banking_intent_32": 0.0008361092338775065,
+              "lost_or_stolen_card": 0.8243953404951396
+            },
+            "entropy": 0.84620946713037,
+            "normalized_entropy": 0.24416444035645543,
+            "passes_used": 2
+          }
+        },
+        "expected": {
+          "intent": "lost_or_stolen_card"
+        }
+      },
+      {
+        "id": "di_wide_clinc150_36opts",
+        "catalog_id": 22,
+        "dataset": "CLINC150+OOS",
+        "area": "retrieval",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 36,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.8216735167933408,
+        "mean_brier": 0.03852301849860215,
+        "mean_entropy": 0.8962220901622096,
+        "mean_normalized_entropy": 0.2500955361347496,
+        "forward_passes": 2,
+        "wall_time_ms": 579,
+        "answers": {
+          "intent": {
+            "type": "choice",
+            "choice": "oos_out_of_scope",
+            "probabilities": {
+              "clinc_domain_intent_01": 0.010586429577636822,
+              "clinc_domain_intent_02": 0.012548697780862997,
+              "clinc_domain_intent_03": 0.00704184574631996,
+              "clinc_domain_intent_04": 0.0016534614033536855,
+              "clinc_domain_intent_05": 0.0016534614033536855,
+              "clinc_domain_intent_06": 0.00201669694772998,
+              "clinc_domain_intent_07": 0.00201669694772998,
+              "clinc_domain_intent_08": 0.0017671888714285847,
+              "clinc_domain_intent_09": 0.00047274870319007523,
+              "clinc_domain_intent_10": 0.00201669694772998,
+              "clinc_domain_intent_11": 0.00704184574631996,
+              "clinc_domain_intent_12": 0.00014814357195795284,
+              "clinc_domain_intent_13": 0.0024526416042930797,
+              "clinc_domain_intent_14": 0.007471203570368256,
+              "clinc_domain_intent_15": 0.0781145839088649,
+              "clinc_domain_intent_16": 0.00040399014406311095,
+              "clinc_domain_intent_17": 0.002974238129181072,
+              "clinc_domain_intent_18": 0.00704184574631996,
+              "clinc_domain_intent_19": 0.0024526416042930797,
+              "clinc_domain_intent_20": 0.00040399014406311095,
+              "clinc_domain_intent_21": 0.0050517812451500044,
+              "clinc_domain_intent_22": 0.001886724289036764,
+              "clinc_domain_intent_23": 0.0006018548162465257,
+              "clinc_domain_intent_24": 0.00032695227781062215,
+              "clinc_domain_intent_25": 0.00130405314204184,
+              "clinc_domain_intent_26": 0.0007827993104295481,
+              "clinc_domain_intent_27": 0.0008353007343675143,
+              "clinc_domain_intent_28": 0.0016702804710026463,
+              "clinc_domain_intent_29": 0.00032695227781062215,
+              "clinc_domain_intent_30": 0.004520156756034814,
+              "clinc_domain_intent_31": 0.004039483608847143,
+              "clinc_domain_intent_32": 0.0004301872822292004,
+              "clinc_domain_intent_34": 0.00046036517414244646,
+              "clinc_domain_intent_35": 0.005638972344056921,
+              "clinc_domain_intent_36": 0.00017157097839232228,
+              "oos_out_of_scope": 0.8216735167933408
+            },
+            "entropy": 0.8962220901622096,
+            "normalized_entropy": 0.2500955361347496,
+            "passes_used": 2
+          }
+        },
+        "expected": {
+          "intent": "oos_out_of_scope"
+        }
+      }
+    ]
+  },
+  "naive_26opt_10slot_report": {
+    "suite_edition": "release-v1-panel",
+    "engine_mode": "naive-djev-26opt-10slot",
+    "temperature_scale": 1,
+    "total_requests": 22,
+    "supported_requests": 16,
+    "coverage_pct": 72.72727272727273,
+    "total_fields": 54,
+    "total_forward_passes": 16,
+    "headline_decision_index": 76.66666666666667,
+    "supported_decision_index": 76.66666666666667,
+    "skill_decision_index": 76.66666666666667,
+    "wide_option_accuracy": 0,
+    "multi_slot_canvas_accuracy": 0,
+    "ece_10bin": 0.022771996803807495,
+    "mean_brier": 0.27508593116981,
+    "p50_latency_ms": 183,
+    "areas": [
+      {
+        "area_id": "knowledge",
+        "label": "Knowledge \u0026 Reasoning",
+        "benchmark_count": 6,
+        "coverage_pct": 100,
+        "plain_score": 100,
+        "supported_score": 100,
+        "skill_score": 100,
+        "benchmarks": [
+          {
+            "catalog_id": 1,
+            "dataset": "MMLU",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.00031808530128879016,
+            "mean_entropy": 0.09366624993609145,
+            "mean_normalized_entropy": 0.06756591714073266,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 2,
+            "dataset": "GPQA Diamond",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.010144805463675468,
+            "mean_entropy": 0.34174040382052917,
+            "mean_normalized_entropy": 0.2465135929316346,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 3,
+            "dataset": "GSM8K",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 20,
+            "skill_score": 100,
+            "mean_brier": 0.00011519615331485815,
+            "mean_entropy": 0.06037123574279207,
+            "mean_normalized_entropy": 0.037510757809530615,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 4,
+            "dataset": "ChessBench",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.0000025870106683533124,
+            "mean_entropy": 0.011206796682705465,
+            "mean_normalized_entropy": 0.008083994999195031,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 5,
+            "dataset": "CRUXEval",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.03585937563335616,
+            "mean_entropy": 0.51875200368341,
+            "mean_normalized_entropy": 0.3742004715826345,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 6,
+            "dataset": "CLadder",
+            "area": "knowledge",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 50,
+            "skill_score": 100,
+            "mean_brier": 0.000005091011376679519,
+            "mean_entropy": 0.011869921763818098,
+            "mean_normalized_entropy": 0.017124677264400347,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "language",
+        "label": "Language Understanding",
+        "benchmark_count": 3,
+        "coverage_pct": 66.66666666666667,
+        "plain_score": 66.66666666666667,
+        "supported_score": 66.66666666666667,
+        "skill_score": 66.66666666666667,
+        "benchmarks": [
+          {
+            "catalog_id": 7,
+            "dataset": "ContractNLI",
+            "area": "language",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 0,
+            "coverage_pct": 0,
+            "supported_score": 0,
+            "coverage_adj_score": 0,
+            "chance_baseline": 33.33,
+            "skill_score": 0,
+            "mean_brier": 1,
+            "mean_entropy": 0,
+            "mean_normalized_entropy": 0,
+            "mean_passes_per_req": 0
+          },
+          {
+            "catalog_id": 8,
+            "dataset": "iSarcasmEval",
+            "area": "language",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.00003184284214067531,
+            "mean_entropy": 0.03572485030012276,
+            "mean_normalized_entropy": 0.025770032182243854,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 9,
+            "dataset": "VAST",
+            "area": "language",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 33.33,
+            "skill_score": 100,
+            "mean_brier": 0.00001824581387429913,
+            "mean_entropy": 0.025097271675774628,
+            "mean_normalized_entropy": 0.02284452116060073,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "retrieval",
+        "label": "Retrieval \u0026 Classification",
+        "benchmark_count": 2,
+        "coverage_pct": 50,
+        "plain_score": 50,
+        "supported_score": 50,
+        "skill_score": 50,
+        "benchmarks": [
+          {
+            "catalog_id": 10,
+            "dataset": "BRIGHT",
+            "area": "retrieval",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 0,
+            "coverage_pct": 0,
+            "supported_score": 0,
+            "coverage_adj_score": 0,
+            "chance_baseline": 11.200000000000001,
+            "skill_score": 0,
+            "mean_brier": 1,
+            "mean_entropy": 0,
+            "mean_normalized_entropy": 0,
+            "mean_passes_per_req": 0
+          },
+          {
+            "catalog_id": 11,
+            "dataset": "Amazon ESCI",
+            "area": "retrieval",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.000004973925736865692,
+            "mean_entropy": 0.014611664542282533,
+            "mean_normalized_entropy": 0.010540087987142057,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "tools",
+        "label": "Tools \u0026 Automation",
+        "benchmark_count": 3,
+        "coverage_pct": 66.66666666666667,
+        "plain_score": 66.66666666666667,
+        "supported_score": 66.66666666666667,
+        "skill_score": 66.66666666666667,
+        "benchmarks": [
+          {
+            "catalog_id": 12,
+            "dataset": "BFCL",
+            "area": "tools",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25.919999999999998,
+            "skill_score": 100,
+            "mean_brier": 0.000028303132524838522,
+            "mean_entropy": 0.03344297579518745,
+            "mean_normalized_entropy": 0.024124007666143286,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 13,
+            "dataset": "ToolRet",
+            "area": "tools",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 0,
+            "coverage_pct": 0,
+            "supported_score": 0,
+            "coverage_adj_score": 0,
+            "chance_baseline": 9.370000000000001,
+            "skill_score": 0,
+            "mean_brier": 1,
+            "mean_entropy": 0,
+            "mean_normalized_entropy": 0,
+            "mean_passes_per_req": 0
+          },
+          {
+            "catalog_id": 14,
+            "dataset": "RouterBench",
+            "area": "tools",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 9.09,
+            "skill_score": 100,
+            "mean_brier": 0.000004425805859090248,
+            "mean_entropy": 0.01402819143826605,
+            "mean_normalized_entropy": 0.010119201110313723,
+            "mean_passes_per_req": 1
+          }
+        ]
+      },
+      {
+        "area_id": "arts",
+        "label": "Arts \u0026 Human Judgment",
+        "benchmark_count": 5,
+        "coverage_pct": 100,
+        "plain_score": 100,
+        "supported_score": 100,
+        "skill_score": 100,
+        "benchmarks": [
+          {
+            "catalog_id": 15,
+            "dataset": "BPoMP",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 0.000025089727133875345,
+            "mean_entropy": 0.03149919820540211,
+            "mean_normalized_entropy": 0.022721868521456082,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 16,
+            "dataset": "Humicroedit",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 50,
+            "skill_score": 100,
+            "mean_brier": 0.001990036227415131,
+            "mean_entropy": 0.14006865139125854,
+            "mean_normalized_entropy": 0.20207634874617372,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 17,
+            "dataset": "POP909-CL",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 12.5,
+            "skill_score": 100,
+            "mean_brier": 0.003170788798504395,
+            "mean_entropy": 0.21810580755465517,
+            "mean_normalized_entropy": 0.1573300834740918,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 18,
+            "dataset": "cfcolor",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 20,
+            "skill_score": 100,
+            "mean_brier": 0.00017122571029852359,
+            "mean_entropy": 0.0694548956320747,
+            "mean_normalized_entropy": 0.05010111674692735,
+            "mean_passes_per_req": 1
+          },
+          {
+            "catalog_id": 19,
+            "dataset": "Habermas Machine",
+            "area": "arts",
+            "in_panel": true,
+            "total_cases": 1,
+            "supported_cases": 1,
+            "coverage_pct": 100,
+            "supported_score": 100,
+            "coverage_adj_score": 100,
+            "chance_baseline": 25,
+            "skill_score": 100,
+            "mean_brier": 4.131786528919932e-7,
+            "mean_entropy": 0.004700562121164799,
+            "mean_normalized_entropy": 0.003390738830797481,
+            "mean_passes_per_req": 1
+          }
+        ]
+      }
+    ],
+    "display_benchmarks": [
+      {
+        "catalog_id": 20,
+        "dataset": "API-Bank",
+        "area": "tools",
+        "in_panel": false,
+        "total_cases": 1,
+        "supported_cases": 0,
+        "coverage_pct": 0,
+        "supported_score": 0,
+        "coverage_adj_score": 0,
+        "chance_baseline": 1.8900000000000001,
+        "skill_score": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "mean_passes_per_req": 0
+      },
+      {
+        "catalog_id": 21,
+        "dataset": "BANKING77",
+        "area": "retrieval",
+        "in_panel": false,
+        "total_cases": 1,
+        "supported_cases": 0,
+        "coverage_pct": 0,
+        "supported_score": 0,
+        "coverage_adj_score": 0,
+        "chance_baseline": 1.27,
+        "skill_score": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "mean_passes_per_req": 0
+      },
+      {
+        "catalog_id": 22,
+        "dataset": "CLINC150+OOS",
+        "area": "retrieval",
+        "in_panel": false,
+        "total_cases": 1,
+        "supported_cases": 0,
+        "coverage_pct": 0,
+        "supported_score": 0,
+        "coverage_adj_score": 0,
+        "chance_baseline": 0.66,
+        "skill_score": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "mean_passes_per_req": 0
+      }
+    ],
+    "results": [
+      {
+        "id": "di_knowledge_mmlu_01",
+        "catalog_id": 1,
+        "dataset": "MMLU",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.984890336592688,
+        "mean_brier": 0.00031808530128879016,
+        "mean_entropy": 0.09366624993609145,
+        "mean_normalized_entropy": 0.06756591714073266,
+        "forward_passes": 1,
+        "wall_time_ms": 126,
+        "answers": {
+          "answer": {
+            "type": "choice",
+            "choice": "B",
+            "probabilities": {
+              "A": 0.008049626394809576,
+              "B": 0.984890336592688,
+              "C": 0.0037100554490711317,
+              "D": 0.003349981563431216
+            },
+            "entropy": 0.09366624993609145,
+            "normalized_entropy": 0.06756591714073266,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "answer": "B"
+        }
+      },
+      {
+        "id": "di_knowledge_gpqa_01",
+        "catalog_id": 2,
+        "dataset": "GPQA Diamond",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9195671452372348,
+        "mean_brier": 0.010144805463675468,
+        "mean_entropy": 0.34174040382052917,
+        "mean_normalized_entropy": 0.2465135929316346,
+        "forward_passes": 1,
+        "wall_time_ms": 151,
+        "answers": {
+          "answer": {
+            "type": "choice",
+            "choice": "B",
+            "probabilities": {
+              "A": 0.058560882344798254,
+              "B": 0.9195671452372348,
+              "C": 0.012778919829590969,
+              "D": 0.00909305258837602
+            },
+            "entropy": 0.34174040382052917,
+            "normalized_entropy": 0.2465135929316346,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "answer": "B"
+        }
+      },
+      {
+        "id": "di_knowledge_gsm8k_01",
+        "catalog_id": 3,
+        "dataset": "GSM8K",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 5,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9910411390868156,
+        "mean_brier": 0.00011519615331485815,
+        "mean_entropy": 0.06037123574279207,
+        "mean_normalized_entropy": 0.037510757809530615,
+        "forward_passes": 1,
+        "wall_time_ms": 144,
+        "answers": {
+          "answer": {
+            "type": "choice",
+            "choice": "opt_30_00",
+            "probabilities": {
+              "opt_15_00": 0.005325174595336004,
+              "opt_30_00": 0.9910411390868156,
+              "opt_45_00": 0.0005900918579673167,
+              "opt_60_00": 0.0024155359958317924,
+              "opt_7_50": 0.0006280584640493413
+            },
+            "entropy": 0.06037123574279207,
+            "normalized_entropy": 0.037510757809530615,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "answer": "opt_30_00"
+        }
+      },
+      {
+        "id": "di_knowledge_chessbench_01",
+        "catalog_id": 4,
+        "dataset": "ChessBench",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9986826042374531,
+        "mean_brier": 0.0000025870106683533124,
+        "mean_entropy": 0.011206796682705465,
+        "mean_normalized_entropy": 0.008083994999195031,
+        "forward_passes": 1,
+        "wall_time_ms": 224,
+        "answers": {
+          "best_move": {
+            "type": "choice",
+            "choice": "Qxf7#",
+            "probabilities": {
+              "Bxf7+": 0.00028251422567836135,
+              "Qg5": 0.00017380223174887297,
+              "Qh6": 0.0008610793051196632,
+              "Qxf7#": 0.9986826042374531
+            },
+            "entropy": 0.011206796682705465,
+            "normalized_entropy": 0.008083994999195031,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "best_move": "Qxf7#"
+        }
+      },
+      {
+        "id": "di_knowledge_cruxeval_01",
+        "catalog_id": 5,
+        "dataset": "CRUXEval",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.8519781342562239,
+        "mean_brier": 0.03585937563335616,
+        "mean_entropy": 0.51875200368341,
+        "mean_normalized_entropy": 0.3742004715826345,
+        "forward_passes": 1,
+        "wall_time_ms": 168,
+        "answers": {
+          "output": {
+            "type": "choice",
+            "choice": "B",
+            "probabilities": {
+              "A": 0.1158531161231781,
+              "B": 0.8519781342562239,
+              "C": 0.018268883171548234,
+              "D": 0.013899866449049949
+            },
+            "entropy": 0.51875200368341,
+            "normalized_entropy": 0.3742004715826345,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "output": "B"
+        }
+      },
+      {
+        "id": "di_knowledge_cladder_01",
+        "catalog_id": 6,
+        "dataset": "CLadder",
+        "area": "knowledge",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 2,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9984045359018957,
+        "mean_brier": 0.000005091011376679519,
+        "mean_entropy": 0.011869921763818098,
+        "mean_normalized_entropy": 0.017124677264400347,
+        "forward_passes": 1,
+        "wall_time_ms": 217,
+        "answers": {
+          "causal_answer": {
+            "type": "choice",
+            "choice": "yes",
+            "probabilities": {
+              "no": 0.0015954640981043018,
+              "yes": 0.9984045359018957
+            },
+            "entropy": 0.011869921763818098,
+            "normalized_entropy": 0.017124677264400347,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "causal_answer": "yes"
+        }
+      },
+      {
+        "id": "di_language_contractnli_12slots",
+        "catalog_id": 7,
+        "dataset": "ContractNLI",
+        "area": "language",
+        "total_fields": 12,
+        "correct_fields": 0,
+        "max_options": 3,
+        "whole_case_exact": false,
+        "field_accuracy": 0,
+        "supported": false,
+        "error_msg": "HTTP 422 Unprocessable Entity: capacity exceeded (fields=12, max_options=3)",
+        "mean_confidence": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "forward_passes": 0,
+        "wall_time_ms": 0,
+        "expected": {
+          "clause_01_confidential_marking": "contradiction",
+          "clause_02_sharing_employees": "entailment",
+          "clause_03_sharing_affiliates": "entailment",
+          "clause_04_return_or_destroy": "entailment",
+          "clause_05_survival_period": "entailment",
+          "clause_06_no_license": "entailment",
+          "clause_07_reverse_engineering": "contradiction",
+          "clause_08_public_domain_exception": "entailment",
+          "clause_09_compelled_disclosure": "entailment",
+          "clause_10_non_solicitation": "not_mentioned",
+          "clause_11_exclusive_dealing": "not_mentioned",
+          "clause_12_injunctive_relief": "entailment"
+        }
+      },
+      {
+        "id": "di_language_isarcasm_01",
+        "catalog_id": 8,
+        "dataset": "iSarcasmEval",
+        "area": "language",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9951625133540951,
+        "mean_brier": 0.00003184284214067531,
+        "mean_entropy": 0.03572485030012276,
+        "mean_normalized_entropy": 0.025770032182243854,
+        "forward_passes": 1,
+        "wall_time_ms": 316,
+        "answers": {
+          "tone": {
+            "type": "choice",
+            "choice": "sarcasm",
+            "probabilities": {
+              "literal_happy": 0.002156793640348561,
+              "neutral_fact": 0.0010266995760436232,
+              "sarcasm": 0.9951625133540951,
+              "understatement": 0.0016539934295126573
+            },
+            "entropy": 0.03572485030012276,
+            "normalized_entropy": 0.025770032182243854,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "tone": "sarcasm"
+        }
+      },
+      {
+        "id": "di_language_vast_01",
+        "catalog_id": 9,
+        "dataset": "VAST",
+        "area": "language",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 3,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.996569232361649,
+        "mean_brier": 0.00001824581387429913,
+        "mean_entropy": 0.025097271675774628,
+        "mean_normalized_entropy": 0.02284452116060073,
+        "forward_passes": 1,
+        "wall_time_ms": 183,
+        "answers": {
+          "stance": {
+            "type": "choice",
+            "choice": "pro",
+            "probabilities": {
+              "con": 0.002258782379977801,
+              "neutral": 0.0011719852583732353,
+              "pro": 0.996569232361649
+            },
+            "entropy": 0.025097271675774628,
+            "normalized_entropy": 0.02284452116060073,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "stance": "pro"
+        }
+      },
+      {
+        "id": "di_retrieval_bright_11slots",
+        "catalog_id": 10,
+        "dataset": "BRIGHT",
+        "area": "retrieval",
+        "total_fields": 11,
+        "correct_fields": 0,
+        "max_options": 2,
+        "whole_case_exact": false,
+        "field_accuracy": 0,
+        "supported": false,
+        "error_msg": "HTTP 422 Unprocessable Entity: capacity exceeded (fields=11, max_options=2)",
+        "mean_confidence": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "forward_passes": 0,
+        "wall_time_ms": 0,
+        "expected": {
+          "passage_01": "not_relevant",
+          "passage_02": "relevant",
+          "passage_03": "not_relevant",
+          "passage_04": "not_relevant",
+          "passage_05": "not_relevant",
+          "passage_06": "not_relevant",
+          "passage_07": "relevant",
+          "passage_08": "not_relevant",
+          "passage_09": "not_relevant",
+          "passage_10": "not_relevant",
+          "passage_11": "not_relevant"
+        }
+      },
+      {
+        "id": "di_retrieval_esci_01",
+        "catalog_id": 11,
+        "dataset": "Amazon ESCI",
+        "area": "retrieval",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9982022627113681,
+        "mean_brier": 0.000004973925736865692,
+        "mean_entropy": 0.014611664542282533,
+        "mean_normalized_entropy": 0.010540087987142057,
+        "forward_passes": 1,
+        "wall_time_ms": 123,
+        "answers": {
+          "esci_label": {
+            "type": "choice",
+            "choice": "Exact",
+            "probabilities": {
+              "Complement": 0.0012603098124193191,
+              "Exact": 0.9982022627113681,
+              "Irrelevant": 0.00020062773163739565,
+              "Substitute": 0.0003367997445751219
+            },
+            "entropy": 0.014611664542282533,
+            "normalized_entropy": 0.010540087987142057,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "esci_label": "Exact"
+        }
+      },
+      {
+        "id": "di_tools_bfcl_01",
+        "catalog_id": 12,
+        "dataset": "BFCL",
+        "area": "tools",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.995484492546379,
+        "mean_brier": 0.000028303132524838522,
+        "mean_entropy": 0.03344297579518745,
+        "mean_normalized_entropy": 0.024124007666143286,
+        "forward_passes": 1,
+        "wall_time_ms": 183,
+        "answers": {
+          "function_call": {
+            "type": "choice",
+            "choice": "get_equity_quote",
+            "probabilities": {
+              "cancel_subscription": 0.0006957056223265837,
+              "execute_wire_transfer": 0.0021686243519798967,
+              "get_equity_quote": 0.995484492546379,
+              "get_weather_forecast": 0.001651177479314425
+            },
+            "entropy": 0.03344297579518745,
+            "normalized_entropy": 0.024124007666143286,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "function_call": "get_equity_quote"
+        }
+      },
+      {
+        "id": "di_tools_toolret_12slots",
+        "catalog_id": 13,
+        "dataset": "ToolRet",
+        "area": "tools",
+        "total_fields": 12,
+        "correct_fields": 0,
+        "max_options": 2,
+        "whole_case_exact": false,
+        "field_accuracy": 0,
+        "supported": false,
+        "error_msg": "HTTP 422 Unprocessable Entity: capacity exceeded (fields=12, max_options=2)",
+        "mean_confidence": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "forward_passes": 0,
+        "wall_time_ms": 0,
+        "expected": {
+          "tool_candidate_01": "skip",
+          "tool_candidate_02": "skip",
+          "tool_candidate_03": "skip",
+          "tool_candidate_04": "invoke",
+          "tool_candidate_05": "skip",
+          "tool_candidate_06": "skip",
+          "tool_candidate_07": "skip",
+          "tool_candidate_08": "skip",
+          "tool_candidate_09": "invoke",
+          "tool_candidate_10": "skip",
+          "tool_candidate_11": "skip",
+          "tool_candidate_12": "skip"
+        }
+      },
+      {
+        "id": "di_tools_routerbench_01",
+        "catalog_id": 14,
+        "dataset": "RouterBench",
+        "area": "tools",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9982915615380495,
+        "mean_brier": 0.000004425805859090248,
+        "mean_entropy": 0.01402819143826605,
+        "mean_normalized_entropy": 0.010119201110313723,
+        "forward_passes": 1,
+        "wall_time_ms": 222,
+        "answers": {
+          "target_model": {
+            "type": "choice",
+            "choice": "dgemma-vllm-l4",
+            "probabilities": {
+              "claude-3-7-opus": 0.0011614866068420742,
+              "dgemma-vllm-l4": 0.9982915615380495,
+              "gemini-2.5-pro": 0.00020861231278486073,
+              "gpt-4.5-preview": 0.0003383395423234517
+            },
+            "entropy": 0.01402819143826605,
+            "normalized_entropy": 0.010119201110313723,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "target_model": "dgemma-vllm-l4"
+        }
+      },
+      {
+        "id": "di_arts_bpomp_01",
+        "catalog_id": 15,
+        "dataset": "BPoMP",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9957772169079427,
+        "mean_brier": 0.000025089727133875345,
+        "mean_entropy": 0.03149919820540211,
+        "mean_normalized_entropy": 0.022721868521456082,
+        "forward_passes": 1,
+        "wall_time_ms": 204,
+        "answers": {
+          "mood": {
+            "type": "choice",
+            "choice": "melancholic_somber",
+            "probabilities": {
+              "euphoric_festive": 0.0023334052348670735,
+              "melancholic_somber": 0.9957772169079427,
+              "playful_whimsical": 0.0010633812990020395,
+              "triumphant_march": 0.0008259965581879597
+            },
+            "entropy": 0.03149919820540211,
+            "normalized_entropy": 0.022721868521456082,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "mood": "melancholic_somber"
+        }
+      },
+      {
+        "id": "di_arts_humicroedit_01",
+        "catalog_id": 16,
+        "dataset": "Humicroedit",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 2,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9684560922885644,
+        "mean_brier": 0.001990036227415131,
+        "mean_entropy": 0.14006865139125854,
+        "mean_normalized_entropy": 0.20207634874617372,
+        "forward_passes": 1,
+        "wall_time_ms": 105,
+        "answers": {
+          "funnier_edit": {
+            "type": "choice",
+            "choice": "Edit_A",
+            "probabilities": {
+              "Edit_A": 0.9684560922885644,
+              "Edit_B": 0.031543907711435615
+            },
+            "entropy": 0.14006865139125854,
+            "normalized_entropy": 0.20207634874617372,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "funnier_edit": "Edit_A"
+        }
+      },
+      {
+        "id": "di_arts_pop909_01",
+        "catalog_id": 17,
+        "dataset": "POP909-CL",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9544621455187181,
+        "mean_brier": 0.003170788798504395,
+        "mean_entropy": 0.21810580755465517,
+        "mean_normalized_entropy": 0.1573300834740918,
+        "forward_passes": 1,
+        "wall_time_ms": 106,
+        "answers": {
+          "section_type": {
+            "type": "choice",
+            "choice": "chorus",
+            "probabilities": {
+              "chorus": 0.9544621455187181,
+              "intro": 0.02932567685548618,
+              "outro_fade": 0.0008369783780365701,
+              "verse": 0.01537519924775907
+            },
+            "entropy": 0.21810580755465517,
+            "normalized_entropy": 0.1573300834740918,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "section_type": "chorus"
+        }
+      },
+      {
+        "id": "di_arts_cfcolor_01",
+        "catalog_id": 18,
+        "dataset": "cfcolor",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.989185035990081,
+        "mean_brier": 0.00017122571029852359,
+        "mean_entropy": 0.0694548956320747,
+        "mean_normalized_entropy": 0.05010111674692735,
+        "forward_passes": 1,
+        "wall_time_ms": 195,
+        "answers": {
+          "palette": {
+            "type": "choice",
+            "choice": "crimson_alert_red",
+            "probabilities": {
+              "baby_sky_blue": 0.006634292201524307,
+              "crimson_alert_red": 0.989185035990081,
+              "muted_slate_gray": 0.001221594756696922,
+              "pastel_mint_green": 0.002959077051697847
+            },
+            "entropy": 0.0694548956320747,
+            "normalized_entropy": 0.05010111674692735,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "palette": "crimson_alert_red"
+        }
+      },
+      {
+        "id": "di_arts_habermas_01",
+        "catalog_id": 19,
+        "dataset": "Habermas Machine",
+        "area": "arts",
+        "total_fields": 1,
+        "correct_fields": 1,
+        "max_options": 4,
+        "whole_case_exact": true,
+        "field_accuracy": 1,
+        "supported": true,
+        "mean_confidence": 0.9994936026099215,
+        "mean_brier": 4.131786528919932e-7,
+        "mean_entropy": 0.004700562121164799,
+        "mean_normalized_entropy": 0.003390738830797481,
+        "forward_passes": 1,
+        "wall_time_ms": 256,
+        "answers": {
+          "consensus_statement": {
+            "type": "choice",
+            "choice": "Statement_1",
+            "probabilities": {
+              "Statement_1": 0.9994936026099215,
+              "Statement_2": 0.0003845470694486725,
+              "Statement_3": 0.00008775925498287552,
+              "Statement_4": 0.00003409106564700237
+            },
+            "entropy": 0.004700562121164799,
+            "normalized_entropy": 0.003390738830797481,
+            "passes_used": 1
+          }
+        },
+        "expected": {
+          "consensus_statement": "Statement_1"
+        }
+      },
+      {
+        "id": "di_wide_apibank_30opts",
+        "catalog_id": 20,
+        "dataset": "API-Bank",
+        "area": "tools",
+        "total_fields": 1,
+        "correct_fields": 0,
+        "max_options": 30,
+        "whole_case_exact": false,
+        "field_accuracy": 0,
+        "supported": false,
+        "error_msg": "HTTP 422 Unprocessable Entity: capacity exceeded (fields=1, max_options=30)",
+        "mean_confidence": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "forward_passes": 0,
+        "wall_time_ms": 0,
+        "expected": {
+          "selected_api": "ConvertCurrencyFX_28"
+        }
+      },
+      {
+        "id": "di_wide_banking77_32opts",
+        "catalog_id": 21,
+        "dataset": "BANKING77",
+        "area": "retrieval",
+        "total_fields": 1,
+        "correct_fields": 0,
+        "max_options": 32,
+        "whole_case_exact": false,
+        "field_accuracy": 0,
+        "supported": false,
+        "error_msg": "HTTP 422 Unprocessable Entity: capacity exceeded (fields=1, max_options=32)",
+        "mean_confidence": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "forward_passes": 0,
+        "wall_time_ms": 0,
+        "expected": {
+          "intent": "lost_or_stolen_card"
+        }
+      },
+      {
+        "id": "di_wide_clinc150_36opts",
+        "catalog_id": 22,
+        "dataset": "CLINC150+OOS",
+        "area": "retrieval",
+        "total_fields": 1,
+        "correct_fields": 0,
+        "max_options": 36,
+        "whole_case_exact": false,
+        "field_accuracy": 0,
+        "supported": false,
+        "error_msg": "HTTP 422 Unprocessable Entity: capacity exceeded (fields=1, max_options=36)",
+        "mean_confidence": 0,
+        "mean_brier": 1,
+        "mean_entropy": 0,
+        "mean_normalized_entropy": 0,
+        "forward_passes": 0,
+        "wall_time_ms": 0,
+        "expected": {
+          "intent": "oos_out_of_scope"
+        }
+      }
+    ]
+  },
+  "headline_index_delta": 22.222222222222214,
+  "coverage_pct_delta": 27.272727272727266
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/intents_clinc150.json b/benchmarks/runs/20260928-v010-verification/intents_clinc150.json
new file mode 100644
index 0000000..21df105
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/intents_clinc150.json
@@ -0,0 +1,1442 @@
+{
+  "avg_wall_time_ms": 184.66666666666666,
+  "candidate_intents": 26,
+  "corpus": "benchmarks/intents/clinc150_eval.jsonl",
+  "correct_mean_confidence": 0.9517962163790836,
+  "correct_mean_entropy_nats": 0,
+  "dataset": "DeepPavlov/clinc150",
+  "domain_accuracy_pct": 93.33333333333333,
+  "domain_correct": 28,
+  "entropy_threshold": 0.08,
+  "in_domain_correct": 24,
+  "in_domain_total": 25,
+  "incorrect_mean_confidence": 0,
+  "incorrect_mean_entropy_nats": 0,
+  "intent_accuracy_pct": 96.66666666666667,
+  "intent_correct": 29,
+  "mean_confidence": 0.9517962163790836,
+  "mean_entropy_nats": 0,
+  "oos_correct": 5,
+  "oos_total": 5,
+  "pass1_accuracy_pct": 100,
+  "pass1_correct": 30,
+  "results": [
+    {
+      "id": "c150-01",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "can you please provide me with assistance in moving money from one account to another",
+      "expected_domain": "banking",
+      "actual_domain": "banking",
+      "domain_accurate": true,
+      "expected_intent": "transfer",
+      "pass1_intent": "transfer",
+      "pass1_accurate": true,
+      "actual_intent": "transfer",
+      "intent_accurate": true,
+      "confidence": 0.9929342767809901,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00039866625500017223,
+        "balance": 0.004737469963004567,
+        "bill_balance": 0.00025006923466697835,
+        "book_flight": 0.00004727337593232968,
+        "card_declined": 0.0007518375018235796,
+        "flight_status": 0.00007447849315291866,
+        "freeze_account": 0.000027317650452063335,
+        "international_fees": 0.00020359549797314383,
+        "jump_start": 0.000001087081751025106,
+        "lost_luggage": 4.1459886889146977e-7,
+        "oil_change_when": 3.7570976438263894e-7,
+        "oos": 0.0002335944154468313,
+        "payday": 0.0000030008563555411182,
+        "pto_balance": 0.000009365579742481112,
+        "pto_request": 3.238783550733061e-7,
+        "redeem_rewards": 0.00016518544986404937,
+        "replacement_card_duration": 0.00004049788870915541,
+        "report_fraud": 0.00001824366499699217,
+        "schedule_maintenance": 7.791846827125006e-7,
+        "schedule_meeting": 0.000012062328813974959,
+        "tire_pressure": 0.000005585247679923137,
+        "traffic": 0.000014250410748955987,
+        "transfer": 0.9929342767809901,
+        "travel_alert": 0.000009365579742481112,
+        "vaccines": 0.0000014407084311557503,
+        "w2": 0.000059442663050420176
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 237
+    },
+    {
+      "id": "c150-02",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "is there a travel alert in spain",
+      "expected_domain": "travel",
+      "actual_domain": "travel",
+      "domain_accurate": true,
+      "expected_intent": "travel_alert",
+      "pass1_intent": "travel_alert",
+      "pass1_accurate": true,
+      "actual_intent": "travel_alert",
+      "intent_accurate": true,
+      "confidence": 0.9927568993802721,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00016099171395402248,
+        "balance": 0.000882910902769422,
+        "bill_balance": 0.0003212211165152398,
+        "book_flight": 0.00014880072716227377,
+        "card_declined": 0.000401520470888568,
+        "flight_status": 0.001977582293641284,
+        "freeze_account": 0.0002372442430125566,
+        "international_fees": 0.00020340316118743683,
+        "jump_start": 0.00005606313561505173,
+        "lost_luggage": 0.00003997605977208594,
+        "oil_change_when": 0.00004737912180441447,
+        "oos": 0.0005001179331009717,
+        "payday": 0.000018245490913560212,
+        "pto_balance": 0.000012742593428000175,
+        "pto_request": 0.0000028466130678580216,
+        "redeem_rewards": 0.00012696662191833574,
+        "replacement_card_duration": 0.000432180803957319,
+        "report_fraud": 0.00008479266985799378,
+        "schedule_maintenance": 0.00009977287974478866,
+        "schedule_meeting": 0.00004737912180441447,
+        "tire_pressure": 0.0000781215218499555,
+        "traffic": 0.0000781215218499555,
+        "transfer": 0.0007677464280971177,
+        "travel_alert": 0.9927568993802721,
+        "vaccines": 0.00008479266985799378,
+        "w2": 0.000432180803957319
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 236
+    },
+    {
+      "id": "c150-03",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "please help me set up a vacation request",
+      "expected_domain": "work",
+      "actual_domain": "work",
+      "domain_accurate": true,
+      "expected_intent": "pto_request",
+      "pass1_intent": "pto_request",
+      "pass1_accurate": true,
+      "actual_intent": "pto_request",
+      "intent_accurate": true,
+      "confidence": 0.9444411277381398,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00031468299680210147,
+        "balance": 0.003264545870924767,
+        "bill_balance": 0.004351223250030017,
+        "book_flight": 0.00011077258506079047,
+        "card_declined": 0.0020235530474006155,
+        "flight_status": 0.0036660337223201056,
+        "freeze_account": 0.0002928159598753471,
+        "international_fees": 0.0007245650946551003,
+        "jump_start": 0.00020307310703888265,
+        "lost_luggage": 0.00009482058564635763,
+        "oil_change_when": 0.00008103630794361774,
+        "oos": 0.018597527521853223,
+        "payday": 0.002285502025662598,
+        "pto_balance": 0.00031468299680210147,
+        "pto_request": 0.9444411277381398,
+        "redeem_rewards": 0.0008848733929857027,
+        "replacement_card_duration": 0.004869264495608595,
+        "report_fraud": 0.00036301211157391267,
+        "schedule_maintenance": 0.00016226879989464645,
+        "schedule_meeting": 0.0016812296553715136,
+        "tire_pressure": 0.001225405489781917,
+        "traffic": 0.002150932568739752,
+        "transfer": 0.0027359199865763795,
+        "travel_alert": 0.0010769847944941018,
+        "vaccines": 0.0004181121724979089,
+        "w2": 0.0036660337223201056
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 234
+    },
+    {
+      "id": "c150-04",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "how long until my paycheck shows up",
+      "expected_domain": "work",
+      "actual_domain": "work",
+      "domain_accurate": true,
+      "expected_intent": "payday",
+      "pass1_intent": "payday",
+      "pass1_accurate": true,
+      "actual_intent": "payday",
+      "intent_accurate": true,
+      "confidence": 0.9535038621028433,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0003172370048198151,
+        "balance": 0.0011398400530526623,
+        "bill_balance": 0.002657734785630019,
+        "book_flight": 0.00009912541066807245,
+        "card_declined": 0.0021983861088103237,
+        "flight_status": 0.00014754016410733012,
+        "freeze_account": 0.00006593485205329269,
+        "international_fees": 0.00025319272163456494,
+        "jump_start": 0.00007159413283436767,
+        "lost_luggage": 0.00003086198730906846,
+        "oil_change_when": 0.00013636778667429914,
+        "oos": 0.02121438724595195,
+        "payday": 0.9535038621028433,
+        "pto_balance": 0.0003172370048198151,
+        "pto_request": 0.0005295483633106005,
+        "redeem_rewards": 0.0001595639500845866,
+        "replacement_card_duration": 0.0002013583190747691,
+        "report_fraud": 0.00007159413283436767,
+        "schedule_maintenance": 0.00003363200021954936,
+        "schedule_meeting": 0.004884985634871419,
+        "tire_pressure": 0.0006108762781460971,
+        "traffic": 0.0006557271812647819,
+        "transfer": 0.0007546711430084452,
+        "travel_alert": 0.000703598584288824,
+        "vaccines": 0.00008430992629060327,
+        "w2": 0.00915683312539706
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 239
+    },
+    {
+      "id": "c150-05",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "how long does it take for a credit card to be reissued and mailed to me",
+      "expected_domain": "credit_cards",
+      "actual_domain": "credit_cards",
+      "domain_accurate": true,
+      "expected_intent": "replacement_card_duration",
+      "pass1_intent": "replacement_card_duration",
+      "pass1_accurate": true,
+      "actual_intent": "replacement_card_duration",
+      "intent_accurate": true,
+      "confidence": 0.9948926898407031,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00041132442810353333,
+        "balance": 0.0028199030866387625,
+        "bill_balance": 0.00022436978910730677,
+        "book_flight": 0.00006723093518372292,
+        "card_declined": 0.0003543323208380377,
+        "flight_status": 0.00011931222449569269,
+        "freeze_account": 0.000021994060021364824,
+        "international_fees": 0.00028246387375383843,
+        "jump_start": 0.000004610271380152177,
+        "lost_luggage": 0.000026234424661283392,
+        "oil_change_when": 0.000005573334483488924,
+        "oos": 0.00016413898180101324,
+        "payday": 0.00001683291827445738,
+        "pto_balance": 0.000026234424661283392,
+        "pto_request": 7.047427819177633e-7,
+        "redeem_rewards": 0.00007936243408479104,
+        "replacement_card_duration": 0.9948926898407031,
+        "report_fraud": 0.000020126220873540855,
+        "schedule_maintenance": 0.00007306010385708645,
+        "schedule_meeting": 0.0000184097374327754,
+        "tire_pressure": 0.000020126220873540855,
+        "traffic": 0.00003407383916480351,
+        "transfer": 0.00004801617586479915,
+        "travel_alert": 0.000026234424661283392,
+        "vaccines": 2.7640003818877865e-7,
+        "w2": 0.0002423647862602226
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 149
+    },
+    {
+      "id": "c150-06",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "what's the status of by delta flight",
+      "expected_domain": "travel",
+      "actual_domain": "travel",
+      "domain_accurate": true,
+      "expected_intent": "flight_status",
+      "pass1_intent": "flight_status",
+      "pass1_accurate": true,
+      "actual_intent": "flight_status",
+      "intent_accurate": true,
+      "confidence": 0.9959534799492056,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00019131006683330268,
+        "balance": 0.0003409152059319178,
+        "bill_balance": 0.00013163591704406662,
+        "book_flight": 0.00036580374141814324,
+        "card_declined": 0.00015305023883294342,
+        "flight_status": 0.9959534799492056,
+        "freeze_account": 0.00017766739937857676,
+        "international_fees": 0.00010468700211736455,
+        "jump_start": 0.000013494835522990035,
+        "lost_luggage": 0.0011429498035407909,
+        "oil_change_when": 0.000008693316218556277,
+        "oos": 0.0001419679646308086,
+        "payday": 0.000006645403452804736,
+        "pto_balance": 0.00001904714597250801,
+        "pto_request": 2.1225637951060868e-7,
+        "redeem_rewards": 0.000006645403452804736,
+        "replacement_card_duration": 0.00010468700211736455,
+        "report_fraud": 0.0000042134377857913045,
+        "schedule_maintenance": 0.00003427979548214297,
+        "schedule_meeting": 0.000006645403452804736,
+        "tire_pressure": 0.00003427979548214297,
+        "traffic": 0.00023828821229050865,
+        "transfer": 0.00007670672631392543,
+        "travel_alert": 0.0005172956287322017,
+        "vaccines": 0.0000038418963193553256,
+        "w2": 0.000221556452091222
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 150
+    },
+    {
+      "id": "c150-07",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "what are my coffers at",
+      "expected_domain": "banking",
+      "actual_domain": "banking",
+      "domain_accurate": true,
+      "expected_intent": "balance",
+      "pass1_intent": "balance",
+      "pass1_accurate": true,
+      "actual_intent": "balance",
+      "intent_accurate": true,
+      "confidence": 0.927622963327195,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.004434052445408807,
+        "balance": 0.927622963327195,
+        "bill_balance": 0.010796312168236054,
+        "book_flight": 0.0001322279028194983,
+        "card_declined": 0.009291465331428808,
+        "flight_status": 0.0002386068074178991,
+        "freeze_account": 0.001648930174505673,
+        "international_fees": 0.004434052445408807,
+        "jump_start": 0.000019317172833078956,
+        "lost_luggage": 0.000010554253982769145,
+        "oil_change_when": 0.000010554253982769145,
+        "oos": 0.026165190118450615,
+        "payday": 0.00003762885369041011,
+        "pto_balance": 0.000061009429560807374,
+        "pto_request": 0.0000013913604023244853,
+        "redeem_rewards": 0.003751457263945048,
+        "replacement_card_duration": 0.0031640427469580854,
+        "report_fraud": 0.0008776012704022456,
+        "schedule_maintenance": 0.00003192794569868351,
+        "schedule_meeting": 0.0002958073724235649,
+        "tire_pressure": 0.00009026520576964007,
+        "traffic": 0.0002386068074178991,
+        "transfer": 0.005224650382796926,
+        "travel_alert": 0.0001781847685948725,
+        "vaccines": 0.00003466836603549901,
+        "w2": 0.0012085318246343377
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 154
+    },
+    {
+      "id": "c150-08",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "can you freeze my bank account",
+      "expected_domain": "banking",
+      "actual_domain": "banking",
+      "domain_accurate": true,
+      "expected_intent": "freeze_account",
+      "pass1_intent": "freeze_account",
+      "pass1_accurate": true,
+      "actual_intent": "freeze_account",
+      "intent_accurate": true,
+      "confidence": 0.984205072036885,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00017779012990943262,
+        "balance": 0.0055473782270015586,
+        "bill_balance": 0.0008069705712799427,
+        "book_flight": 0.00003892501621980905,
+        "card_declined": 0.0009237486562215117,
+        "flight_status": 0.00008815378881909551,
+        "freeze_account": 0.984205072036885,
+        "international_fees": 0.00020687682846818245,
+        "jump_start": 0.000005598441366111781,
+        "lost_luggage": 0.000008847269251457501,
+        "oil_change_when": 0.00002144696315006622,
+        "oos": 0.000863551373542003,
+        "payday": 0.00002338129536324741,
+        "pto_balance": 0.000027755650089160032,
+        "pto_request": 0.000004649151910515118,
+        "redeem_rewards": 0.00003289564639017454,
+        "replacement_card_duration": 0.00011177764154940949,
+        "report_fraud": 0.000025479879922023748,
+        "schedule_maintenance": 0.000042316904251846164,
+        "schedule_meeting": 0.000012667968388804053,
+        "tire_pressure": 0.00003892501621980905,
+        "traffic": 0.00006927298874307485,
+        "transfer": 0.006207829452157966,
+        "travel_alert": 0.00004995278609738334,
+        "vaccines": 0.000027755650089160032,
+        "w2": 0.0004309806667131969
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 148
+    },
+    {
+      "id": "c150-09",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "is my luggage lost",
+      "expected_domain": "travel",
+      "actual_domain": "travel",
+      "domain_accurate": true,
+      "expected_intent": "lost_luggage",
+      "pass1_intent": "lost_luggage",
+      "pass1_accurate": true,
+      "actual_intent": "lost_luggage",
+      "intent_accurate": true,
+      "confidence": 0.9701150155281777,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00011017741810095427,
+        "balance": 0.002702967698187319,
+        "bill_balance": 0.00029577042612120955,
+        "book_flight": 0.0012680738816177655,
+        "card_declined": 0.0004561788806842771,
+        "flight_status": 0.008973217148930002,
+        "freeze_account": 0.0006937996199750547,
+        "international_fees": 0.0002747874956122037,
+        "jump_start": 0.00004171108987330692,
+        "lost_luggage": 0.9701150155281777,
+        "oil_change_when": 0.000038367760557379425,
+        "oos": 0.002391437668289278,
+        "payday": 0.000007270496019684888,
+        "pto_balance": 0.000029789962334988338,
+        "pto_request": 5.934727298644813e-7,
+        "redeem_rewards": 0.00008689176755989647,
+        "replacement_card_duration": 0.0010407004411105418,
+        "report_fraud": 0.00006295946636078692,
+        "schedule_maintenance": 0.00023690060485293105,
+        "schedule_meeting": 0.00006828126750620978,
+        "tire_pressure": 0.00011017741810095427,
+        "traffic": 0.0004896718079992044,
+        "transfer": 0.0011119754440405971,
+        "travel_alert": 0.008973217148930002,
+        "vaccines": 0.00010183430890714008,
+        "w2": 0.0003182317774208948
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 161
+    },
+    {
+      "id": "c150-10",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "am i due for an oil change yet",
+      "expected_domain": "auto_and_commute",
+      "actual_domain": "auto_and_commute",
+      "domain_accurate": true,
+      "expected_intent": "oil_change_when",
+      "pass1_intent": "oil_change_when",
+      "pass1_accurate": true,
+      "actual_intent": "oil_change_when",
+      "intent_accurate": true,
+      "confidence": 0.9890055192796449,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00006032665058355645,
+        "balance": 0.00003933114542533819,
+        "bill_balance": 0.00007760410196230203,
+        "book_flight": 0.000027732624874051867,
+        "card_declined": 0.0000994701795266038,
+        "flight_status": 0.0005112462151486915,
+        "freeze_account": 0.000014817500913302555,
+        "international_fees": 0.00006032665058355645,
+        "jump_start": 0.0005112462151486915,
+        "lost_luggage": 0.000025387558150109176,
+        "oil_change_when": 0.9890055192796449,
+        "oos": 0.0006846086125266264,
+        "payday": 0.000033052998571751405,
+        "pto_balance": 0.00008433266224258028,
+        "pto_request": 0.000003133401073735887,
+        "redeem_rewards": 0.000007069219497285686,
+        "replacement_card_duration": 0.00006563601527212664,
+        "report_fraud": 0.00007760410196230203,
+        "schedule_maintenance": 0.004182768039336619,
+        "schedule_meeting": 0.00017504773386248362,
+        "tire_pressure": 0.0019362395444495035,
+        "traffic": 0.001694044504323812,
+        "transfer": 0.00011713702187583887,
+        "travel_alert": 0.00017504773386248362,
+        "vaccines": 0.00009160738891794709,
+        "w2": 0.0002396629002639113
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 159
+    },
+    {
+      "id": "c150-11",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "give me the pressure for the tires on my car",
+      "expected_domain": "auto_and_commute",
+      "actual_domain": "auto_and_commute",
+      "domain_accurate": true,
+      "expected_intent": "tire_pressure",
+      "pass1_intent": "tire_pressure",
+      "pass1_accurate": true,
+      "actual_intent": "tire_pressure",
+      "intent_accurate": true,
+      "confidence": 0.840583948364757,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00031226538378425003,
+        "balance": 0.002075223016701004,
+        "bill_balance": 0.0012922813056014887,
+        "book_flight": 0.0005223639273063025,
+        "card_declined": 0.010432792349371394,
+        "flight_status": 0.0008613047985386185,
+        "freeze_account": 0.0005223639273063025,
+        "international_fees": 0.0005683373165695032,
+        "jump_start": 0.0005223639273063025,
+        "lost_luggage": 0.00006121183960773346,
+        "oil_change_when": 0.0007302301756039767,
+        "oos": 0.003816482492461477,
+        "payday": 0.00008102896701100468,
+        "pto_balance": 0.0005223639273063025,
+        "pto_request": 0.000018324644361577563,
+        "redeem_rewards": 0.00031226538378425003,
+        "replacement_card_duration": 0.005927974453211302,
+        "report_fraud": 0.0024224175861728716,
+        "schedule_maintenance": 0.00788864500606057,
+        "schedule_meeting": 0.0007932214864005407,
+        "tire_pressure": 0.840583948364757,
+        "traffic": 0.11639248409756384,
+        "transfer": 0.0009348554879647742,
+        "travel_alert": 0.0004046058249014,
+        "vaccines": 0.00008102896701100468,
+        "w2": 0.0019196153433352061
+      },
+      "tie_break_triggered": true,
+      "finalist_options": [
+        "tire_pressure",
+        "traffic",
+        "replacement_card_duration",
+        "schedule_maintenance",
+        "card_declined"
+      ],
+      "wall_time_ms": 315
+    },
+    {
+      "id": "c150-12",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "you need to tell me my credit card's apr",
+      "expected_domain": "credit_cards",
+      "actual_domain": "credit_cards",
+      "domain_accurate": true,
+      "expected_intent": "apr",
+      "pass1_intent": "apr",
+      "pass1_accurate": true,
+      "actual_intent": "apr",
+      "intent_accurate": true,
+      "confidence": 0.9833913839818356,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.9833913839818356,
+        "balance": 0.008897606523888326,
+        "bill_balance": 0.00018889704602586527,
+        "book_flight": 0.00023528264952038492,
+        "card_declined": 0.0005107709061371546,
+        "flight_status": 0.00027184162160304767,
+        "freeze_account": 0.00008855148959621647,
+        "international_fees": 0.0012026658406484806,
+        "jump_start": 0.00000237698091022642,
+        "lost_luggage": 0.000005994805455786691,
+        "oil_change_when": 0.00000858366619716306,
+        "oos": 0.00027184162160304767,
+        "payday": 0.00000858366619716306,
+        "pto_balance": 0.000043280197988708286,
+        "pto_request": 3.422878349182695e-8,
+        "redeem_rewards": 0.00016285247754363986,
+        "replacement_card_duration": 0.0001401773027053064,
+        "report_fraud": 0.000012212537926926426,
+        "schedule_maintenance": 0.000043280197988708286,
+        "schedule_meeting": 0.00012046849829855788,
+        "tire_pressure": 0.00003384741959548779,
+        "traffic": 0.0002033213326233562,
+        "transfer": 0.0018581016559110074,
+        "travel_alert": 0.00007000391372899835,
+        "vaccines": 0.000001339200818587769,
+        "w2": 0.0022267002364688058
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 156
+    },
+    {
+      "id": "c150-13",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "can you see if i can make an appointment to have my oil changed",
+      "expected_domain": "auto_and_commute",
+      "actual_domain": "auto_and_commute",
+      "domain_accurate": true,
+      "expected_intent": "schedule_maintenance",
+      "pass1_intent": "schedule_maintenance",
+      "pass1_accurate": true,
+      "actual_intent": "oil_change_when",
+      "intent_accurate": false,
+      "confidence": 0.5614064708871818,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0004885621416914461,
+        "balance": 0.000032386790610679915,
+        "bill_balance": 0.0008549709593307823,
+        "book_flight": 0.000007994214157359162,
+        "card_declined": 0.0012599216013196729,
+        "flight_status": 0.00005641863913543364,
+        "freeze_account": 0.000024408766208888922,
+        "international_fees": 0.00006180169525258479,
+        "jump_start": 0.0030686441149347537,
+        "lost_luggage": 0.00004282164515918091,
+        "oil_change_when": 0.4048953470476007,
+        "oos": 0.0026559535421373857,
+        "payday": 0.0001640446305847454,
+        "pto_balance": 0.0007902288715764586,
+        "pto_request": 0.00006180169525258479,
+        "redeem_rewards": 0.00003556067133404203,
+        "replacement_card_duration": 0.00009689540565999325,
+        "report_fraud": 0.00004282164515918091,
+        "schedule_maintenance": 0.5614064708871818,
+        "schedule_meeting": 0.005765759410103698,
+        "tire_pressure": 0.0022951629228669915,
+        "traffic": 0.012739279473890633,
+        "transfer": 0.00008104017415357471,
+        "travel_alert": 0.0012599216013196729,
+        "vaccines": 0.000105887905199435,
+        "w2": 0.0017058935481783384
+      },
+      "tie_break_triggered": true,
+      "finalist_options": [
+        "schedule_maintenance",
+        "traffic",
+        "oil_change_when",
+        "schedule_meeting"
+      ],
+      "wall_time_ms": 314
+    },
+    {
+      "id": "c150-14",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "what site can i get my w2",
+      "expected_domain": "work",
+      "actual_domain": "work",
+      "domain_accurate": true,
+      "expected_intent": "w2",
+      "pass1_intent": "w2",
+      "pass1_accurate": true,
+      "actual_intent": "w2",
+      "intent_accurate": true,
+      "confidence": 0.9928747155706318,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.000042833609121716,
+        "balance": 0.0003695582901753461,
+        "bill_balance": 0.0006463757407026194,
+        "book_flight": 0.00023904753539313647,
+        "card_declined": 0.00019124075899375098,
+        "flight_status": 0.0000643953684942523,
+        "freeze_account": 0.000020049037192766144,
+        "international_fees": 0.00003628597877265565,
+        "jump_start": 0.000059400332880918746,
+        "lost_luggage": 0.000005264814000108817,
+        "oil_change_when": 0.000021848535433449922,
+        "oos": 0.0014281485959202792,
+        "payday": 0.0026538758993449557,
+        "pto_balance": 0.00004651007783799789,
+        "pto_request": 0.000015454858367278133,
+        "redeem_rewards": 0.000028207420834065017,
+        "replacement_card_duration": 0.00020608836510944,
+        "report_fraud": 0.00009584730263774991,
+        "schedule_maintenance": 0.00005048178250753309,
+        "schedule_meeting": 0.0002768412802291415,
+        "tire_pressure": 0.00020608836510944,
+        "traffic": 0.00007559027689695228,
+        "transfer": 0.00019124075899375098,
+        "travel_alert": 0.00013080947729560748,
+        "vaccines": 0.000023799967123364896,
+        "w2": 0.9928747155706318
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 158
+    },
+    {
+      "id": "c150-15",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "please book a meeting room for thusday at 4:00",
+      "expected_domain": "work",
+      "actual_domain": "work",
+      "domain_accurate": true,
+      "expected_intent": "schedule_meeting",
+      "pass1_intent": "schedule_meeting",
+      "pass1_accurate": true,
+      "actual_intent": "schedule_meeting",
+      "intent_accurate": true,
+      "confidence": 0.9956500903492285,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00002256757451945913,
+        "balance": 0.00015300371151045916,
+        "bill_balance": 0.00020585601375586619,
+        "book_flight": 0.000024553752802212206,
+        "card_declined": 0.00007087660916651766,
+        "flight_status": 0.0001419248063232935,
+        "freeze_account": 0.00001471337372017043,
+        "international_fees": 0.00004381974541266941,
+        "jump_start": 0.000016040393893253134,
+        "lost_luggage": 0.000007268597359261088,
+        "oil_change_when": 0.00012197030445809553,
+        "oos": 0.0019989617425630055,
+        "payday": 0.00042054721222028584,
+        "pto_balance": 0.00002670393503725388,
+        "pto_request": 0.000010374487434844672,
+        "redeem_rewards": 0.000024553752802212206,
+        "replacement_card_duration": 0.00011300393358071254,
+        "report_fraud": 0.00004752371536045991,
+        "schedule_maintenance": 0.000019041355633924906,
+        "schedule_meeting": 0.9956500903492285,
+        "tire_pressure": 0.00003721076285880334,
+        "traffic": 0.00006548354235136055,
+        "transfer": 0.00017761338843126393,
+        "travel_alert": 0.00011300393358071254,
+        "vaccines": 0.00002256757451945913,
+        "w2": 0.0004507254314759867
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 167
+    },
+    {
+      "id": "c150-16",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "will i be charged transaction fees if i use my american card internationally",
+      "expected_domain": "credit_cards",
+      "actual_domain": "credit_cards",
+      "domain_accurate": true,
+      "expected_intent": "international_fees",
+      "pass1_intent": "international_fees",
+      "pass1_accurate": true,
+      "actual_intent": "international_fees",
+      "intent_accurate": true,
+      "confidence": 0.9750063623201227,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0036758201202385867,
+        "balance": 0.007902190211217968,
+        "bill_balance": 0.000979636907420923,
+        "book_flight": 0.0006724994889675405,
+        "card_declined": 0.002650226209353661,
+        "flight_status": 0.0006724994889675405,
+        "freeze_account": 0.00007745320825089928,
+        "international_fees": 0.9750063623201227,
+        "jump_start": 0.000018568145751874084,
+        "lost_luggage": 0.0003043712373805336,
+        "oil_change_when": 0.00005276835267712296,
+        "oos": 0.0004263161759137256,
+        "payday": 0.0000665105616658457,
+        "pto_balance": 0.00004171574329550869,
+        "pto_request": 0.0000012885258277500307,
+        "redeem_rewards": 0.0009209509641386439,
+        "replacement_card_duration": 0.0011072531606500446,
+        "report_fraud": 0.0001507353398119612,
+        "schedule_maintenance": 0.0000900531730804861,
+        "schedule_meeting": 0.0001507353398119612,
+        "tire_pressure": 0.0001507353398119612,
+        "traffic": 0.00039883890169715465,
+        "transfer": 0.0005194582289313532,
+        "travel_alert": 0.002959604630634059,
+        "vaccines": 0.000023771316959220395,
+        "w2": 0.000979636907420923
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 208
+    },
+    {
+      "id": "c150-17",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "i want to report fraudulent activity on my navy federal card",
+      "expected_domain": "banking",
+      "actual_domain": "credit_cards",
+      "expected_intent": "report_fraud",
+      "pass1_intent": "report_fraud",
+      "pass1_accurate": true,
+      "actual_intent": "report_fraud",
+      "intent_accurate": true,
+      "confidence": 0.9827180094670226,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0008271420609161393,
+        "balance": 0.010222421714908104,
+        "bill_balance": 0.00028741459462512946,
+        "book_flight": 0.0000053623956411509416,
+        "card_declined": 0.0018110846819769926,
+        "flight_status": 0.000028948363975036937,
+        "freeze_account": 0.00006118079396648112,
+        "international_fees": 0.001700085012105128,
+        "jump_start": 0.0000025264831417537755,
+        "lost_luggage": 0.00001581641311688189,
+        "oil_change_when": 0.0000036922610344806456,
+        "oos": 0.0000844369820100068,
+        "payday": 0.0000036922610344806456,
+        "pto_balance": 0.0000033600950570688837,
+        "pto_request": 5.767064892592288e-7,
+        "redeem_rewards": 0.00011579003308523848,
+        "replacement_card_duration": 0.0010805614613655693,
+        "report_fraud": 0.9827180094670226,
+        "schedule_maintenance": 0.00005638991595747823,
+        "schedule_meeting": 0.00001213385191127577,
+        "tire_pressure": 0.000014484780143783435,
+        "traffic": 0.000010148360134782703,
+        "transfer": 0.0007729468354492751,
+        "travel_alert": 0.00005638991595747823,
+        "vaccines": 0.000006447178423170257,
+        "w2": 0.0000989573805512296
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 207
+    },
+    {
+      "id": "c150-18",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "do i need to get any shots before i travel to morocco",
+      "expected_domain": "travel",
+      "actual_domain": "travel",
+      "domain_accurate": true,
+      "expected_intent": "vaccines",
+      "pass1_intent": "vaccines",
+      "pass1_accurate": true,
+      "actual_intent": "vaccines",
+      "intent_accurate": true,
+      "confidence": 0.7245174994464768,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00017336896475739338,
+        "balance": 0.0008821228426336997,
+        "bill_balance": 0.002168922353948642,
+        "book_flight": 0.0002054747102434896,
+        "card_declined": 0.0005506313510191315,
+        "flight_status": 0.000596218786328198,
+        "freeze_account": 0.0018742911313705576,
+        "international_fees": 0.000312018013796518,
+        "jump_start": 0.00017336896475739338,
+        "lost_luggage": 0.000050468933336129444,
+        "oil_change_when": 0.00010314216253955245,
+        "oos": 0.0011105231067356208,
+        "payday": 0.000038351118222471274,
+        "pto_balance": 0.0002054747102434896,
+        "pto_request": 0.00017336896475739338,
+        "redeem_rewards": 0.0001339632754618528,
+        "replacement_card_duration": 0.004103795980078424,
+        "report_fraud": 0.0001339632754618528,
+        "schedule_maintenance": 0.00022355859457003533,
+        "schedule_meeting": 0.00007912740498209701,
+        "tire_pressure": 0.00028724003687815387,
+        "traffic": 0.25570121614834856,
+        "transfer": 0.005769975629318132,
+        "travel_alert": 0.000243135712675259,
+        "vaccines": 0.7245174994464768,
+        "w2": 0.00018877838105913213
+      },
+      "tie_break_triggered": true,
+      "finalist_options": [
+        "vaccines",
+        "transfer",
+        "traffic"
+      ],
+      "wall_time_ms": 336
+    },
+    {
+      "id": "c150-19",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "i would like to know my vacation days balance",
+      "expected_domain": "work",
+      "actual_domain": "work",
+      "domain_accurate": true,
+      "expected_intent": "pto_balance",
+      "pass1_intent": "pto_balance",
+      "pass1_accurate": true,
+      "actual_intent": "pto_balance",
+      "intent_accurate": true,
+      "confidence": 0.9487636403572826,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00038311160549496097,
+        "balance": 0.0032869445515602514,
+        "bill_balance": 0.004223830196132564,
+        "book_flight": 0.0010588042480326135,
+        "card_declined": 0.0009871494052163632,
+        "flight_status": 0.0008570599317538792,
+        "freeze_account": 0.00024201578194332258,
+        "international_fees": 0.0004130179852416625,
+        "jump_start": 0.00038311160549496097,
+        "lost_luggage": 0.00006628039442999083,
+        "oil_change_when": 0.00024201578194332258,
+        "oos": 0.0077074879105583465,
+        "payday": 0.0020891576496304717,
+        "pto_balance": 0.9487636403572826,
+        "pto_request": 0.011496177023416411,
+        "redeem_rewards": 0.00044508399667055477,
+        "replacement_card_duration": 0.001303497115111367,
+        "report_fraud": 0.00044508399667055477,
+        "schedule_maintenance": 0.00022386856959894899,
+        "schedule_meeting": 0.0022314002963057886,
+        "tire_pressure": 0.00044508399667055477,
+        "traffic": 0.0032869445515602514,
+        "transfer": 0.0037288041383843145,
+        "travel_alert": 0.0004794493095470485,
+        "vaccines": 0.0009871494052163632,
+        "w2": 0.004223830196132564
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 133
+    },
+    {
+      "id": "c150-20",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "book me a flight to jackson, mississippi from austin texas on american airlines",
+      "expected_domain": "travel",
+      "actual_domain": "travel",
+      "domain_accurate": true,
+      "expected_intent": "book_flight",
+      "pass1_intent": "book_flight",
+      "pass1_accurate": true,
+      "actual_intent": "book_flight",
+      "intent_accurate": true,
+      "confidence": 0.9969549150299761,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0003459199608889704,
+        "balance": 0.00010680391266334533,
+        "bill_balance": 0.00003586159314695375,
+        "book_flight": 0.9969549150299761,
+        "card_declined": 0.00010680391266334533,
+        "flight_status": 0.0016516490443887347,
+        "freeze_account": 0.00011583177165900928,
+        "international_fees": 0.00005053544318876392,
+        "jump_start": 0.0000014318968139823532,
+        "lost_luggage": 0.000017716091917756587,
+        "oil_change_when": 0.0000034684752526367256,
+        "oos": 0.000019375642133052934,
+        "payday": 2.1159691192444073e-7,
+        "pto_balance": 0.000008532362968095745,
+        "pto_request": 4.5685712012677304e-9,
+        "redeem_rewards": 0.00003586159314695375,
+        "replacement_card_duration": 0.00010680391266334533,
+        "report_fraud": 0.000008532362968095745,
+        "schedule_maintenance": 0.000016192228355627092,
+        "schedule_meeting": 0.000004852158740681069,
+        "tire_pressure": 0.00003013726588047365,
+        "traffic": 0.00023614181109786456,
+        "transfer": 0.00010680391266334533,
+        "travel_alert": 0.00001026648679411117,
+        "vaccines": 6.074164580964659e-8,
+        "w2": 0.00002528622289982001
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 142
+    },
+    {
+      "id": "c150-21",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "what do i owe to jcp",
+      "expected_domain": "banking",
+      "actual_domain": "credit_cards",
+      "expected_intent": "bill_balance",
+      "pass1_intent": "bill_balance",
+      "pass1_accurate": true,
+      "actual_intent": "bill_balance",
+      "intent_accurate": true,
+      "confidence": 0.8708256929691905,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.005308523490954181,
+        "balance": 0.05682738759575137,
+        "bill_balance": 0.8708256929691905,
+        "book_flight": 0.00007317606197375872,
+        "card_declined": 0.006013268862034375,
+        "flight_status": 0.00043427490174323505,
+        "freeze_account": 0.000545418016594563,
+        "international_fees": 0.004679468729881317,
+        "jump_start": 0.000019446615269116095,
+        "lost_luggage": 0.000009248631472559755,
+        "oil_change_when": 0.000017745800717243883,
+        "oos": 0.0312312721480914,
+        "payday": 0.00006164334515789134,
+        "pto_balance": 0.000013453125037838952,
+        "pto_request": 0.0000030424148420496676,
+        "redeem_rewards": 0.002783594225780784,
+        "replacement_card_duration": 0.0016161666285818268,
+        "report_fraud": 0.000372328899185395,
+        "schedule_maintenance": 0.00007317606197375872,
+        "schedule_meeting": 0.00029469486749635374,
+        "tire_pressure": 0.00018263396825824954,
+        "traffic": 0.00016839958601891166,
+        "transfer": 0.017304420498608232,
+        "travel_alert": 0.00013169744392517874,
+        "vaccines": 0.00009436024258387725,
+        "w2": 0.0009154648688761238
+      },
+      "tie_break_triggered": true,
+      "finalist_options": [
+        "bill_balance",
+        "balance",
+        "pto_balance",
+        "card_declined",
+        "apr",
+        "oos"
+      ],
+      "wall_time_ms": 280
+    },
+    {
+      "id": "c150-22",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "how do i get stuff with visa points",
+      "expected_domain": "credit_cards",
+      "actual_domain": "credit_cards",
+      "domain_accurate": true,
+      "expected_intent": "redeem_rewards",
+      "pass1_intent": "redeem_rewards",
+      "pass1_accurate": true,
+      "actual_intent": "redeem_rewards",
+      "intent_accurate": true,
+      "confidence": 0.9845730780814883,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0005302416632048263,
+        "balance": 0.0034821304983748214,
+        "bill_balance": 0.0010989304158930601,
+        "book_flight": 0.00028220458389371385,
+        "card_declined": 0.0015077086073100334,
+        "flight_status": 0.00022710142356078633,
+        "freeze_account": 0.0003749584830853418,
+        "international_fees": 0.00026259444251456584,
+        "jump_start": 0.000035137728014176476,
+        "lost_luggage": 0.0000381536484666944,
+        "oil_change_when": 0.00001508619672239999,
+        "oos": 0.0027611830424939462,
+        "payday": 0.00004141176005925711,
+        "pto_balance": 0.000007452776755571983,
+        "pto_request": 8.11735915595826e-7,
+        "redeem_rewards": 0.9845730780814883,
+        "replacement_card_duration": 0.00026259444251456584,
+        "report_fraud": 0.00006714283350513532,
+        "schedule_maintenance": 0.00013493041556798571,
+        "schedule_meeting": 0.00026259444251456584,
+        "tire_pressure": 0.00009933975331234877,
+        "traffic": 0.0003749584830853418,
+        "transfer": 0.0021770550504853917,
+        "travel_alert": 0.00010730704064251661,
+        "vaccines": 0.00002738058754546033,
+        "w2": 0.0012485118630735828
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 153
+    },
+    {
+      "id": "c150-23",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "what should i do to get my car jump started",
+      "expected_domain": "auto_and_commute",
+      "actual_domain": "auto_and_commute",
+      "domain_accurate": true,
+      "expected_intent": "jump_start",
+      "pass1_intent": "jump_start",
+      "pass1_accurate": true,
+      "actual_intent": "jump_start",
+      "intent_accurate": true,
+      "confidence": 0.9809800514285794,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00010940950879310434,
+        "balance": 0.00026361066521240415,
+        "bill_balance": 0.000307713969136871,
+        "book_flight": 0.00007851664923815737,
+        "card_declined": 0.0007530165560024329,
+        "flight_status": 0.0007530165560024329,
+        "freeze_account": 0.000027924351598429803,
+        "international_fees": 0.00007219454337206079,
+        "jump_start": 0.9809800514285794,
+        "lost_luggage": 0.00002337358270406747,
+        "oil_change_when": 0.00048479873272059854,
+        "oos": 0.006628704422093769,
+        "payday": 0.000039666332186808486,
+        "pto_balance": 0.004061524418704474,
+        "pto_request": 0.000004834408646611526,
+        "redeem_rewards": 0.00016415543383511172,
+        "replacement_card_duration": 0.000522230454051174,
+        "report_fraud": 0.00012884166979406096,
+        "schedule_maintenance": 0.0006052712606994415,
+        "schedule_meeting": 0.00017781726991808241,
+        "tire_pressure": 0.0007530165560024329,
+        "traffic": 0.0012365208966044357,
+        "transfer": 0.0007004175314929503,
+        "travel_alert": 0.000192538976690176,
+        "vaccines": 0.00017781726991808241,
+        "w2": 0.0007530165560024329
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 138
+    },
+    {
+      "id": "c150-24",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "i need to know traffic patterns heading to the george washington bridge from manhattan",
+      "expected_domain": "auto_and_commute",
+      "actual_domain": "auto_and_commute",
+      "domain_accurate": true,
+      "expected_intent": "traffic",
+      "pass1_intent": "traffic",
+      "pass1_accurate": true,
+      "actual_intent": "traffic",
+      "intent_accurate": true,
+      "confidence": 0.997073447227156,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00004716970550661472,
+        "balance": 0.00017223514223768915,
+        "bill_balance": 0.0000989740690826324,
+        "book_flight": 0.00011618036300610711,
+        "card_declined": 0.00010725413735782963,
+        "flight_status": 0.0000989740690826324,
+        "freeze_account": 0.000060606159988213286,
+        "international_fees": 0.00003358065193054413,
+        "jump_start": 0.000011660065517506686,
+        "lost_luggage": 0.000010648683524231104,
+        "oil_change_when": 0.00002176246205905511,
+        "oos": 0.00012579907573091297,
+        "payday": 0.000015271583713791336,
+        "pto_balance": 0.00003081486818938254,
+        "pto_request": 0.0000013106869634318719,
+        "redeem_rewards": 0.000012762465839722958,
+        "replacement_card_duration": 0.00007148482514526624,
+        "report_fraud": 0.000252806154883862,
+        "schedule_maintenance": 0.0000989740690826324,
+        "schedule_meeting": 0.00007758923578115655,
+        "tire_pressure": 0.0008079049166502333,
+        "traffic": 0.997073447227156,
+        "transfer": 0.00036741007676887596,
+        "travel_alert": 0.0000989740690826324,
+        "vaccines": 0.000060606159988213286,
+        "w2": 0.00012579907573091297
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 141
+    },
+    {
+      "id": "c150-25",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "my card got declined; why",
+      "expected_domain": "credit_cards",
+      "actual_domain": "credit_cards",
+      "domain_accurate": true,
+      "expected_intent": "card_declined",
+      "pass1_intent": "card_declined",
+      "pass1_accurate": true,
+      "actual_intent": "card_declined",
+      "intent_accurate": true,
+      "confidence": 0.9965495252526978,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.0001650316206438875,
+        "balance": 0.001298331143812643,
+        "bill_balance": 0.00010474972724544687,
+        "book_flight": 0.0000267280605264466,
+        "card_declined": 0.9965495252526978,
+        "flight_status": 0.0000767526865490597,
+        "freeze_account": 0.0005929619463583098,
+        "international_fees": 0.00010474972724544687,
+        "jump_start": 0.0000035038394048289154,
+        "lost_luggage": 0.000007275164124301238,
+        "oil_change_when": 0.0000029074372015238497,
+        "oos": 0.00034111914618326647,
+        "payday": 0.000001645693851813206,
+        "pto_balance": 0.0000019925529490077123,
+        "pto_request": 4.3488766150519124e-7,
+        "redeem_rewards": 0.000029057117709294514,
+        "replacement_card_duration": 0.00003430033484728634,
+        "report_fraud": 0.00004385933406478777,
+        "schedule_maintenance": 0.000020752525161831242,
+        "schedule_meeting": 0.00001038386019523149,
+        "tire_pressure": 0.00003724438067040865,
+        "traffic": 0.000029057117709294514,
+        "transfer": 0.0003660225703400306,
+        "travel_alert": 0.000016054885478443806,
+        "vaccines": 0.000003844198261657966,
+        "w2": 0.00013171478910640308
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 142
+    },
+    {
+      "id": "c150-26",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "how much has the dow changed today",
+      "is_oos": true,
+      "expected_domain": "oos",
+      "actual_domain": "oos",
+      "domain_accurate": true,
+      "expected_intent": "oos",
+      "pass1_intent": "oos",
+      "pass1_accurate": true,
+      "actual_intent": "oos",
+      "intent_accurate": true,
+      "confidence": 0.9948865350898454,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00003815634005612698,
+        "balance": 0.0000497365916255426,
+        "bill_balance": 0.000011610753834626127,
+        "book_flight": 4.2341461807597945e-8,
+        "card_declined": 0.00001056621842395237,
+        "flight_status": 0.000007573520153727762,
+        "freeze_account": 0.000002579334629359765,
+        "international_fees": 0.0000038363496114872325,
+        "jump_start": 5.523966492352665e-7,
+        "lost_luggage": 1.1336885758844012e-8,
+        "oil_change_when": 2.763989785159719e-7,
+        "oos": 0.9948865350898454,
+        "payday": 0.0000014798670972521829,
+        "pto_balance": 4.2396840086032054e-7,
+        "pto_request": 6.276153958644706e-8,
+        "redeem_rewards": 1.892588350859704e-7,
+        "replacement_card_duration": 0.0019789086933690034,
+        "report_fraud": 0.0000022191360575886285,
+        "schedule_maintenance": 0.0000016385213366988388,
+        "schedule_meeting": 0.00009908277566740151,
+        "tire_pressure": 3.572997148827767e-8,
+        "traffic": 0.0000036518241896114408,
+        "transfer": 0.0004594880315562199,
+        "travel_alert": 1.5215454877687011e-7,
+        "vaccines": 0.002430624386851098,
+        "w2": 0.00001056621842395237
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 147
+    },
+    {
+      "id": "c150-27",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "how many prime numbers are there between 0 and 100",
+      "is_oos": true,
+      "expected_domain": "oos",
+      "actual_domain": "oos",
+      "domain_accurate": true,
+      "expected_intent": "oos",
+      "pass1_intent": "oos",
+      "pass1_accurate": true,
+      "actual_intent": "oos",
+      "intent_accurate": true,
+      "confidence": 0.9992086969963456,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 9.46074045143873e-7,
+        "balance": 0.000006852788845338901,
+        "bill_balance": 0.0000050996510251757705,
+        "book_flight": 1.023181637020661e-7,
+        "card_declined": 0.0000037821891339302017,
+        "flight_status": 0.000002167798082846862,
+        "freeze_account": 0.0000012945562940183075,
+        "international_fees": 2.4774259987703075e-7,
+        "jump_start": 0.0000024006438592574064,
+        "lost_luggage": 6.527337799908346e-8,
+        "oil_change_when": 1.3517595187217675e-7,
+        "oos": 0.9992086969963456,
+        "payday": 8.649478544227478e-8,
+        "pto_balance": 2.3456066353406928e-7,
+        "pto_request": 4.645016542704591e-8,
+        "redeem_rewards": 5.204295572784229e-8,
+        "replacement_card_duration": 0.0007083440294209476,
+        "report_fraud": 3.624354082254715e-7,
+        "schedule_maintenance": 7.662439868779234e-7,
+        "schedule_meeting": 0.000023693844248823952,
+        "tire_pressure": 4.0375448191810547e-7,
+        "traffic": 0.000006212347385647736,
+        "transfer": 0.0000087432444646546,
+        "travel_alert": 5.204295572784229e-8,
+        "vaccines": 0.00001481765893150746,
+        "w2": 0.0000043936424207393944
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 143
+    },
+    {
+      "id": "c150-28",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "can you tell me how to solve simple algebraic equations with one variable",
+      "is_oos": true,
+      "expected_domain": "oos",
+      "actual_domain": "oos",
+      "domain_accurate": true,
+      "expected_intent": "oos",
+      "pass1_intent": "oos",
+      "pass1_accurate": true,
+      "actual_intent": "oos",
+      "intent_accurate": true,
+      "confidence": 0.9936523300740212,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.000003806825733001151,
+        "balance": 0.00018628497964345777,
+        "bill_balance": 0.000059647784225854564,
+        "book_flight": 0.0000013574884981865219,
+        "card_declined": 0.00004246393864459756,
+        "flight_status": 0.00004246393864459756,
+        "freeze_account": 0.000011217623684638952,
+        "international_fees": 0.000009330180181518913,
+        "jump_start": 0.000016138595745332406,
+        "lost_luggage": 4.64637070531347e-7,
+        "oil_change_when": 0.000005846322130523547,
+        "oos": 0.9936523300740212,
+        "payday": 0.0000016574106694700803,
+        "pto_balance": 0.000002122664394597444,
+        "pto_request": 1.1056650840930222e-7,
+        "redeem_rewards": 0.000011217623684638952,
+        "replacement_card_duration": 0.0054766928920504795,
+        "report_fraud": 0.00001765741189960827,
+        "schedule_maintenance": 0.000006424266882611422,
+        "schedule_meeting": 0.000035742810802266226,
+        "tire_pressure": 0.000004610962197316387,
+        "traffic": 0.00001765741189960827,
+        "transfer": 0.00005482281369698861,
+        "travel_alert": 0.0000011675940541193165,
+        "vaccines": 0.00029629924439178993,
+        "w2": 0.00004246393864459756
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 139
+    },
+    {
+      "id": "c150-29",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "can you dim the brightness of my screen",
+      "is_oos": true,
+      "expected_domain": "oos",
+      "actual_domain": "oos",
+      "domain_accurate": true,
+      "expected_intent": "oos",
+      "pass1_intent": "oos",
+      "pass1_accurate": true,
+      "actual_intent": "oos",
+      "intent_accurate": true,
+      "confidence": 0.9934335607095298,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.00002686587996673119,
+        "balance": 0.0003313804935970695,
+        "bill_balance": 0.00012666500206473218,
+        "book_flight": 0.00000454843250538023,
+        "card_declined": 0.00007037984766250303,
+        "flight_status": 0.0003313804935970695,
+        "freeze_account": 0.000041870987101255454,
+        "international_fees": 0.00008341330771439035,
+        "jump_start": 0.00003212209703370038,
+        "lost_luggage": 0.0000018681902841831625,
+        "oil_change_when": 0.000012939052269405375,
+        "oos": 0.9934335607095298,
+        "payday": 0.0000032296492350713193,
+        "pto_balance": 0.000006688390234761827,
+        "pto_request": 7.842944909916102e-7,
+        "redeem_rewards": 0.0000073581416777369324,
+        "replacement_card_duration": 0.004794506997126906,
+        "report_fraud": 0.000049863873029031764,
+        "schedule_maintenance": 0.000022433971466945018,
+        "schedule_meeting": 0.00007037984766250303,
+        "tire_pressure": 0.000008895221423076974,
+        "traffic": 0.00009075453528548212,
+        "transfer": 0.00016196467657264395,
+        "travel_alert": 0.000011788828659058482,
+        "vaccines": 0.00017565513252986248,
+        "w2": 0.00009870194727957557
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 132
+    },
+    {
+      "id": "c150-30",
+      "dataset": "DeepPavlov/clinc150",
+      "text": "what is the account number to the internet service i have",
+      "is_oos": true,
+      "expected_domain": "oos",
+      "actual_domain": "oos",
+      "domain_accurate": true,
+      "expected_intent": "oos",
+      "pass1_intent": "oos",
+      "pass1_accurate": true,
+      "actual_intent": "oos",
+      "intent_accurate": true,
+      "confidence": 0.9954056318050842,
+      "logprob": 0,
+      "entropy": 0,
+      "top_probabilities": {
+        "apr": 0.000025467343226735233,
+        "balance": 0.0005097504578299323,
+        "bill_balance": 0.00012068525611353843,
+        "book_flight": 0.000007551422263242129,
+        "card_declined": 0.00007728173693199806,
+        "flight_status": 0.000021050320543522525,
+        "freeze_account": 0.000022080184246479688,
+        "international_fees": 0.000025467343226735233,
+        "jump_start": 0.00006448195584824338,
+        "lost_luggage": 0.000002224974243130358,
+        "oil_change_when": 0.000004567532525249083,
+        "oos": 0.9954056318050842,
+        "payday": 0.0000030348168315008784,
+        "pto_balance": 0.000010677259062670748,
+        "pto_request": 0.0000010099028079285315,
+        "redeem_rewards": 0.0000028824516905475918,
+        "replacement_card_duration": 0.0017798759727523533,
+        "report_fraud": 0.000005316282280319043,
+        "schedule_maintenance": 0.00001912689310683893,
+        "schedule_meeting": 0.00033771326836954296,
+        "tire_pressure": 0.000004805041544828726,
+        "traffic": 0.0002215059679681998,
+        "transfer": 0.0006494661834674278,
+        "travel_alert": 0.000015027151638856005,
+        "vaccines": 0.0005528383479783906,
+        "w2": 0.00011048012841763236
+      },
+      "tie_break_triggered": false,
+      "wall_time_ms": 122
+    }
+  ],
+  "samples_policy": 1,
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "target_url": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "tie_break_enabled": true,
+  "tie_breaks_triggered": 4,
+  "timestamp": "2026-09-28T22:40:05Z",
+  "total_cases": 30,
+  "workers": 4
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/jevbench.json b/benchmarks/runs/20260928-v010-verification/jevbench.json
new file mode 100644
index 0000000..c76dbe3
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/jevbench.json
@@ -0,0 +1,6042 @@
+{
+  "timestamp": "2026-09-28T22:40:15Z",
+  "idc_config": "none",
+  "source": "live-evaluation",
+  "target_url": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "target_model": "diffgemma-26b-a4b-it-q4",
+  "lock_commit_sha": "1df665e3956d7aab7fa0208ff6c4f2d8557f9f90",
+  "total_cases": 231,
+  "total_correct": 187,
+  "overall_accuracy_pct": 80.95238095238095,
+  "chance_baseline_pct": 31.760461760461773,
+  "chance_corrected_acc_pct": 72.08712201311059,
+  "paraphrase_pairs": 36,
+  "paraphrase_both_correct": 34,
+  "paraphrase_consistency_pct": 94.44444444444444,
+  "temperature_scale": 1,
+  "jev_parity": {
+    "protocol_version": "jevbench::v1.4",
+    "temperature_applied": 1,
+    "optimal_temperature": 1.75,
+    "chance_baseline_pct": 31.760461760461773,
+    "chance_corrected_acc_pct": 72.08712201311059,
+    "intelligence_score": 75.66581046370149,
+    "calibration_score": 74.6341055229255,
+    "speed_score": 82.79724900171038,
+    "cost_score": 57.55079956087546,
+    "composite_jevbench_score": 72.02380683212145,
+    "ece_10bin": 0.07348310899769057,
+    "brier_mean": 0.2566240466736764,
+    "soft_tvd_mean": 0.36035167154610914,
+    "probability_fidelity": 63.96483284538908,
+    "p50_latency_sec": 0.142,
+    "p95_latency_sec": 0.53,
+    "usd_per_1k_decisions": 0.026,
+    "presets": {
+      "Balanced 33:33:33 (no calibration)": 71.17415488299102,
+      "Emphasis on Accuracy 60:20:20": 72.93790194887347,
+      "Emphasis on Speed 20:60:20": 75.61357193863746,
+      "JevBench Score (25:25:25:25)": 76.02476293924009
+    },
+    "raw_t1_ece_10bin": 0.07348310899769057,
+    "raw_t1_brier_mean": 0.2566240466736764,
+    "raw_t1_calibration_score": 74.63410552292548,
+    "raw_t1_composite_score": 72.02380683212145,
+    "reliability_bins": [
+      {
+        "bin_index": 0,
+        "range_label": "[0.0, 0.1)",
+        "lower": 0,
+        "upper": 0.1,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 1,
+        "range_label": "[0.1, 0.2)",
+        "lower": 0.1,
+        "upper": 0.2,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 2,
+        "range_label": "[0.2, 0.3)",
+        "lower": 0.2,
+        "upper": 0.30000000000000004,
+        "count": 0,
+        "avg_confidence": 0,
+        "empirical_accuracy": 0,
+        "abs_gap": 0
+      },
+      {
+        "bin_index": 3,
+        "range_label": "[0.3, 0.4)",
+        "lower": 0.30000000000000004,
+        "upper": 0.4,
+        "count": 5,
+        "avg_confidence": 0.38693905084180685,
+        "empirical_accuracy": 0.2,
+        "abs_gap": 0.18693905084180684
+      },
+      {
+        "bin_index": 4,
+        "range_label": "[0.4, 0.5)",
+        "lower": 0.4,
+        "upper": 0.5,
+        "count": 9,
+        "avg_confidence": 0.4510346909504418,
+        "empirical_accuracy": 0.2222222222222222,
+        "abs_gap": 0.2288124687282196
+      },
+      {
+        "bin_index": 5,
+        "range_label": "[0.5, 0.6)",
+        "lower": 0.5,
+        "upper": 0.6000000000000001,
+        "count": 12,
+        "avg_confidence": 0.5419517629358149,
+        "empirical_accuracy": 0.4166666666666667,
+        "abs_gap": 0.12528509626914824
+      },
+      {
+        "bin_index": 6,
+        "range_label": "[0.6, 0.7)",
+        "lower": 0.6000000000000001,
+        "upper": 0.7000000000000001,
+        "count": 14,
+        "avg_confidence": 0.6531098716891751,
+        "empirical_accuracy": 0.5,
+        "abs_gap": 0.15310987168917511
+      },
+      {
+        "bin_index": 7,
+        "range_label": "[0.7, 0.8)",
+        "lower": 0.7000000000000001,
+        "upper": 0.8,
+        "count": 12,
+        "avg_confidence": 0.7476335058754326,
+        "empirical_accuracy": 0.5833333333333334,
+        "abs_gap": 0.16430017254209928
+      },
+      {
+        "bin_index": 8,
+        "range_label": "[0.8, 0.9)",
+        "lower": 0.8,
+        "upper": 0.9,
+        "count": 20,
+        "avg_confidence": 0.8511041262804284,
+        "empirical_accuracy": 0.7,
+        "abs_gap": 0.15110412628042846
+      },
+      {
+        "bin_index": 9,
+        "range_label": "[0.9, 1.0]",
+        "lower": 0.9,
+        "upper": 1,
+        "count": 159,
+        "avg_confidence": 0.983270105350387,
+        "empirical_accuracy": 0.949685534591195,
+        "abs_gap": 0.03358457075919197
+      }
+    ],
+    "endpoint_kind": "gpu",
+    "raw_speed_score": 91.23435786016154,
+    "adjusted_speed_score": 82.79724900171038,
+    "v14_intelligence_score": 75.66581046370149,
+    "v14_calibration_score": 92.65168910023095,
+    "v14_composite_score": 71.3414865564667,
+    "v14_public_only_estimate": true
+  },
+  "by_tier": [
+    {
+      "name": "easy",
+      "total": 48,
+      "correct": 48,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 28.437499999999975,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9707679636137944,
+      "avg_entropy_nats": 0.11399549455891696,
+      "avg_normalized_entropy": 0.10716851351412532,
+      "avg_brier_score": 0.006470111636364066,
+      "ece_10bin": 0.029232036386205786,
+      "avg_latency_ms": 153.58333333333334
+    },
+    {
+      "name": "hard",
+      "total": 111,
+      "correct": 69,
+      "accuracy_pct": 62.16216216216216,
+      "chance_baseline_pct": 33.61861861861862,
+      "chance_corrected_pct": 42.9993214204931,
+      "avg_confidence": 0.8185681635976575,
+      "avg_entropy_nats": 0.4284389116347307,
+      "avg_normalized_entropy": 0.372418937797444,
+      "avg_brier_score": 0.4984046760609831,
+      "ece_10bin": 0.19694654197603575,
+      "avg_latency_ms": 267.7477477477477
+    },
+    {
+      "name": "standard",
+      "total": 72,
+      "correct": 70,
+      "accuracy_pct": 97.22222222222221,
+      "chance_baseline_pct": 31.11111111111113,
+      "chance_corrected_pct": 95.96774193548389,
+      "avg_confidence": 0.92384263563423,
+      "avg_entropy_nats": 0.19974121224973918,
+      "avg_normalized_entropy": 0.17523422870622224,
+      "avg_brier_score": 0.050648199726453326,
+      "ece_10bin": 0.048379586587992504,
+      "avg_latency_ms": 123.40277777777777
+    }
+  ],
+  "by_family": [
+    {
+      "name": "adequacy",
+      "total": 12,
+      "correct": 12,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9472255281349686,
+      "avg_entropy_nats": 0.14312444995063858,
+      "avg_normalized_entropy": 0.20648493417374694,
+      "avg_brier_score": 0.02650333047154936,
+      "ece_10bin": 0.05277447186503143,
+      "avg_latency_ms": 121.16666666666667
+    },
+    {
+      "name": "adversarial",
+      "total": 6,
+      "correct": 5,
+      "accuracy_pct": 83.33333333333334,
+      "chance_baseline_pct": 33.33333333333333,
+      "chance_corrected_pct": 74.99999999999999,
+      "avg_confidence": 0.9133446245470555,
+      "avg_entropy_nats": 0.13995583096208697,
+      "avg_normalized_entropy": 0.10417882860719278,
+      "avg_brier_score": 0.08336448462296776,
+      "ece_10bin": 0.0860219968275978,
+      "avg_latency_ms": 175.83333333333334
+    },
+    {
+      "name": "ambiguous",
+      "total": 7,
+      "correct": 5,
+      "accuracy_pct": 71.42857142857143,
+      "chance_baseline_pct": 32.142857142857146,
+      "chance_corrected_pct": 57.89473684210525,
+      "avg_confidence": 0.8849264873305668,
+      "avg_entropy_nats": 0.29902731172747377,
+      "avg_normalized_entropy": 0.2717763296705983,
+      "avg_brier_score": 0.3340204913204884,
+      "ece_10bin": 0.24551278775092988,
+      "avg_latency_ms": 224.28571428571428
+    },
+    {
+      "name": "extraction",
+      "total": 24,
+      "correct": 24,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 24.375000000000004,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9757059410525257,
+      "avg_entropy_nats": 0.09339898718121555,
+      "avg_normalized_entropy": 0.0667400029457547,
+      "avg_brier_score": 0.006203365329269925,
+      "ece_10bin": 0.024294058947474242,
+      "avg_latency_ms": 130.79166666666666
+    },
+    {
+      "name": "fact",
+      "total": 12,
+      "correct": 12,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.951382517345912,
+      "avg_entropy_nats": 0.16003175453093663,
+      "avg_normalized_entropy": 0.23087701864654217,
+      "avg_brier_score": 0.012594470045164894,
+      "ece_10bin": 0.04861748265408805,
+      "avg_latency_ms": 165.16666666666666
+    },
+    {
+      "name": "intent",
+      "total": 24,
+      "correct": 24,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 20.000000000000007,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9840706374845295,
+      "avg_entropy_nats": 0.06605384556412432,
+      "avg_normalized_entropy": 0.04104156181099589,
+      "avg_brier_score": 0.004261782132656398,
+      "ece_10bin": 0.015929362515470665,
+      "avg_latency_ms": 154.70833333333334
+    },
+    {
+      "name": "judge_hard",
+      "total": 17,
+      "correct": 13,
+      "accuracy_pct": 76.47058823529412,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 52.941176470588225,
+      "avg_confidence": 0.9456338645025245,
+      "avg_entropy_nats": 0.13400614033157945,
+      "avg_normalized_entropy": 0.19332999410504018,
+      "avg_brier_score": 0.4169309095806367,
+      "ece_10bin": 0.1980340888999522,
+      "avg_latency_ms": 153.23529411764707
+    },
+    {
+      "name": "long_policy",
+      "total": 19,
+      "correct": 11,
+      "accuracy_pct": 57.89473684210527,
+      "chance_baseline_pct": 30.087719298245613,
+      "chance_corrected_pct": 39.7741530740276,
+      "avg_confidence": 0.7327298583690393,
+      "avg_entropy_nats": 0.6011870340491228,
+      "avg_normalized_entropy": 0.5074060538242634,
+      "avg_brier_score": 0.5133365935831226,
+      "ece_10bin": 0.20051040601663078,
+      "avg_latency_ms": 467.10526315789474
+    },
+    {
+      "name": "multi_hop",
+      "total": 18,
+      "correct": 12,
+      "accuracy_pct": 66.66666666666666,
+      "chance_baseline_pct": 25.185185185185183,
+      "chance_corrected_pct": 55.44554455445544,
+      "avg_confidence": 0.7981124625549145,
+      "avg_entropy_nats": 0.48678326350636364,
+      "avg_normalized_entropy": 0.3528019571789276,
+      "avg_brier_score": 0.4369780536229752,
+      "ece_10bin": 0.23457552363908588,
+      "avg_latency_ms": 363.8333333333333
+    },
+    {
+      "name": "ordinal",
+      "total": 12,
+      "correct": 11,
+      "accuracy_pct": 91.66666666666666,
+      "chance_baseline_pct": 25,
+      "chance_corrected_pct": 88.88888888888889,
+      "avg_confidence": 0.7771615772139023,
+      "avg_entropy_nats": 0.5098731961869937,
+      "avg_normalized_entropy": 0.3677957658105906,
+      "avg_brier_score": 0.1736471309653276,
+      "ece_10bin": 0.16290830250200858,
+      "avg_latency_ms": 123.66666666666667
+    },
+    {
+      "name": "policy",
+      "total": 12,
+      "correct": 12,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 50,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9251950687187208,
+      "avg_entropy_nats": 0.18257341013106512,
+      "avg_normalized_entropy": 0.26339775339427457,
+      "avg_brier_score": 0.03857292658803803,
+      "ece_10bin": 0.07480493128127928,
+      "avg_latency_ms": 121.91666666666667
+    },
+    {
+      "name": "probability",
+      "total": 10,
+      "correct": 5,
+      "accuracy_pct": 50,
+      "chance_baseline_pct": 38.333333333333336,
+      "chance_corrected_pct": 18.918918918918916,
+      "avg_confidence": 0.7926863702071273,
+      "avg_entropy_nats": 0.43809268509034094,
+      "avg_normalized_entropy": 0.47832735959106054,
+      "avg_brier_score": 0.6086937265636657,
+      "ece_10bin": 0.2926863702071273,
+      "avg_latency_ms": 264.9
+    },
+    {
+      "name": "routing",
+      "total": 12,
+      "correct": 11,
+      "accuracy_pct": 91.66666666666666,
+      "chance_baseline_pct": 16.666666666666664,
+      "chance_corrected_pct": 89.99999999999999,
+      "avg_confidence": 0.9244746255445907,
+      "avg_entropy_nats": 0.24110707559856326,
+      "avg_normalized_entropy": 0.13456442102825308,
+      "avg_brier_score": 0.05645626783158402,
+      "ece_10bin": 0.0613038170452441,
+      "avg_latency_ms": 124.16666666666667
+    },
+    {
+      "name": "routing_hard",
+      "total": 5,
+      "correct": 5,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 20,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.997675926782545,
+      "avg_entropy_nats": 0.017280278957315084,
+      "avg_normalized_entropy": 0.010736840995115205,
+      "avg_brier_score": 0.000011512809608990049,
+      "ece_10bin": 0.0023240732174549805,
+      "avg_latency_ms": 160.6
+    },
+    {
+      "name": "temporal_numeric",
+      "total": 15,
+      "correct": 2,
+      "accuracy_pct": 13.333333333333334,
+      "chance_baseline_pct": 30.888888888888893,
+      "avg_confidence": 0.6591355967923568,
+      "avg_entropy_nats": 0.8073202161899972,
+      "avg_normalized_entropy": 0.6304757693943938,
+      "avg_brier_score": 1.1550882785930592,
+      "ece_10bin": 0.5258022634590235,
+      "avg_latency_ms": 229.13333333333333
+    },
+    {
+      "name": "tool_selection",
+      "total": 12,
+      "correct": 12,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 20,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9811351942283505,
+      "avg_entropy_nats": 0.09881369984522546,
+      "avg_normalized_entropy": 0.06139640372692629,
+      "avg_brier_score": 0.001065224078659757,
+      "ece_10bin": 0.018864805771649507,
+      "avg_latency_ms": 127.66666666666667
+    },
+    {
+      "name": "tradeoff",
+      "total": 6,
+      "correct": 4,
+      "accuracy_pct": 66.66666666666666,
+      "chance_baseline_pct": 30.83333333333333,
+      "chance_corrected_pct": 51.80722891566265,
+      "avg_confidence": 0.8507963658861627,
+      "avg_entropy_nats": 0.432632268498232,
+      "avg_normalized_entropy": 0.38247183825290265,
+      "avg_brier_score": 0.5454422089519565,
+      "ece_10bin": 0.35642951310682,
+      "avg_latency_ms": 172.5
+    },
+    {
+      "name": "trap",
+      "total": 8,
+      "correct": 7,
+      "accuracy_pct": 87.5,
+      "chance_baseline_pct": 37.5,
+      "chance_corrected_pct": 80,
+      "avg_confidence": 0.8644737732517784,
+      "avg_entropy_nats": 0.373513893547024,
+      "avg_normalized_entropy": 0.3680386616490057,
+      "avg_brier_score": 0.13647384091648396,
+      "ece_10bin": 0.11213063112909925,
+      "avg_latency_ms": 142.75
+    }
+  ],
+  "by_topic": [
+    {
+      "name": "coding",
+      "total": 14,
+      "correct": 14,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 36.904761904761905,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9145964360765594,
+      "avg_entropy_nats": 0.21871895865009058,
+      "avg_normalized_entropy": 0.22914862112604104,
+      "avg_brier_score": 0.049151660305100674,
+      "ece_10bin": 0.08540356392344045,
+      "avg_latency_ms": 169.35714285714286
+    },
+    {
+      "name": "everyday_language",
+      "total": 49,
+      "correct": 49,
+      "accuracy_pct": 100,
+      "chance_baseline_pct": 29.18367346938776,
+      "chance_corrected_pct": 100,
+      "avg_confidence": 0.9771624662595045,
+      "avg_entropy_nats": 0.09745269521549484,
+      "avg_normalized_entropy": 0.09552590367567111,
+      "avg_brier_score": 0.003596854802827207,
+      "ece_10bin": 0.022837533740495505,
+      "avg_latency_ms": 144.53061224489795
+    },
+    {
+      "name": "finance_commerce",
+      "total": 42,
+      "correct": 30,
+      "accuracy_pct": 71.42857142857143,
+      "chance_baseline_pct": 29.761904761904756,
+      "chance_corrected_pct": 59.322033898305094,
+      "avg_confidence": 0.839402432025075,
+      "avg_entropy_nats": 0.3986479844728886,
+      "avg_normalized_entropy": 0.32596507910238826,
+      "avg_brier_score": 0.3319135465934463,
+      "ece_10bin": 0.12511671773936064,
+      "avg_latency_ms": 251.45238095238096
+    },
+    {
+      "name": "law_policy",
+      "total": 35,
+      "correct": 29,
+      "accuracy_pct": 82.85714285714286,
+      "chance_baseline_pct": 37.47619047619048,
+      "chance_corrected_pct": 72.5818735719726,
+      "avg_confidence": 0.839425854609324,
+      "avg_entropy_nats": 0.3678455320626675,
+      "avg_normalized_entropy": 0.3607394261524205,
+      "avg_brier_score": 0.2094403647936816,
+      "ece_10bin": 0.1015471129260463,
+      "avg_latency_ms": 247.4857142857143
+    },
+    {
+      "name": "math",
+      "total": 24,
+      "correct": 13,
+      "accuracy_pct": 54.166666666666664,
+      "chance_baseline_pct": 41.31944444444445,
+      "chance_corrected_pct": 21.893491124260343,
+      "avg_confidence": 0.8281677537500353,
+      "avg_entropy_nats": 0.38699174666544933,
+      "avg_normalized_entropy": 0.4060942761432873,
+      "avg_brier_score": 0.6375991076281532,
+      "ece_10bin": 0.3359989494293223,
+      "avg_latency_ms": 174.91666666666666
+    },
+    {
+      "name": "safety_security",
+      "total": 10,
+      "correct": 7,
+      "accuracy_pct": 70,
+      "chance_baseline_pct": 35.333333333333336,
+      "chance_corrected_pct": 53.6082474226804,
+      "avg_confidence": 0.9236870473102148,
+      "avg_entropy_nats": 0.16687379523509466,
+      "avg_normalized_entropy": 0.13968506308873535,
+      "avg_brier_score": 0.37554077505472516,
+      "ece_10bin": 0.22368704731021488,
+      "avg_latency_ms": 172.1
+    },
+    {
+      "name": "support_ops",
+      "total": 57,
+      "correct": 45,
+      "accuracy_pct": 78.94736842105263,
+      "chance_baseline_pct": 26.023391812865487,
+      "chance_corrected_pct": 71.5415019762846,
+      "avg_confidence": 0.8691505895999401,
+      "avg_entropy_nats": 0.33330305058293674,
+      "avg_normalized_entropy": 0.24124699542471173,
+      "avg_brier_score": 0.3173196069067155,
+      "ece_10bin": 0.0973052757299425,
+      "avg_latency_ms": 199.68421052631578
+    }
+  ],
+  "cases": [
+    {
+      "id": "easy-intent-00",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "track_order",
+      "actual": "track_order",
+      "accurate": true,
+      "confidence": 0.9881317092583755,
+      "entropy_nats": 0.07541702231221578,
+      "normalized_entropy": 0.04685923062304138,
+      "brier_score": 0.00021502100750504558,
+      "wall_time_ms": 243,
+      "top_probabilities": {
+        "billing_question": 0.0014214673653694541,
+        "cancel_order": 0.008362821453673975,
+        "change_address": 0.0011757885425782422,
+        "report_damage": 0.0009082133800029562,
+        "track_order": 0.9881317092583755
+      }
+    },
+    {
+      "id": "easy-intent-01",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "cancel_order",
+      "actual": "cancel_order",
+      "accurate": true,
+      "confidence": 0.9981575764650411,
+      "entropy_nats": 0.015512300739001408,
+      "normalized_entropy": 0.009638334364536458,
+      "brier_score": 0.000004666621258760573,
+      "wall_time_ms": 245,
+      "top_probabilities": {
+        "billing_question": 0.00037152465727324655,
+        "cancel_order": 0.9981575764650411,
+        "change_address": 0.00007015388836954282,
+        "report_damage": 0.00042824993492837166,
+        "track_order": 0.0009724950543877473
+      }
+    },
+    {
+      "id": "easy-intent-02",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "change_address",
+      "actual": "change_address",
+      "accurate": true,
+      "confidence": 0.9956343279529345,
+      "entropy_nats": 0.033095287656204904,
+      "normalized_entropy": 0.020563258390099603,
+      "brier_score": 0.000025801040094402647,
+      "wall_time_ms": 242,
+      "top_probabilities": {
+        "billing_question": 0.00018877097958161864,
+        "cancel_order": 0.0015250838131171673,
+        "change_address": 0.9956343279529345,
+        "report_damage": 0.001983314581229537,
+        "track_order": 0.0006685026731372368
+      }
+    },
+    {
+      "id": "easy-intent-03",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "report_damage",
+      "actual": "report_damage",
+      "accurate": true,
+      "confidence": 0.9982554588855101,
+      "entropy_nats": 0.014924160607274821,
+      "normalized_entropy": 0.009272902354278238,
+      "brier_score": 0.000004045891450579679,
+      "wall_time_ms": 243,
+      "top_probabilities": {
+        "billing_question": 0.00031413068437360973,
+        "cancel_order": 0.0005964485261848302,
+        "change_address": 0.00010052195732610426,
+        "report_damage": 0.9982554588855101,
+        "track_order": 0.00073343994660533
+      }
+    },
+    {
+      "id": "easy-intent-04",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "billing_question",
+      "actual": "billing_question",
+      "accurate": true,
+      "confidence": 0.9913277330014147,
+      "entropy_nats": 0.05894879213831493,
+      "normalized_entropy": 0.03662694390562807,
+      "brier_score": 0.00010349715705528451,
+      "wall_time_ms": 149,
+      "top_probabilities": {
+        "billing_question": 0.9913277330014147,
+        "cancel_order": 0.003644502935992922,
+        "change_address": 0.0000721819425767832,
+        "report_damage": 0.003644502935992922,
+        "track_order": 0.0013110791840226167
+      }
+    },
+    {
+      "id": "easy-intent-05",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "track_order",
+      "actual": "track_order",
+      "accurate": true,
+      "confidence": 0.9863705895742602,
+      "entropy_nats": 0.08802022136708056,
+      "normalized_entropy": 0.054690038483037544,
+      "brier_score": 0.0002543741013049746,
+      "wall_time_ms": 150,
+      "top_probabilities": {
+        "billing_question": 0.0073118132801476676,
+        "cancel_order": 0.0015010673205487221,
+        "change_address": 0.00160266686467801,
+        "report_damage": 0.0032138629603654205,
+        "track_order": 0.9863705895742602
+      }
+    },
+    {
+      "id": "easy-intent-06",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "set_alarm",
+      "actual": "set_alarm",
+      "accurate": true,
+      "confidence": 0.9930471836918924,
+      "entropy_nats": 0.04471482890182688,
+      "normalized_entropy": 0.027782885289560848,
+      "brier_score": 0.00008644837407291256,
+      "wall_time_ms": 150,
+      "top_probabilities": {
+        "play_music": 0.006149638514664827,
+        "send_message": 0.00019035208735472173,
+        "set_alarm": 0.9930471836918924,
+        "turn_off_lights": 0.00012661573488755853,
+        "weather": 0.0004862099712005122
+      }
+    },
+    {
+      "id": "easy-intent-07",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "play_music",
+      "actual": "play_music",
+      "accurate": true,
+      "confidence": 0.9963962586251148,
+      "entropy_nats": 0.028080825764024597,
+      "normalized_entropy": 0.017447598038470086,
+      "brier_score": 0.000017906025450782254,
+      "wall_time_ms": 149,
+      "top_probabilities": {
+        "play_music": 0.9963962586251148,
+        "send_message": 0.000553369556136765,
+        "set_alarm": 0.0020205390067272264,
+        "turn_off_lights": 0.0005149164060105947,
+        "weather": 0.0005149164060105947
+      }
+    },
+    {
+      "id": "easy-intent-08",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "weather",
+      "actual": "weather",
+      "accurate": true,
+      "confidence": 0.9991616599615895,
+      "entropy_nats": 0.007870308696560852,
+      "normalized_entropy": 0.004890097738941581,
+      "brier_score": 9.059283386858168e-7,
+      "wall_time_ms": 161,
+      "top_probabilities": {
+        "play_music": 0.0003267087347491685,
+        "send_message": 0.00019250666203296616,
+        "set_alarm": 0.00022435675710278264,
+        "turn_off_lights": 0.00009476788452565446,
+        "weather": 0.9991616599615895
+      }
+    },
+    {
+      "id": "easy-intent-09",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "send_message",
+      "actual": "send_message",
+      "accurate": true,
+      "confidence": 0.9956860003385859,
+      "entropy_nats": 0.03339794620473799,
+      "normalized_entropy": 0.020751310719546313,
+      "brier_score": 0.000024016220396277802,
+      "wall_time_ms": 159,
+      "top_probabilities": {
+        "play_music": 0.0013749579463996284,
+        "send_message": 0.9956860003385859,
+        "set_alarm": 0.0008977569343348338,
+        "turn_off_lights": 0.001579985161887761,
+        "weather": 0.0004612996187918972
+      }
+    },
+    {
+      "id": "easy-intent-10",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "turn_off_lights",
+      "actual": "turn_off_lights",
+      "accurate": true,
+      "confidence": 0.9970979445632864,
+      "entropy_nats": 0.022528839938434873,
+      "normalized_entropy": 0.013997955288851402,
+      "brier_score": 0.000012383691509158047,
+      "wall_time_ms": 159,
+      "top_probabilities": {
+        "play_music": 0.0017638552895731208,
+        "send_message": 0.0009076553413202595,
+        "set_alarm": 0.00010614561619930628,
+        "turn_off_lights": 0.9970979445632864,
+        "weather": 0.00012439918962090185
+      }
+    },
+    {
+      "id": "easy-intent-11",
+      "tier": "easy",
+      "family": "intent",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "set_alarm",
+      "actual": "set_alarm",
+      "accurate": true,
+      "confidence": 0.9771650315985824,
+      "entropy_nats": 0.12186830116290806,
+      "normalized_entropy": 0.07572103292794655,
+      "brier_score": 0.000891119025457673,
+      "wall_time_ms": 158,
+      "top_probabilities": {
+        "play_music": 0.018982114799360007,
+        "send_message": 0.00042653544338668747,
+        "set_alarm": 0.9771650315985824,
+        "turn_off_lights": 0.00042653544338668747,
+        "weather": 0.0029997827152843776
+      }
+    },
+    {
+      "id": "easy-fact-00",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "finance_commerce",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9858709245914467,
+      "entropy_nats": 0.07421187319159532,
+      "normalized_entropy": 0.10706510142859518,
+      "brier_score": 0.00039926154380117394,
+      "wall_time_ms": 159,
+      "top_probabilities": {
+        "no": 0.014129075408553304,
+        "yes": 0.9858709245914467
+      }
+    },
+    {
+      "id": "easy-fact-01",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "finance_commerce",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9910332499519221,
+      "entropy_nats": 0.05119776789652865,
+      "normalized_entropy": 0.07386276584890605,
+      "brier_score": 0.00016080521284940998,
+      "wall_time_ms": 158,
+      "top_probabilities": {
+        "no": 0.9910332499519221,
+        "yes": 0.00896675004807798
+      }
+    },
+    {
+      "id": "easy-fact-02",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "finance_commerce",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9883315169974276,
+      "entropy_nats": 0.06353496888941057,
+      "normalized_entropy": 0.0916615845397872,
+      "brier_score": 0.00027230699116264175,
+      "wall_time_ms": 158,
+      "top_probabilities": {
+        "no": 0.011668483002572371,
+        "yes": 0.9883315169974276
+      }
+    },
+    {
+      "id": "easy-fact-03",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "finance_commerce",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.8093748767888638,
+      "entropy_nats": 0.4871281311552818,
+      "normalized_entropy": 0.7027773390952337,
+      "brier_score": 0.07267587519852174,
+      "wall_time_ms": 157,
+      "top_probabilities": {
+        "no": 0.8093748767888638,
+        "yes": 0.19062512321113617
+      }
+    },
+    {
+      "id": "easy-fact-04",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "everyday_language",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9826046248124993,
+      "entropy_nats": 0.08772143866339961,
+      "normalized_entropy": 0.126555284539332,
+      "brier_score": 0.0006051981558278281,
+      "wall_time_ms": 156,
+      "top_probabilities": {
+        "no": 0.017395375187500617,
+        "yes": 0.9826046248124993
+      }
+    },
+    {
+      "id": "easy-fact-05",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "everyday_language",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9953749469466124,
+      "entropy_nats": 0.02947986309380083,
+      "normalized_entropy": 0.04253045229151203,
+      "brier_score": 0.000042782231493299225,
+      "wall_time_ms": 207,
+      "top_probabilities": {
+        "no": 0.9953749469466124,
+        "yes": 0.004625053053387517
+      }
+    },
+    {
+      "id": "easy-fact-06",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "support_ops",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9429998001539802,
+      "entropy_nats": 0.2186324129627778,
+      "normalized_entropy": 0.31541989795898745,
+      "brier_score": 0.0064980455649723875,
+      "wall_time_ms": 207,
+      "top_probabilities": {
+        "no": 0.057000199846019735,
+        "yes": 0.9429998001539802
+      }
+    },
+    {
+      "id": "easy-fact-07",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "support_ops",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9334400599541646,
+      "entropy_nats": 0.24464827754689492,
+      "normalized_entropy": 0.35295285677893207,
+      "brier_score": 0.008860451237810414,
+      "wall_time_ms": 207,
+      "top_probabilities": {
+        "no": 0.9334400599541646,
+        "yes": 0.06655994004583544
+      }
+    },
+    {
+      "id": "easy-fact-08",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "safety_security",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9795278243043928,
+      "entropy_nats": 0.09987109207334285,
+      "normalized_entropy": 0.1440835292623768,
+      "brier_score": 0.0008382199554236219,
+      "wall_time_ms": 156,
+      "top_probabilities": {
+        "no": 0.020472175695607237,
+        "yes": 0.9795278243043928
+      }
+    },
+    {
+      "id": "easy-fact-09",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "safety_security",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.994491816768475,
+      "entropy_nats": 0.03414391288344786,
+      "normalized_entropy": 0.04925925379349502,
+      "brier_score": 0.00006068016502410685,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "no": 0.994491816768475,
+        "yes": 0.005508183231525107
+      }
+    },
+    {
+      "id": "easy-fact-10",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "everyday_language",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9873197931940412,
+      "entropy_nats": 0.06798297537815233,
+      "normalized_entropy": 0.09807870144293687,
+      "brier_score": 0.0003215752892837691,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "no": 0.012680206805958808,
+        "yes": 0.9873197931940412
+      }
+    },
+    {
+      "id": "easy-fact-11",
+      "tier": "easy",
+      "family": "fact",
+      "topic": "everyday_language",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.8262207736871171,
+      "entropy_nats": 0.46182834063660694,
+      "normalized_entropy": 0.6662774567784118,
+      "brier_score": 0.060398438995808344,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "no": 0.8262207736871171,
+        "yes": 0.17377922631288295
+      }
+    },
+    {
+      "id": "easy-extraction-00",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "finance_commerce",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "paypal",
+      "actual": "paypal",
+      "accurate": true,
+      "confidence": 0.9979963872051787,
+      "entropy_nats": 0.015295148869012527,
+      "normalized_entropy": 0.011033117711491404,
+      "brier_score": 0.000007117178477309917,
+      "wall_time_ms": 175,
+      "top_probabilities": {
+        "bank_transfer": 0.000022200283568156438,
+        "cash": 0.00023597993990932034,
+        "credit_card": 0.0017454325713438922,
+        "paypal": 0.9979963872051787
+      }
+    },
+    {
+      "id": "easy-extraction-01",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "finance_commerce",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "cash",
+      "actual": "cash",
+      "accurate": true,
+      "confidence": 0.7553673508086226,
+      "entropy_nats": 0.7219249595968757,
+      "normalized_entropy": 0.520758779552189,
+      "brier_score": 0.09754393899274198,
+      "wall_time_ms": 138,
+      "top_probabilities": {
+        "bank_transfer": 0.18993873011573179,
+        "cash": 0.7553673508086226,
+        "credit_card": 0.019397986256730598,
+        "paypal": 0.03529593281891509
+      }
+    },
+    {
+      "id": "easy-extraction-02",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "finance_commerce",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "bank_transfer",
+      "actual": "bank_transfer",
+      "accurate": true,
+      "confidence": 0.924553521461875,
+      "entropy_nats": 0.32129336171509526,
+      "normalized_entropy": 0.23176416980845593,
+      "brier_score": 0.009011399619824285,
+      "wall_time_ms": 140,
+      "top_probabilities": {
+        "bank_transfer": 0.924553521461875,
+        "cash": 0.0037381382778221787,
+        "credit_card": 0.01669416247367173,
+        "paypal": 0.05501417778663111
+      }
+    },
+    {
+      "id": "easy-extraction-03",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "support_ops",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "high",
+      "actual": "high",
+      "accurate": true,
+      "confidence": 0.9974061230810629,
+      "entropy_nats": 0.020455864619494824,
+      "normalized_entropy": 0.014755787221820593,
+      "brier_score": 0.000009687072005229962,
+      "wall_time_ms": 138,
+      "top_probabilities": {
+        "high": 0.9974061230810629,
+        "low": 0.0015001199431034858,
+        "medium": 0.00031197155970230975,
+        "urgent": 0.0007817854161312445
+      }
+    },
+    {
+      "id": "easy-extraction-04",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "support_ops",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "low",
+      "actual": "low",
+      "accurate": true,
+      "confidence": 0.8579056564240805,
+      "entropy_nats": 0.44863619348148004,
+      "normalized_entropy": 0.3236226057495164,
+      "brier_score": 0.03792356438878542,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "high": 0.005502888284722798,
+        "low": 0.8579056564240805,
+        "medium": 0.13300224703799157,
+        "urgent": 0.0035892082532052298
+      }
+    },
+    {
+      "id": "easy-extraction-05",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "support_ops",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "urgent",
+      "actual": "urgent",
+      "accurate": true,
+      "confidence": 0.995102353774945,
+      "entropy_nats": 0.03595424562489407,
+      "normalized_entropy": 0.02593550593096919,
+      "brier_score": 0.000033109785496879595,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "high": 0.0016664772624185565,
+        "low": 0.0008654264805029476,
+        "medium": 0.002365742482133422,
+        "urgent": 0.995102353774945
+      }
+    },
+    {
+      "id": "easy-extraction-06",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "finance_commerce",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "M",
+      "actual": "M",
+      "accurate": true,
+      "confidence": 0.9957609259715712,
+      "entropy_nats": 0.02992047427553475,
+      "normalized_entropy": 0.021583059929179894,
+      "brier_score": 0.000029992152361445204,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "L": 0.00016490103350856353,
+        "M": 0.9957609259715712,
+        "S": 0.0033964555782885117,
+        "XL": 0.0006777174166318023
+      }
+    },
+    {
+      "id": "easy-extraction-07",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "finance_commerce",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "XL",
+      "actual": "XL",
+      "accurate": true,
+      "confidence": 0.9906330704825991,
+      "entropy_nats": 0.05714608817019186,
+      "normalized_entropy": 0.041222189004669625,
+      "brier_score": 0.0001536992080458591,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "L": 0.00004903147715272768,
+        "M": 0.0013014333141354649,
+        "S": 0.00801646472611269,
+        "XL": 0.9906330704825991
+      }
+    },
+    {
+      "id": "easy-extraction-08",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "thursday",
+      "actual": "thursday",
+      "accurate": true,
+      "confidence": 0.9892746095842199,
+      "entropy_nats": 0.0718086206303573,
+      "normalized_entropy": 0.044617204600179046,
+      "brier_score": 0.0001547732058376603,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "friday": 0.0035563861564715856,
+        "monday": 0.0004625562653210739,
+        "thursday": 0.9892746095842199,
+        "tuesday": 0.0018718153554988235,
+        "wednesday": 0.004834632638488701
+      }
+    },
+    {
+      "id": "easy-extraction-09",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "monday",
+      "actual": "monday",
+      "accurate": true,
+      "confidence": 0.9984026920943598,
+      "entropy_nats": 0.013128741902270109,
+      "normalized_entropy": 0.008157345990697031,
+      "brier_score": 0.000004015326428530406,
+      "wall_time_ms": 122,
+      "top_probabilities": {
+        "friday": 0.00002071222817494126,
+        "monday": 0.9984026920943598,
+        "thursday": 0.00010288760324554223,
+        "tuesday": 0.0011652296221996512,
+        "wednesday": 0.00030847845202008314
+      }
+    },
+    {
+      "id": "easy-extraction-10",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "friday",
+      "actual": "friday",
+      "accurate": true,
+      "confidence": 0.9901985605134686,
+      "entropy_nats": 0.0669934842917724,
+      "normalized_entropy": 0.04162539217834879,
+      "brier_score": 0.00012907818327521223,
+      "wall_time_ms": 122,
+      "top_probabilities": {
+        "friday": 0.9901985605134686,
+        "monday": 0.00494304158113277,
+        "thursday": 0.0022095731302019936,
+        "tuesday": 0.0016293365025093157,
+        "wednesday": 0.001019488272687275
+      }
+    },
+    {
+      "id": "easy-extraction-11",
+      "tier": "easy",
+      "family": "extraction",
+      "topic": "finance_commerce",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "credit_card",
+      "actual": "credit_card",
+      "accurate": true,
+      "confidence": 0.9976169892523986,
+      "entropy_nats": 0.01870226764850598,
+      "normalized_entropy": 0.013490834394938837,
+      "brier_score": 0.000008468862405057734,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "bank_transfer": 0.00010687483001308914,
+        "cash": 0.0008312271247158925,
+        "credit_card": 0.9976169892523986,
+        "paypal": 0.0014449087928724877
+      }
+    },
+    {
+      "id": "easy-tool_selection-00",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "get_weather",
+      "actual": "get_weather",
+      "accurate": true,
+      "confidence": 0.9674876522361793,
+      "entropy_nats": 0.15387981462205766,
+      "normalized_entropy": 0.0956109045482414,
+      "brier_score": 0.001972124345305369,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "convert_currency": 0.0006134850758828735,
+        "create_calendar_event": 0.0013312317456261084,
+        "get_weather": 0.9674876522361793,
+        "send_email": 0.030212529675302347,
+        "translate_text": 0.00035510126700944784
+      }
+    },
+    {
+      "id": "easy-tool_selection-01",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "send_email",
+      "actual": "send_email",
+      "accurate": true,
+      "confidence": 0.9988766494714733,
+      "entropy_nats": 0.009716361988863045,
+      "normalized_entropy": 0.00603711514050772,
+      "brier_score": 0.0000018886891278845716,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "convert_currency": 0.00005017967223069764,
+        "create_calendar_event": 0.00034282672095444915,
+        "get_weather": 0.0007115990295428177,
+        "send_email": 0.9988766494714733,
+        "translate_text": 0.000018745105798942887
+      }
+    },
+    {
+      "id": "easy-tool_selection-02",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "create_calendar_event",
+      "actual": "create_calendar_event",
+      "accurate": true,
+      "confidence": 0.9942395634011578,
+      "entropy_nats": 0.0414477914982493,
+      "normalized_entropy": 0.025752960818205165,
+      "brier_score": 0.00004570834730025216,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "convert_currency": 0.0010932303965751858,
+        "create_calendar_event": 0.9942395634011578,
+        "get_weather": 0.00183231856391287,
+        "send_email": 0.0028236585094483495,
+        "translate_text": 0.000011229128905836059
+      }
+    },
+    {
+      "id": "easy-tool_selection-03",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "convert_currency",
+      "actual": "convert_currency",
+      "accurate": true,
+      "confidence": 0.9619366531313809,
+      "entropy_nats": 0.18100964445272222,
+      "normalized_entropy": 0.11246761559069077,
+      "brier_score": 0.002538552432607384,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "convert_currency": 0.9619366531313809,
+        "create_calendar_event": 0.0003637830294733621,
+        "get_weather": 0.003860188510060248,
+        "send_email": 0.03276501825449058,
+        "translate_text": 0.0010743570745948925
+      }
+    },
+    {
+      "id": "easy-tool_selection-04",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "translate_text",
+      "actual": "translate_text",
+      "accurate": true,
+      "confidence": 0.9961363730310521,
+      "entropy_nats": 0.030539714466815138,
+      "normalized_entropy": 0.018975391489707814,
+      "brier_score": 0.00001893908005089985,
+      "wall_time_ms": 134,
+      "top_probabilities": {
+        "convert_currency": 0.0006649651729674,
+        "create_calendar_event": 0.0009349472685759328,
+        "get_weather": 0.0013896943136480757,
+        "send_email": 0.0008740202137565925,
+        "translate_text": 0.9961363730310521
+      }
+    },
+    {
+      "id": "easy-tool_selection-05",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "create_calendar_event",
+      "actual": "create_calendar_event",
+      "accurate": true,
+      "confidence": 0.9952128374142776,
+      "entropy_nats": 0.034367267695113904,
+      "normalized_entropy": 0.02135358402433626,
+      "brier_score": 0.00003365539215059724,
+      "wall_time_ms": 132,
+      "top_probabilities": {
+        "convert_currency": 0.00010037452957672794,
+        "create_calendar_event": 0.9952128374142776,
+        "get_weather": 0.0024456860129112522,
+        "send_email": 0.0021778452626930707,
+        "translate_text": 0.00006325678054150829
+      }
+    },
+    {
+      "id": "easy-tool_selection-06",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "search_flights",
+      "actual": "search_flights",
+      "accurate": true,
+      "confidence": 0.9493873091838346,
+      "entropy_nats": 0.20848115450785631,
+      "normalized_entropy": 0.12953662449305123,
+      "brier_score": 0.0049733202541173555,
+      "wall_time_ms": 137,
+      "top_probabilities": {
+        "book_restaurant": 0.04909858314536507,
+        "call_taxi": 0.000651602212867629,
+        "get_stock_price": 0.0007540315278222324,
+        "search_flights": 0.9493873091838346,
+        "set_reminder": 0.0001084739301104551
+      }
+    },
+    {
+      "id": "easy-tool_selection-07",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "book_restaurant",
+      "actual": "book_restaurant",
+      "accurate": true,
+      "confidence": 0.9955133837459514,
+      "entropy_nats": 0.031540482548144545,
+      "normalized_entropy": 0.019597203660029972,
+      "brier_score": 0.000033351401184986104,
+      "wall_time_ms": 135,
+      "top_probabilities": {
+        "book_restaurant": 0.9955133837459514,
+        "call_taxi": 0.00009608608155653892,
+        "get_stock_price": 0.0007916895544691928,
+        "search_flights": 0.0035472538238524856,
+        "set_reminder": 0.00005158679417042546
+      }
+    },
+    {
+      "id": "easy-tool_selection-08",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "set_reminder",
+      "actual": "set_reminder",
+      "accurate": true,
+      "confidence": 0.960172377006294,
+      "entropy_nats": 0.21388027448160185,
+      "normalized_entropy": 0.1328912863486179,
+      "brier_score": 0.002124294140883184,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "book_restaurant": 0.01718921210416772,
+        "call_taxi": 0.001397261782056821,
+        "get_stock_price": 0.013362838819052363,
+        "search_flights": 0.007878310288429155,
+        "set_reminder": 0.960172377006294
+      }
+    },
+    {
+      "id": "easy-tool_selection-09",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "get_stock_price",
+      "actual": "get_stock_price",
+      "accurate": true,
+      "confidence": 0.9868314593626202,
+      "entropy_nats": 0.0805542274445604,
+      "normalized_entropy": 0.05005115563776602,
+      "brier_score": 0.0002710505964080745,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "book_restaurant": 0.009539200182716388,
+        "call_taxi": 0.0021501624274604526,
+        "get_stock_price": 0.9868314593626202,
+        "search_flights": 0.001420254673551968,
+        "set_reminder": 0.00005892335365115736
+      }
+    },
+    {
+      "id": "easy-tool_selection-10",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "call_taxi",
+      "actual": "call_taxi",
+      "accurate": true,
+      "confidence": 0.989258583698834,
+      "entropy_nats": 0.07123728987065193,
+      "normalized_entropy": 0.04426221683998562,
+      "brier_score": 0.00015544990254498132,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "book_restaurant": 0.0030528848412292157,
+        "call_taxi": 0.989258583698834,
+        "get_stock_price": 0.004735133285548273,
+        "search_flights": 0.0028854227854530675,
+        "set_reminder": 0.00006797538893546553
+      }
+    },
+    {
+      "id": "easy-tool_selection-11",
+      "tier": "easy",
+      "family": "tool_selection",
+      "topic": "everyday_language",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "set_reminder",
+      "actual": "set_reminder",
+      "accurate": true,
+      "confidence": 0.9785694890571505,
+      "entropy_nats": 0.12911037456606905,
+      "normalized_entropy": 0.08022078613197549,
+      "brier_score": 0.0006143543622361152,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "book_restaurant": 0.003957492891049975,
+        "call_taxi": 0.0011967683978870046,
+        "get_stock_price": 0.009801766462854055,
+        "search_flights": 0.00647448319105849,
+        "set_reminder": 0.9785694890571505
+      }
+    },
+    {
+      "id": "original-policy-01-0",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-01",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9988938313223493,
+      "entropy_nats": 0.008635084092256084,
+      "normalized_entropy": 0.012457792997557029,
+      "brier_score": 0.0000024472182868308916,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "no": 0.9988938313223493,
+        "yes": 0.0011061686776507702
+      }
+    },
+    {
+      "id": "original-policy-01-1",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-01",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9989618000956377,
+      "entropy_nats": 0.008170371254471637,
+      "normalized_entropy": 0.011787354091047971,
+      "brier_score": 0.00000215571808283565,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "no": 0.9989618000956377,
+        "yes": 0.0010381999043622212
+      }
+    },
+    {
+      "id": "original-policy-02-0",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9922712766884041,
+      "entropy_nats": 0.04528210478465792,
+      "normalized_entropy": 0.06532826801384038,
+      "brier_score": 0.00011946632805441206,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "no": 0.007728723311595917,
+        "yes": 0.9922712766884041
+      }
+    },
+    {
+      "id": "original-policy-02-1",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.8336065447945751,
+      "entropy_nats": 0.4501212231446924,
+      "normalized_entropy": 0.6493876564297222,
+      "brier_score": 0.0553735638703995,
+      "wall_time_ms": 121,
+      "top_probabilities": {
+        "no": 0.16639345520542492,
+        "yes": 0.8336065447945751
+      }
+    },
+    {
+      "id": "original-policy-03-0",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9791121960770974,
+      "entropy_nats": 0.10147446471436751,
+      "normalized_entropy": 0.14639670702028013,
+      "brier_score": 0.000872600705443248,
+      "wall_time_ms": 121,
+      "top_probabilities": {
+        "no": 0.020887803922902612,
+        "yes": 0.9791121960770974
+      }
+    },
+    {
+      "id": "original-policy-03-1",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.5967117888558712,
+      "entropy_nats": 0.6743224165815859,
+      "normalized_entropy": 0.9728416063625157,
+      "brier_score": 0.32528276249566285,
+      "wall_time_ms": 119,
+      "top_probabilities": {
+        "no": 0.4032882111441288,
+        "yes": 0.5967117888558712
+      }
+    },
+    {
+      "id": "original-policy-04-0",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-04",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9978720069335191,
+      "entropy_nats": 0.015218366277566888,
+      "normalized_entropy": 0.021955461559077585,
+      "brier_score": 0.000009056708981981794,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "no": 0.9978720069335191,
+        "yes": 0.0021279930664809743
+      }
+    },
+    {
+      "id": "original-policy-04-1",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-04",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9980914702549898,
+      "entropy_nats": 0.013856817666972377,
+      "normalized_entropy": 0.019991162130643624,
+      "brier_score": 0.000007284971575177492,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "no": 0.9980914702549898,
+        "yes": 0.0019085297450101828
+      }
+    },
+    {
+      "id": "original-policy-05-0",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-05",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.995475077260146,
+      "entropy_nats": 0.028940903100041934,
+      "normalized_entropy": 0.04175289738127853,
+      "brier_score": 0.00004094985160329663,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "no": 0.995475077260146,
+        "yes": 0.004524922739854142
+      }
+    },
+    {
+      "id": "original-policy-05-1",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-05",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.8499363872992839,
+      "entropy_nats": 0.42281941459671185,
+      "normalized_entropy": 0.6099994726302508,
+      "brier_score": 0.04503817571358105,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "no": 0.8499363872992839,
+        "yes": 0.15006361270071616
+      }
+    },
+    {
+      "id": "original-policy-06-0",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9957408206206955,
+      "entropy_nats": 0.027499588230608486,
+      "normalized_entropy": 0.039673519566787364,
+      "brier_score": 0.000036281217970185914,
+      "wall_time_ms": 122,
+      "top_probabilities": {
+        "no": 0.004259179379304562,
+        "yes": 0.9957408206206955
+      }
+    },
+    {
+      "id": "original-policy-06-1",
+      "tier": "standard",
+      "family": "policy",
+      "topic": "law_policy",
+      "group": "original-policy-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.8656676244220795,
+      "entropy_nats": 0.3945401671288489,
+      "normalized_entropy": 0.5692011425482931,
+      "brier_score": 0.03609037425681501,
+      "wall_time_ms": 119,
+      "top_probabilities": {
+        "no": 0.13433237557792058,
+        "yes": 0.8656676244220795
+      }
+    },
+    {
+      "id": "original-intent-01-0",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-01",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "refund",
+      "actual": "refund",
+      "accurate": true,
+      "confidence": 0.9840724634836164,
+      "entropy_nats": 0.09179363901045791,
+      "normalized_entropy": 0.057034594687551496,
+      "brier_score": 0.000407616407510484,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "cancel": 0.003969340576651861,
+        "change_address": 0.00017802867092610712,
+        "other": 0.011753378730511558,
+        "refund": 0.9840724634836164,
+        "status": 0.00002678853829393049
+      }
+    },
+    {
+      "id": "original-intent-01-1",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-01",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "refund",
+      "actual": "refund",
+      "accurate": true,
+      "confidence": 0.984955178113406,
+      "entropy_nats": 0.08921513217692424,
+      "normalized_entropy": 0.05543247831287634,
+      "brier_score": 0.0003382406830707779,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "cancel": 0.006823658552989601,
+        "change_address": 0.00010595052882115264,
+        "other": 0.008082040305168318,
+        "refund": 0.984955178113406,
+        "status": 0.00003317249961491096
+      }
+    },
+    {
+      "id": "original-intent-02-0",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "support_ops",
+      "group": "original-intent-02",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "status",
+      "actual": "status",
+      "accurate": true,
+      "confidence": 0.9921068801853851,
+      "entropy_nats": 0.051765267736519634,
+      "normalized_entropy": 0.032163569241531215,
+      "brier_score": 0.00010222038953907022,
+      "wall_time_ms": 122,
+      "top_probabilities": {
+        "cancel": 0.0009779991629711119,
+        "change_address": 0.00034617423747439815,
+        "other": 0.006222772176694978,
+        "refund": 0.00034617423747439815,
+        "status": 0.9921068801853851
+      }
+    },
+    {
+      "id": "original-intent-02-1",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "support_ops",
+      "group": "original-intent-02",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "status",
+      "actual": "status",
+      "accurate": true,
+      "confidence": 0.77211646729418,
+      "entropy_nats": 0.5873352986659299,
+      "normalized_entropy": 0.36493193936114565,
+      "brier_score": 0.09947358702793016,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "cancel": 0.005538159663729253,
+        "change_address": 0.0017963112981892644,
+        "other": 0.21794959828462307,
+        "refund": 0.0025994634592783773,
+        "status": 0.77211646729418
+      }
+    },
+    {
+      "id": "original-intent-03-0",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-03",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "change_address",
+      "actual": "change_address",
+      "accurate": true,
+      "confidence": 0.9970264437400275,
+      "entropy_nats": 0.020430290784881736,
+      "normalized_entropy": 0.012694053387858334,
+      "brier_score": 0.00001754375753428864,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "cancel": 0.000002661808273719511,
+        "change_address": 0.9970264437400275,
+        "other": 0.0029498284949729123,
+        "refund": 0.000009194915776017019,
+        "status": 0.000011871040949994759
+      }
+    },
+    {
+      "id": "original-intent-03-1",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-03",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "change_address",
+      "actual": "change_address",
+      "accurate": true,
+      "confidence": 0.9966696019875689,
+      "entropy_nats": 0.02286258813885488,
+      "normalized_entropy": 0.014205324705118753,
+      "brier_score": 0.000021535979780982264,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "cancel": 0.000008872709837438002,
+        "change_address": 0.9966696019875689,
+        "other": 0.0032311389374241792,
+        "refund": 0.00004339674796750475,
+        "status": 0.0000469896172019406
+      }
+    },
+    {
+      "id": "original-intent-04-0",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-04",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "cancel",
+      "actual": "cancel",
+      "accurate": true,
+      "confidence": 0.9974478998252598,
+      "entropy_nats": 0.019155168829293526,
+      "normalized_entropy": 0.01190177557102741,
+      "brier_score": 0.000011134959338927212,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "cancel": 0.9974478998252598,
+        "change_address": 0.000021504884728984524,
+        "other": 0.0021150606768439175,
+        "refund": 0.0003830747548705304,
+        "status": 0.00003245985829666741
+      }
+    },
+    {
+      "id": "original-intent-04-1",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-04",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "cancel",
+      "actual": "cancel",
+      "accurate": true,
+      "confidence": 0.9964798585892186,
+      "entropy_nats": 0.025735101090759757,
+      "normalized_entropy": 0.01599011735211221,
+      "brier_score": 0.0000206673142705784,
+      "wall_time_ms": 127,
+      "top_probabilities": {
+        "cancel": 0.9964798585892186,
+        "change_address": 0.0001586119239310948,
+        "other": 0.002837474414482522,
+        "refund": 0.0004383554466053425,
+        "status": 0.00008569962576238387
+      }
+    },
+    {
+      "id": "original-intent-05-0",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "everyday_language",
+      "group": "original-intent-05",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "other",
+      "actual": "other",
+      "accurate": true,
+      "confidence": 0.9971907283986183,
+      "entropy_nats": 0.021746145726798574,
+      "normalized_entropy": 0.013511640032084176,
+      "brier_score": 0.000011687751560398708,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "cancel": 0.0008901961329498273,
+        "change_address": 0.00018170835626066283,
+        "other": 0.9971907283986183,
+        "refund": 0.0017233926606903072,
+        "status": 0.000013974451480773818
+      }
+    },
+    {
+      "id": "original-intent-05-1",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "everyday_language",
+      "group": "original-intent-05",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "other",
+      "actual": "other",
+      "accurate": true,
+      "confidence": 0.9985292914643763,
+      "entropy_nats": 0.012496289048458883,
+      "normalized_entropy": 0.007764380938162194,
+      "brier_score": 0.0000030430829708982543,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "cancel": 0.0006069989968999911,
+        "change_address": 0.00016541970977257617,
+        "other": 0.9985292914643763,
+        "refund": 0.0006959038861797664,
+        "status": 0.0000023859427713689474
+      }
+    },
+    {
+      "id": "original-intent-06-0",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-06",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "status",
+      "actual": "status",
+      "accurate": true,
+      "confidence": 0.9953540735832292,
+      "entropy_nats": 0.03381138985098493,
+      "normalized_entropy": 0.021008197700431247,
+      "brier_score": 0.0000331668341824689,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "cancel": 0.0004767853597287562,
+        "change_address": 0.0003105828954897928,
+        "other": 0.0033102534388214397,
+        "refund": 0.0005483047227308099,
+        "status": 0.9953540735832292
+      }
+    },
+    {
+      "id": "original-intent-06-1",
+      "tier": "standard",
+      "family": "intent",
+      "topic": "finance_commerce",
+      "group": "original-intent-06",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "status",
+      "actual": "status",
+      "accurate": true,
+      "confidence": 0.9893149390472282,
+      "entropy_nats": 0.06456714699053388,
+      "normalized_entropy": 0.040117824050064206,
+      "brier_score": 0.0002021419121699623,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "cancel": 0.0006535508021661272,
+        "change_address": 0.0003690481487056506,
+        "other": 0.009343795319015642,
+        "refund": 0.0003186666828844091,
+        "status": 0.9893149390472282
+      }
+    },
+    {
+      "id": "original-ordinal-01-0",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-01",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.8941593995245184,
+      "entropy_nats": 0.35829059615109404,
+      "normalized_entropy": 0.25845203313216686,
+      "brier_score": 0.021449608260172993,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.8941593995245184,
+        "1": 0.10113611726148404,
+        "2": 0.004326482635651483,
+        "3": 0.0003780005783460648
+      }
+    },
+    {
+      "id": "original-ordinal-01-1",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-01",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.9783282932829165,
+      "entropy_nats": 0.10958069789329479,
+      "normalized_entropy": 0.07904576471390404,
+      "brier_score": 0.0008933326951040273,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.9783282932829165,
+        "1": 0.020567857042741715,
+        "2": 0.00044275137356325294,
+        "3": 0.0006610983007786191
+      }
+    },
+    {
+      "id": "original-ordinal-02-0",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-02",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "1",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.6109112548394805,
+      "entropy_nats": 0.8166477193936954,
+      "normalized_entropy": 0.5890868074612831,
+      "brier_score": 0.2711783676471991,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.34327958232990075,
+        "1": 0.6109112548394805,
+        "2": 0.04409661638601528,
+        "3": 0.0017125464446034783
+      }
+    },
+    {
+      "id": "original-ordinal-02-1",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-02",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "1",
+      "actual": "0",
+      "accurate": false,
+      "confidence": 0.5626850708999533,
+      "entropy_nats": 0.7722301292088023,
+      "normalized_entropy": 0.5570462889172912,
+      "brier_score": 0.6576508269287135,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.5626850708999533,
+        "1": 0.41637010545552394,
+        "2": 0.020299448259603028,
+        "3": 0.000645375384919626
+      }
+    },
+    {
+      "id": "original-ordinal-03-0",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-03",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "2",
+      "actual": "2",
+      "accurate": true,
+      "confidence": 0.7159193246366097,
+      "entropy_nats": 0.8127234865852918,
+      "normalized_entropy": 0.5862560718552943,
+      "brier_score": 0.12072085412647234,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.005497750470874489,
+        "1": 0.11495449744979848,
+        "2": 0.7159193246366097,
+        "3": 0.16362842744271744
+      }
+    },
+    {
+      "id": "original-ordinal-03-1",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-03",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "2",
+      "actual": "2",
+      "accurate": true,
+      "confidence": 0.49759144529493704,
+      "entropy_nats": 1.0211837043064558,
+      "normalized_entropy": 0.7366283330197727,
+      "brier_score": 0.40845553716682226,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.01723631829256129,
+        "1": 0.10465852639635358,
+        "2": 0.49759144529493704,
+        "3": 0.38051371001614814
+      }
+    },
+    {
+      "id": "original-ordinal-04-0",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-04",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9603357769174673,
+      "entropy_nats": 0.19577937555510896,
+      "normalized_entropy": 0.14122496711084684,
+      "brier_score": 0.0025064046670124213,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.0034767880773751134,
+        "1": 0.00655463623910146,
+        "2": 0.02963279876605619,
+        "3": 0.9603357769174673
+      }
+    },
+    {
+      "id": "original-ordinal-04-1",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-04",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "3",
+      "actual": "3",
+      "accurate": true,
+      "confidence": 0.9622225782297332,
+      "entropy_nats": 0.1959486213371924,
+      "normalized_entropy": 0.1413470521360984,
+      "brier_score": 0.0020758714778017157,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "0": 0.0068594363175229385,
+        "1": 0.007593717671712377,
+        "2": 0.02332426778103142,
+        "3": 0.9622225782297332
+      }
+    },
+    {
+      "id": "original-ordinal-05-0",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-05",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.9644550665300328,
+      "entropy_nats": 0.15953971789488694,
+      "normalized_entropy": 0.1150835799158888,
+      "brier_score": 0.002444568202966484,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "0": 0.9644550665300328,
+        "1": 0.03435714844509504,
+        "2": 0.000652380422022917,
+        "3": 0.0005354046028493007
+      }
+    },
+    {
+      "id": "original-ordinal-05-1",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-05",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "0",
+      "actual": "0",
+      "accurate": true,
+      "confidence": 0.9727287511855139,
+      "entropy_nats": 0.1310602284982751,
+      "normalized_entropy": 0.09453997085611794,
+      "brier_score": 0.0014215510866015702,
+      "wall_time_ms": 127,
+      "top_probabilities": {
+        "0": 0.9727287511855139,
+        "1": 0.0260198780612777,
+        "2": 0.000706481401288459,
+        "3": 0.0005448893519199389
+      }
+    },
+    {
+      "id": "original-ordinal-06-0",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-06",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "2",
+      "actual": "2",
+      "accurate": true,
+      "confidence": 0.6288677578301529,
+      "entropy_nats": 0.7569899723607287,
+      "normalized_entropy": 0.5460528395637484,
+      "brier_score": 0.2594539848110542,
+      "wall_time_ms": 127,
+      "top_probabilities": {
+        "0": 0.004929702509976285,
+        "1": 0.3483864652340152,
+        "2": 0.6288677578301529,
+        "3": 0.017816074425855703
+      }
+    },
+    {
+      "id": "original-ordinal-06-1",
+      "tier": "standard",
+      "family": "ordinal",
+      "topic": "support_ops",
+      "group": "original-ordinal-06",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "2",
+      "actual": "2",
+      "accurate": true,
+      "confidence": 0.5777342073955117,
+      "entropy_nats": 0.7885041050590968,
+      "normalized_entropy": 0.5687854810446746,
+      "brier_score": 0.3355146645140104,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "0": 0.0025820162643924414,
+        "1": 0.39576171234243457,
+        "2": 0.5777342073955117,
+        "3": 0.023922063997661277
+      }
+    },
+    {
+      "id": "original-extraction-01-0",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-01",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "pickup",
+      "actual": "pickup",
+      "accurate": true,
+      "confidence": 0.9988352653282243,
+      "entropy_nats": 0.009668308864147561,
+      "normalized_entropy": 0.006974210626044247,
+      "brier_score": 0.0000023109274832199648,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "courier": 0.0009622193975722116,
+        "pickup": 0.9988352653282243,
+        "post": 0.00003821701378453118,
+        "unknown": 0.00016429826041897242
+      }
+    },
+    {
+      "id": "original-extraction-01-1",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-01",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "pickup",
+      "actual": "pickup",
+      "accurate": true,
+      "confidence": 0.9984507082891237,
+      "entropy_nats": 0.01203391074190371,
+      "normalized_entropy": 0.008680631674922456,
+      "brier_score": 0.000004473502443227341,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "courier": 0.0014369770427798893,
+        "pickup": 0.9984507082891237,
+        "post": 0.000024634929109483297,
+        "unknown": 0.00008767973898685356
+      }
+    },
+    {
+      "id": "original-extraction-02-0",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-02",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "unknown",
+      "actual": "unknown",
+      "accurate": true,
+      "confidence": 0.998983160140019,
+      "entropy_nats": 0.008696325768770987,
+      "normalized_entropy": 0.006273073030280403,
+      "brier_score": 0.000001652527265223348,
+      "wall_time_ms": 118,
+      "top_probabilities": {
+        "courier": 0.0007472926286070918,
+        "pickup": 0.000243837442494817,
+        "post": 0.000025709788879189773,
+        "unknown": 0.998983160140019
+      }
+    },
+    {
+      "id": "original-extraction-02-1",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-02",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "unknown",
+      "actual": "unknown",
+      "accurate": true,
+      "confidence": 0.9992879484479188,
+      "entropy_nats": 0.006334561745631366,
+      "normalized_entropy": 0.004569420408313654,
+      "brier_score": 8.037600033281634e-7,
+      "wall_time_ms": 121,
+      "top_probabilities": {
+        "courier": 0.0005088031226399012,
+        "pickup": 0.00019437929777170563,
+        "post": 0.00000886913166968162,
+        "unknown": 0.9992879484479188
+      }
+    },
+    {
+      "id": "original-extraction-03-0",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-03",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "courier",
+      "actual": "courier",
+      "accurate": true,
+      "confidence": 0.9992994747042864,
+      "entropy_nats": 0.006272232416953733,
+      "normalized_entropy": 0.004524459301621074,
+      "brier_score": 7.452075923297783e-7,
+      "wall_time_ms": 130,
+      "top_probabilities": {
+        "courier": 0.9992994747042864,
+        "pickup": 0.0004220888453747516,
+        "post": 0.000002197276831876438,
+        "unknown": 0.0002762391735070281
+      }
+    },
+    {
+      "id": "original-extraction-03-1",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-03",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "courier",
+      "actual": "courier",
+      "accurate": true,
+      "confidence": 0.9977613651365439,
+      "entropy_nats": 0.01744094775272737,
+      "normalized_entropy": 0.012580984415631644,
+      "brier_score": 0.000007789563036527194,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "courier": 0.9977613651365439,
+        "pickup": 0.0015193083732658863,
+        "post": 0.00003480654451356885,
+        "unknown": 0.0006845199456767253
+      }
+    },
+    {
+      "id": "original-extraction-04-0",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-04",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "post",
+      "actual": "post",
+      "accurate": true,
+      "confidence": 0.9469813749392951,
+      "entropy_nats": 0.26471798405580094,
+      "normalized_entropy": 0.19095366141571385,
+      "brier_score": 0.0037784155928118615,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "courier": 0.01466253360936583,
+        "pickup": 0.02208116662624641,
+        "post": 0.9469813749392951,
+        "unknown": 0.01627492482509268
+      }
+    },
+    {
+      "id": "original-extraction-04-1",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-04",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "post",
+      "actual": "post",
+      "accurate": true,
+      "confidence": 0.9971034153215057,
+      "entropy_nats": 0.022808909600352485,
+      "normalized_entropy": 0.0164531503842566,
+      "brier_score": 0.000011571357030688578,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "courier": 0.0005981788108221955,
+        "pickup": 0.0008475395659826176,
+        "post": 0.9971034153215057,
+        "unknown": 0.001450866301689447
+      }
+    },
+    {
+      "id": "original-extraction-05-0",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-05",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "unknown",
+      "actual": "unknown",
+      "accurate": true,
+      "confidence": 0.998315039572763,
+      "entropy_nats": 0.013161113366558884,
+      "normalized_entropy": 0.009493736493255977,
+      "brier_score": 0.000005018802523053369,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "courier": 0.0014617226663076357,
+        "pickup": 0.00020690838552388913,
+        "post": 0.000016329375405484587,
+        "unknown": 0.998315039572763
+      }
+    },
+    {
+      "id": "original-extraction-05-1",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-05",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "unknown",
+      "actual": "unknown",
+      "accurate": true,
+      "confidence": 0.9947910270908331,
+      "entropy_nats": 0.034657664517352714,
+      "normalized_entropy": 0.025000220363989075,
+      "brier_score": 0.000048713394799114556,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "courier": 0.0005149041219012901,
+        "pickup": 0.0046161453303367,
+        "post": 0.00007792345692889299,
+        "unknown": 0.9947910270908331
+      }
+    },
+    {
+      "id": "original-extraction-06-0",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-06",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "courier",
+      "actual": "courier",
+      "accurate": true,
+      "confidence": 0.9974021856812857,
+      "entropy_nats": 0.01996079381892963,
+      "normalized_entropy": 0.014398669127388426,
+      "brier_score": 0.000010073249841790499,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "courier": 0.9974021856812857,
+        "pickup": 0.0012892315812188609,
+        "post": 0.000019351156276640264,
+        "unknown": 0.0012892315812188609
+      }
+    },
+    {
+      "id": "original-extraction-06-1",
+      "tier": "standard",
+      "family": "extraction",
+      "topic": "support_ops",
+      "group": "original-extraction-06",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "courier",
+      "actual": "courier",
+      "accurate": true,
+      "confidence": 0.9995133799544408,
+      "entropy_nats": 0.004563488874559549,
+      "normalized_entropy": 0.0032918613842395092,
+      "brier_score": 3.56041963013602e-7,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "courier": 0.9995133799544408,
+        "pickup": 0.0001884163117066643,
+        "post": 0.000008960056044316984,
+        "unknown": 0.0002892436778083285
+      }
+    },
+    {
+      "id": "original-adequacy-01-0",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "math",
+      "group": "original-adequacy-01",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9902575416022018,
+      "entropy_nats": 0.05481472110396465,
+      "normalized_entropy": 0.07908092630440142,
+      "brier_score": 0.00018983099126565706,
+      "wall_time_ms": 132,
+      "top_probabilities": {
+        "no": 0.00974245839779828,
+        "yes": 0.9902575416022018
+      }
+    },
+    {
+      "id": "original-adequacy-01-1",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "math",
+      "group": "original-adequacy-01",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9799340701825623,
+      "entropy_nats": 0.0982955895144033,
+      "normalized_entropy": 0.14181055953368685,
+      "brier_score": 0.0008052830788766716,
+      "wall_time_ms": 132,
+      "top_probabilities": {
+        "no": 0.02006592981743771,
+        "yes": 0.9799340701825623
+      }
+    },
+    {
+      "id": "original-adequacy-02-0",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9877585429878064,
+      "entropy_nats": 0.06606446401108718,
+      "normalized_entropy": 0.09531087460778287,
+      "brier_score": 0.00029970653956276725,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "no": 0.9877585429878064,
+        "yes": 0.012241457012193677
+      }
+    },
+    {
+      "id": "original-adequacy-02-1",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9955236168630563,
+      "entropy_nats": 0.028678836481591162,
+      "normalized_entropy": 0.04137481517045706,
+      "brier_score": 0.000040076011977426965,
+      "wall_time_ms": 131,
+      "top_probabilities": {
+        "no": 0.9955236168630563,
+        "yes": 0.004476383136943684
+      }
+    },
+    {
+      "id": "original-adequacy-03-0",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "coding",
+      "group": "original-adequacy-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9828019974842712,
+      "entropy_nats": 0.08692409301938496,
+      "normalized_entropy": 0.12540495793283765,
+      "brier_score": 0.0005915425810620271,
+      "wall_time_ms": 120,
+      "top_probabilities": {
+        "no": 0.9828019974842712,
+        "yes": 0.01719800251572889
+      }
+    },
+    {
+      "id": "original-adequacy-03-1",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "coding",
+      "group": "original-adequacy-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.6212218589658707,
+      "entropy_nats": 0.6634627979798786,
+      "normalized_entropy": 0.9571744884598871,
+      "brier_score": 0.2869457602505415,
+      "wall_time_ms": 115,
+      "top_probabilities": {
+        "no": 0.6212218589658707,
+        "yes": 0.3787781410341293
+      }
+    },
+    {
+      "id": "original-adequacy-04-0",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-04",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9955043991505841,
+      "entropy_nats": 0.028782656335601925,
+      "normalized_entropy": 0.041524595558984204,
+      "brier_score": 0.000040420853994538195,
+      "wall_time_ms": 115,
+      "top_probabilities": {
+        "no": 0.0044956008494159705,
+        "yes": 0.9955043991505841
+      }
+    },
+    {
+      "id": "original-adequacy-04-1",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-04",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9891958079556399,
+      "entropy_nats": 0.0596650637821808,
+      "normalized_entropy": 0.08607849163287594,
+      "brier_score": 0.0002334611314628289,
+      "wall_time_ms": 114,
+      "top_probabilities": {
+        "no": 0.01080419204436013,
+        "yes": 0.9891958079556399
+      }
+    },
+    {
+      "id": "original-adequacy-05-0",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-05",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9399555680229332,
+      "entropy_nats": 0.2270897611567966,
+      "normalized_entropy": 0.3276212722575696,
+      "brier_score": 0.007210667622897197,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "no": 0.9399555680229332,
+        "yes": 0.06004443197706678
+      }
+    },
+    {
+      "id": "original-adequacy-05-1",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-05",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9969712143216,
+      "entropy_nats": 0.020589920020744796,
+      "normalized_entropy": 0.0297049755062289,
+      "brier_score": 0.00001834708537136234,
+      "wall_time_ms": 117,
+      "top_probabilities": {
+        "no": 0.9969712143216,
+        "yes": 0.0030287856784000475
+      }
+    },
+    {
+      "id": "original-adequacy-06-0",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9912961225187344,
+      "entropy_nats": 0.04995696706744025,
+      "normalized_entropy": 0.07207266864604932,
+      "brier_score": 0.0001515149664177632,
+      "wall_time_ms": 117,
+      "top_probabilities": {
+        "no": 0.008703877481265623,
+        "yes": 0.9912961225187344
+      }
+    },
+    {
+      "id": "original-adequacy-06-1",
+      "tier": "standard",
+      "family": "adequacy",
+      "topic": "everyday_language",
+      "group": "original-adequacy-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.8962855975643628,
+      "entropy_nats": 0.333168528934589,
+      "normalized_entropy": 0.4806605844742027,
+      "brier_score": 0.02151335454516263,
+      "wall_time_ms": 117,
+      "top_probabilities": {
+        "no": 0.10371440243563727,
+        "yes": 0.8962855975643628
+      }
+    },
+    {
+      "id": "original-routing-01-0",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "math",
+      "group": "original-routing-01",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "math",
+      "actual": "math",
+      "accurate": true,
+      "confidence": 0.946616620002462,
+      "entropy_nats": 0.21842543862158376,
+      "normalized_entropy": 0.12190555840382311,
+      "brier_score": 0.005493250831280094,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "coding": 0.051391776874425706,
+        "coding_agent": 0.00017844476707977973,
+        "document": 0.00004182173751868964,
+        "general": 0.001497400565693913,
+        "math": 0.946616620002462,
+        "tools": 0.00027393605281989203
+      }
+    },
+    {
+      "id": "original-routing-01-1",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "math",
+      "group": "original-routing-01",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "math",
+      "actual": "math",
+      "accurate": true,
+      "confidence": 0.9698544932042932,
+      "entropy_nats": 0.15208415973695413,
+      "normalized_entropy": 0.08487978567931144,
+      "brier_score": 0.0015458352915301392,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "coding": 0.024805929804806087,
+        "coding_agent": 0.0003843956083637954,
+        "document": 0.00012255946115309109,
+        "general": 0.0046422556990444435,
+        "math": 0.9698544932042932,
+        "tools": 0.00019036622233953573
+      }
+    },
+    {
+      "id": "original-routing-02-0",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "coding",
+      "group": "original-routing-02",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "coding",
+      "actual": "coding",
+      "accurate": true,
+      "confidence": 0.9988639508844733,
+      "entropy_nats": 0.009833050511966777,
+      "normalized_entropy": 0.005487929982143841,
+      "brier_score": 0.0000019611596884951016,
+      "wall_time_ms": 119,
+      "top_probabilities": {
+        "coding": 0.9988639508844733,
+        "coding_agent": 0.000019379523688944913,
+        "document": 0.0000029811548177634696,
+        "general": 0.0002572532655289707,
+        "math": 0.0007726357895871254,
+        "tools": 0.00008379938190387093
+      }
+    },
+    {
+      "id": "original-routing-02-1",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "coding",
+      "group": "original-routing-02",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "coding",
+      "actual": "coding",
+      "accurate": true,
+      "confidence": 0.9975631492961307,
+      "entropy_nats": 0.01964791221803443,
+      "normalized_entropy": 0.010965708598431103,
+      "brier_score": 0.000008385117930399221,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "coding": 0.9975631492961307,
+        "coding_agent": 0.0001548017787163279,
+        "document": 0.00000987331489133913,
+        "general": 0.0008758880065741091,
+        "math": 0.0012815931005167216,
+        "tools": 0.00011469450317081424
+      }
+    },
+    {
+      "id": "original-routing-03-0",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "coding",
+      "group": "original-routing-03",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "coding_agent",
+      "actual": "coding_agent",
+      "accurate": true,
+      "confidence": 0.9962759181235729,
+      "entropy_nats": 0.028789191248871245,
+      "normalized_entropy": 0.016067553565811216,
+      "brier_score": 0.000019698202983876052,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "coding": 0.0022580911428939487,
+        "coding_agent": 0.9962759181235729,
+        "document": 0.00023808516473006885,
+        "general": 0.00046664962688737987,
+        "math": 0.0006689332149590458,
+        "tools": 0.00009232272695669904
+      }
+    },
+    {
+      "id": "original-routing-03-1",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "coding",
+      "group": "original-routing-03",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "coding_agent",
+      "actual": "coding_agent",
+      "accurate": true,
+      "confidence": 0.993974990676316,
+      "entropy_nats": 0.04252813261747446,
+      "normalized_entropy": 0.023735402741193203,
+      "brier_score": 0.00005640657043891342,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "coding": 0.00440840228186726,
+        "coding_agent": 0.993974990676316,
+        "document": 0.00032865403439290313,
+        "general": 0.0003527840109257219,
+        "math": 0.0004998478750770539,
+        "tools": 0.000435321121421043
+      }
+    },
+    {
+      "id": "original-routing-04-0",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "law_policy",
+      "group": "original-routing-04",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "document",
+      "actual": "document",
+      "accurate": true,
+      "confidence": 0.9179308479762062,
+      "entropy_nats": 0.38659802020038675,
+      "normalized_entropy": 0.2157644632775096,
+      "brier_score": 0.008960426331345717,
+      "wall_time_ms": 119,
+      "top_probabilities": {
+        "coding": 0.034963195936525776,
+        "coding_agent": 0.0012805908238224147,
+        "document": 0.9179308479762062,
+        "general": 0.02921529539117919,
+        "math": 0.010488488520771742,
+        "tools": 0.00612158135149461
+      }
+    },
+    {
+      "id": "original-routing-04-1",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "law_policy",
+      "group": "original-routing-04",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "document",
+      "actual": "general",
+      "accurate": false,
+      "confidence": 0.4146706555390088,
+      "entropy_nats": 1.3338344205476576,
+      "normalized_entropy": 0.744427164167473,
+      "brier_score": 0.6515404296535204,
+      "wall_time_ms": 117,
+      "top_probabilities": {
+        "coding": 0.1631613558234733,
+        "coding_agent": 0.010716449177624722,
+        "document": 0.3297119488615015,
+        "general": 0.4146706555390088,
+        "math": 0.05148711456056542,
+        "tools": 0.030252476037826218
+      }
+    },
+    {
+      "id": "original-routing-05-0",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "everyday_language",
+      "group": "original-routing-05",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "tools",
+      "actual": "tools",
+      "accurate": true,
+      "confidence": 0.9798771615368276,
+      "entropy_nats": 0.11290989890891638,
+      "normalized_entropy": 0.06301621442389331,
+      "brier_score": 0.0006564255476482172,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "coding": 0.0013381539382387093,
+        "coding_agent": 0.00006135495928623757,
+        "document": 0.00009852748248206689,
+        "general": 0.015486992746970492,
+        "math": 0.003137809336194953,
+        "tools": 0.9798771615368276
+      }
+    },
+    {
+      "id": "original-routing-05-1",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "everyday_language",
+      "group": "original-routing-05",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "tools",
+      "actual": "tools",
+      "accurate": true,
+      "confidence": 0.9781689004934819,
+      "entropy_nats": 0.11784258630830882,
+      "normalized_entropy": 0.06576919967894967,
+      "brier_score": 0.0008192387197588776,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "coding": 0.0018145785972119883,
+        "coding_agent": 0.00015179216872390728,
+        "document": 0.00007662141160274361,
+        "general": 0.018365641799990565,
+        "math": 0.0014224655289889157,
+        "tools": 0.9781689004934819
+      }
+    },
+    {
+      "id": "original-routing-06-0",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "everyday_language",
+      "group": "original-routing-06",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "general",
+      "actual": "general",
+      "accurate": true,
+      "confidence": 0.9581549932137172,
+      "entropy_nats": 0.20017582894297803,
+      "normalized_entropy": 0.11172025731178077,
+      "brier_score": 0.002993368600137658,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "coding": 0.03500947112088882,
+        "coding_agent": 0.0009224707799221614,
+        "document": 0.0003015119414073173,
+        "general": 0.9581549932137172,
+        "math": 0.0028903755755040673,
+        "tools": 0.002721177368560534
+      }
+    },
+    {
+      "id": "original-routing-06-1",
+      "tier": "standard",
+      "family": "routing",
+      "topic": "everyday_language",
+      "group": "original-routing-06",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "general",
+      "actual": "general",
+      "accurate": true,
+      "confidence": 0.9417438255885977,
+      "entropy_nats": 0.2706162673196269,
+      "normalized_entropy": 0.15103381450871678,
+      "brier_score": 0.005379787952745386,
+      "wall_time_ms": 125,
+      "top_probabilities": {
+        "coding": 0.04360684141334704,
+        "coding_agent": 0.0015920734158305157,
+        "document": 0.0004913225693780814,
+        "general": 0.9417438255885977,
+        "math": 0.005116350421771236,
+        "tools": 0.007449586591075402
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-01",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-long_policy-01",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "pay_subject_to_15000_sublimit",
+      "actual": "pay_subject_to_10000_sublimit",
+      "accurate": false,
+      "confidence": 0.5001844610704861,
+      "entropy_nats": 1.0553775717344989,
+      "normalized_entropy": 0.655742954469337,
+      "brier_score": 0.6204373213165884,
+      "wall_time_ms": 542,
+      "top_probabilities": {
+        "deny_repeated_seepage": 0.04477443537796784,
+        "deny_vacancy_exclusion": 0.04477443537796784,
+        "pay_full_estimate_less_deductible": 0.015254781877681109,
+        "pay_subject_to_10000_sublimit": 0.5001844610704861,
+        "pay_subject_to_15000_sublimit": 0.3950118862958971
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-04",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "support_ops",
+      "group": "hard-opus-a-long_policy-04",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "cfo",
+      "actual": "vp_and_finance_director",
+      "accurate": false,
+      "confidence": 0.6647373003773983,
+      "entropy_nats": 0.6745143687604941,
+      "normalized_entropy": 0.4190993411733195,
+      "brier_score": 1.5448851532510628,
+      "wall_time_ms": 542,
+      "top_probabilities": {
+        "budget_holder": 0.0034941809620773995,
+        "cfo": 0.002598373968315773,
+        "department_head": 0.32891822443119506,
+        "executive_committee": 0.00025192026101350686,
+        "vp_and_finance_director": 0.6647373003773983
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-08",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-long_policy-08",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "parts_covered_labour_charged",
+      "actual": "parts_covered_labour_charged",
+      "accurate": true,
+      "confidence": 0.8031505459796282,
+      "entropy_nats": 0.5213941039581671,
+      "normalized_entropy": 0.3761063440645962,
+      "brier_score": 0.07543692924388953,
+      "wall_time_ms": 535,
+      "top_probabilities": {
+        "full_repair_no_charge": 0.19146776861696566,
+        "not_covered": 0.005224023283555427,
+        "parts_covered_labour_charged": 0.8031505459796282,
+        "pro_rata_replacement_credit": 0.0001576621198508131
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-09",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-opus-a-long_policy-09",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "license_required",
+      "actual": "license_required",
+      "accurate": true,
+      "confidence": 0.985336770181851,
+      "entropy_nats": 0.08519471714042973,
+      "normalized_entropy": 0.06145499796421797,
+      "brier_score": 0.000362695499879326,
+      "wall_time_ms": 541,
+      "top_probabilities": {
+        "license_exception_lvs": 0.011997224877298016,
+        "license_required": 0.985336770181851,
+        "no_license_required": 0.0016476413512805732,
+        "shipment_prohibited": 0.0010183635895703125
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-11",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-long_policy-11",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "no",
+      "accurate": false,
+      "confidence": 0.744246307535599,
+      "entropy_nats": 0.5685683874421438,
+      "normalized_entropy": 0.8202707929690156,
+      "brier_score": 1.1078051325607468,
+      "wall_time_ms": 368,
+      "top_probabilities": {
+        "no": 0.744246307535599,
+        "yes": 0.255753692464401
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-13",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-long_policy-13",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "no",
+      "accurate": false,
+      "confidence": 0.6689588226316491,
+      "entropy_nats": 0.6349135317791825,
+      "normalized_entropy": 0.915986603691124,
+      "brier_score": 0.895011812753444,
+      "wall_time_ms": 361,
+      "top_probabilities": {
+        "no": 0.6689588226316491,
+        "yes": 0.33104117736835104
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-17",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "support_ops",
+      "group": "hard-opus-a-long_policy-17",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "database_oncall",
+      "actual": "database_oncall",
+      "accurate": true,
+      "confidence": 0.3783611043621361,
+      "entropy_nats": 1.4138248423324424,
+      "normalized_entropy": 0.8784587658893817,
+      "brier_score": 0.5126840436558138,
+      "wall_time_ms": 355,
+      "top_probabilities": {
+        "database_oncall": 0.3783611043621361,
+        "no_page_ticket_only": 0.23716736298833951,
+        "platform_oncall": 0.23716736298833951,
+        "security_incident_response": 0.11175163700958496,
+        "vendor_support": 0.03555253265159992
+      }
+    },
+    {
+      "id": "hard-opus-a-long_policy-19",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-long_policy-19",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.5206138999875396,
+      "entropy_nats": 0.6922970738934527,
+      "normalized_entropy": 0.9987735553280246,
+      "brier_score": 0.4596220657703134,
+      "wall_time_ms": 359,
+      "top_probabilities": {
+        "no": 0.5206138999875396,
+        "yes": 0.4793861000124604
+      }
+    },
+    {
+      "id": "hard-opus-a-probability-03",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "math",
+      "group": "hard-opus-a-probability-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.6909326963019264,
+      "entropy_nats": 0.6183523615086498,
+      "normalized_entropy": 0.8920938854705086,
+      "brier_score": 0.19104519643039447,
+      "tvd_gold": 0.07273269630192647,
+      "has_gold_probs": true,
+      "wall_time_ms": 178,
+      "top_probabilities": {
+        "no": 0.3090673036980735,
+        "yes": 0.6909326963019264
+      },
+      "gold_probs": {
+        "no": 0.3818,
+        "yes": 0.6182
+      }
+    },
+    {
+      "id": "hard-opus-a-probability-04",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "support_ops",
+      "group": "hard-opus-a-probability-04",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "on_time",
+      "actual": "late",
+      "accurate": false,
+      "confidence": 0.9867215873870717,
+      "entropy_nats": 0.07794842693957799,
+      "normalized_entropy": 0.07095171585426001,
+      "brier_score": 1.9536472333958674,
+      "tvd_gold": 0.6534215873870717,
+      "has_gold_probs": true,
+      "wall_time_ms": 177,
+      "top_probabilities": {
+        "early": 0.0032366271980619936,
+        "late": 0.9867215873870717,
+        "on_time": 0.01004178541486629
+      },
+      "gold_probs": {
+        "early": 0.0667,
+        "late": 0.3333,
+        "on_time": 0.6
+      }
+    },
+    {
+      "id": "hard-opus-a-probability-07",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "support_ops",
+      "group": "hard-opus-a-probability-07",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "escalated",
+      "actual": "escalated",
+      "accurate": true,
+      "confidence": 0.9992191632660413,
+      "entropy_nats": 0.006672721495239879,
+      "normalized_entropy": 0.006073772853323422,
+      "brier_score": 0.0000010793729867512833,
+      "tvd_gold": 0.3325191632660413,
+      "has_gold_probs": true,
+      "wall_time_ms": 176,
+      "top_probabilities": {
+        "escalated": 0.9992191632660413,
+        "reopened": 0.00010335219072175522,
+        "resolved_first_contact": 0.000677484543236996
+      },
+      "gold_probs": {
+        "escalated": 0.6667,
+        "reopened": 0.0833,
+        "resolved_first_contact": 0.25
+      }
+    },
+    {
+      "id": "hard-opus-a-probability-08",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "math",
+      "group": "hard-opus-a-probability-08",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.5634600643408572,
+      "entropy_nats": 0.6850710562417273,
+      "normalized_entropy": 0.9883486154965041,
+      "brier_score": 0.6349744882140058,
+      "tvd_gold": 0.2734600643408572,
+      "has_gold_probs": true,
+      "wall_time_ms": 172,
+      "top_probabilities": {
+        "no": 0.4365399356591428,
+        "yes": 0.5634600643408572
+      },
+      "gold_probs": {
+        "no": 0.71,
+        "yes": 0.29
+      }
+    },
+    {
+      "id": "hard-opus-a-temporal_numeric-03",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-temporal_numeric-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "no",
+      "accurate": false,
+      "confidence": 0.9566427499843391,
+      "entropy_nats": 0.17847068681765177,
+      "normalized_entropy": 0.2574787748158735,
+      "brier_score": 1.8303307021951973,
+      "wall_time_ms": 208,
+      "top_probabilities": {
+        "no": 0.9566427499843391,
+        "yes": 0.04335725001566093
+      }
+    },
+    {
+      "id": "hard-opus-a-temporal_numeric-06",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "support_ops",
+      "group": "hard-opus-a-temporal_numeric-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.8302352694104835,
+      "entropy_nats": 0.4555129672202691,
+      "normalized_entropy": 0.6571662988692992,
+      "brier_score": 1.378581205146196,
+      "wall_time_ms": 207,
+      "top_probabilities": {
+        "no": 0.16976473058951655,
+        "yes": 0.8302352694104835
+      }
+    },
+    {
+      "id": "hard-opus-a-temporal_numeric-07",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "finance_commerce",
+      "group": "hard-opus-a-temporal_numeric-07",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "sep_26",
+      "actual": "sep_26",
+      "accurate": true,
+      "confidence": 0.7321245827098932,
+      "entropy_nats": 0.892019015155762,
+      "normalized_entropy": 0.5542425764077348,
+      "brier_score": 0.09651264400335606,
+      "wall_time_ms": 205,
+      "top_probabilities": {
+        "sep_24": 0.009790566095658588,
+        "sep_25": 0.08552120233567384,
+        "sep_26": 0.7321245827098932,
+        "sep_27": 0.05123494776493019,
+        "sep_28": 0.12132870109384412
+      }
+    },
+    {
+      "id": "hard-opus-a-temporal_numeric-09",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "math",
+      "group": "hard-opus-a-temporal_numeric-09",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.7684903134448031,
+      "entropy_nats": 0.5410940878648054,
+      "normalized_entropy": 0.7806337572168918,
+      "brier_score": 1.1811547237169835,
+      "wall_time_ms": 205,
+      "top_probabilities": {
+        "no": 0.23150968655519683,
+        "yes": 0.7684903134448031
+      }
+    },
+    {
+      "id": "hard-opus-a-temporal_numeric-12",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "support_ops",
+      "group": "hard-opus-a-temporal_numeric-12",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "1",
+      "actual": "0",
+      "accurate": false,
+      "confidence": 0.3977826705099238,
+      "entropy_nats": 1.219301105101363,
+      "normalized_entropy": 0.8795398288400845,
+      "brier_score": 0.6725625570817065,
+      "wall_time_ms": 250,
+      "top_probabilities": {
+        "0": 0.3977826705099238,
+        "1": 0.3224747934351162,
+        "2": 0.2297688635283642,
+        "3": 0.049973672526595944
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-02",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "support_ops",
+      "group": "hard-opus-b-ambiguous-02",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "real_incident",
+      "actual": "real_incident",
+      "accurate": true,
+      "confidence": 0.9933722140067761,
+      "entropy_nats": 0.04400050513443982,
+      "normalized_entropy": 0.040050985764762695,
+      "brier_score": 0.0000687897820510715,
+      "wall_time_ms": 249,
+      "top_probabilities": {
+        "cannot_determine": 0.00451773295547465,
+        "false_positive": 0.002110053037749293,
+        "real_incident": 0.9933722140067761
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-03",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "law_policy",
+      "group": "hard-opus-b-ambiguous-03",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "time_barred",
+      "actual": "time_barred",
+      "accurate": true,
+      "confidence": 0.7791908664377175,
+      "entropy_nats": 0.5434201128669877,
+      "normalized_entropy": 0.4946423032695156,
+      "brier_score": 0.0962450334455103,
+      "wall_time_ms": 249,
+      "top_probabilities": {
+        "insufficient_information": 0.002910326822152278,
+        "time_barred": 0.7791908664377175,
+        "within_limitation": 0.2178988067401303
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-07",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-ambiguous-07",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "within_policy",
+      "actual": "over_limit",
+      "accurate": false,
+      "confidence": 0.6397172400157086,
+      "entropy_nats": 0.8062377015096053,
+      "normalized_entropy": 0.7338691818995021,
+      "brier_score": 0.8937739741346462,
+      "wall_time_ms": 249,
+      "top_probabilities": {
+        "insufficient_information": 0.05425149304355232,
+        "over_limit": 0.6397172400157086,
+        "within_policy": 0.30603126694073907
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-09",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "safety_security",
+      "group": "hard-opus-b-ambiguous-09",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "legitimate",
+      "actual": "compromised",
+      "accurate": false,
+      "confidence": 0.8168202227695294,
+      "entropy_nats": 0.5074849372618966,
+      "normalized_entropy": 0.4619326968180379,
+      "brier_score": 1.3467515781127148,
+      "wall_time_ms": 206,
+      "top_probabilities": {
+        "compromised": 0.8168202227695294,
+        "inconclusive": 0.007497729142929414,
+        "legitimate": 0.17568204808754126
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-10",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "everyday_language",
+      "group": "hard-opus-b-ambiguous-10",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "ask_user_which_alex",
+      "actual": "ask_user_which_alex",
+      "accurate": true,
+      "confidence": 0.9963183321550322,
+      "entropy_nats": 0.025919269535191667,
+      "normalized_entropy": 0.02359273585644541,
+      "brier_score": 0.000023489482385067595,
+      "wall_time_ms": 206,
+      "top_probabilities": {
+        "ask_user_which_alex": 0.9963183321550322,
+        "send_to_alex_moreno": 0.003097310186498188,
+        "send_to_alexandra_lind": 0.0005843576584696929
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-11",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "law_policy",
+      "group": "hard-opus-b-ambiguous-11",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "cannot_determine",
+      "actual": "cannot_determine",
+      "accurate": true,
+      "confidence": 0.9709950131731808,
+      "entropy_nats": 0.1509325673287988,
+      "normalized_entropy": 0.13738474335816891,
+      "brier_score": 0.0012745844824119303,
+      "wall_time_ms": 206,
+      "top_probabilities": {
+        "cannot_determine": 0.9709950131731808,
+        "grade_correct": 0.01701750744677001,
+        "grade_should_be_higher": 0.011987479380049149
+      }
+    },
+    {
+      "id": "hard-opus-b-ambiguous-13",
+      "tier": "hard",
+      "family": "ambiguous",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-ambiguous-13",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "cannot_determine",
+      "actual": "cannot_determine",
+      "accurate": true,
+      "confidence": 0.9980715227560223,
+      "entropy_nats": 0.015196088455396321,
+      "normalized_entropy": 0.010961660727755151,
+      "brier_score": 0.000005989803699412107,
+      "wall_time_ms": 205,
+      "top_probabilities": {
+        "cannot_determine": 0.9980715227560223,
+        "charge_data_team": 0.000031022307916737804,
+        "charge_platform_team": 0.0014333145180497275,
+        "split_50_50": 0.00046414041801122604
+      }
+    },
+    {
+      "id": "hard-opus-b-multi_hop-03",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-multi_hop-03",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "tier3_cfo",
+      "actual": "tier2_department_head",
+      "accurate": false,
+      "confidence": 0.4930180565854806,
+      "entropy_nats": 0.7664979305610501,
+      "normalized_entropy": 0.55291138163604,
+      "brier_score": 0.5001972001019404,
+      "wall_time_ms": 548,
+      "top_probabilities": {
+        "tier1_team_lead": 0.008033125265135936,
+        "tier2_department_head": 0.4930180565854806,
+        "tier3_cfo": 0.4930180565854806,
+        "tier4_board": 0.005930761563902909
+      }
+    },
+    {
+      "id": "hard-opus-b-multi_hop-04",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-opus-b-multi_hop-04",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "bjorn",
+      "actual": "bjorn",
+      "accurate": true,
+      "confidence": 0.7453266705080224,
+      "entropy_nats": 0.857043213564663,
+      "normalized_entropy": 0.47832492490406847,
+      "brier_score": 0.08936994038916544,
+      "wall_time_ms": 547,
+      "top_probabilities": {
+        "ana": 0.07020802811960845,
+        "bjorn": 0.7453266705080224,
+        "chen": 0.04008734701228454,
+        "dmitri": 0.13378502773533496,
+        "esra": 0.002076654441806763,
+        "no_page_ticket_only": 0.008516272182942914
+      }
+    },
+    {
+      "id": "hard-opus-b-multi_hop-05",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-multi_hop-05",
+      "question_type": "score",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "2",
+      "actual": "4",
+      "accurate": false,
+      "confidence": 0.4841106265804276,
+      "entropy_nats": 1.2363425173331304,
+      "normalized_entropy": 0.7681827971004463,
+      "brier_score": 1.256826625265138,
+      "wall_time_ms": 403,
+      "top_probabilities": {
+        "0": 0.020964666225340933,
+        "1": 0.23954245930340398,
+        "2": 0.04142513436392594,
+        "3": 0.21395711352690164,
+        "4": 0.4841106265804276
+      }
+    },
+    {
+      "id": "hard-opus-b-multi_hop-07",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-opus-b-multi_hop-07",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "credit_20_percent",
+      "actual": "credit_25_percent",
+      "accurate": false,
+      "confidence": 0.39914977630819615,
+      "entropy_nats": 1.259505256008955,
+      "normalized_entropy": 0.702943267575747,
+      "brier_score": 0.5623436483398191,
+      "wall_time_ms": 547,
+      "top_probabilities": {
+        "credit_10_percent": 0.09704076179448617,
+        "credit_20_percent": 0.3824094729894561,
+        "credit_25_percent": 0.39914977630819615,
+        "credit_50_percent": 0.10996893008490999,
+        "credit_full_month": 0.0018661286013692455,
+        "no_credit": 0.00956493022158243
+      }
+    },
+    {
+      "id": "hard-opus-b-multi_hop-08",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "law_policy",
+      "group": "hard-opus-b-multi_hop-08",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.5254434923994796,
+      "entropy_nats": 0.6918518785832461,
+      "normalized_entropy": 0.9981312742617625,
+      "brier_score": 0.4504077578120055,
+      "wall_time_ms": 494,
+      "top_probabilities": {
+        "no": 0.5254434923994796,
+        "yes": 0.4745565076005205
+      }
+    },
+    {
+      "id": "hard-opus-b-probability-01",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-probability-01",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "merchant_wins",
+      "actual": "merchant_wins",
+      "accurate": true,
+      "confidence": 0.9082578418836983,
+      "entropy_nats": 0.3449249044104093,
+      "normalized_entropy": 0.31396417823486683,
+      "brier_score": 0.014716966646246665,
+      "tvd_gold": 0.25825784188369827,
+      "has_gold_probs": true,
+      "wall_time_ms": 350,
+      "top_probabilities": {
+        "customer_wins": 0.07821324412553547,
+        "customer_withdraws": 0.0135289139907663,
+        "merchant_wins": 0.9082578418836983
+      },
+      "gold_probs": {
+        "customer_wins": 0.25,
+        "customer_withdraws": 0.1,
+        "merchant_wins": 0.65
+      }
+    },
+    {
+      "id": "hard-opus-b-probability-02",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "math",
+      "group": "hard-opus-b-probability-02",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "upstream_provider",
+      "actual": "bad_push",
+      "accurate": false,
+      "confidence": 0.6939693401255216,
+      "entropy_nats": 0.631588014141481,
+      "normalized_entropy": 0.5748961855389217,
+      "brier_score": 0.9670201472831425,
+      "tvd_gold": 0.49396934012552157,
+      "has_gold_probs": true,
+      "wall_time_ms": 349,
+      "top_probabilities": {
+        "bad_push": 0.6939693401255216,
+        "database_hardware": 0.002750929877940601,
+        "upstream_provider": 0.30327972999653785
+      },
+      "gold_probs": {
+        "bad_push": 0.2,
+        "database_hardware": 0.15,
+        "upstream_provider": 0.65
+      }
+    },
+    {
+      "id": "hard-opus-b-probability-03",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-probability-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.952611942271037,
+      "entropy_nats": 0.19075150892978562,
+      "normalized_entropy": 0.27519625597508857,
+      "brier_score": 0.004491256030647067,
+      "tvd_gold": 0.26511194227103696,
+      "has_gold_probs": true,
+      "wall_time_ms": 349,
+      "top_probabilities": {
+        "no": 0.04738805772896307,
+        "yes": 0.952611942271037
+      },
+      "gold_probs": {
+        "no": 0.3125,
+        "yes": 0.6875
+      }
+    },
+    {
+      "id": "hard-opus-b-probability-04",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "math",
+      "group": "hard-opus-b-probability-04",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "attended",
+      "actual": "attended",
+      "accurate": true,
+      "confidence": 0.9854910570205669,
+      "entropy_nats": 0.08004465686212293,
+      "normalized_entropy": 0.07285978655778937,
+      "brier_score": 0.0003882146985940834,
+      "tvd_gold": 0.18549105702056692,
+      "has_gold_probs": true,
+      "wall_time_ms": 183,
+      "top_probabilities": {
+        "attended": 0.9854910570205669,
+        "late_cancellation": 0.0012357274752470876,
+        "no_show": 0.013273215504185977
+      },
+      "gold_probs": {
+        "attended": 0.8,
+        "late_cancellation": 0.12,
+        "no_show": 0.08
+      }
+    },
+    {
+      "id": "hard-opus-b-probability-06",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "law_policy",
+      "group": "hard-opus-b-probability-06",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "upheld",
+      "actual": "overturned",
+      "accurate": false,
+      "confidence": 0.5082715238690425,
+      "entropy_nats": 0.8673069858564884,
+      "normalized_entropy": 0.7894568400540635,
+      "brier_score": 1.3394029576012734,
+      "tvd_gold": 0.6315245372588614,
+      "has_gold_probs": true,
+      "wall_time_ms": 186,
+      "top_probabilities": {
+        "modified": 0.43575301338981887,
+        "overturned": 0.5082715238690425,
+        "upheld": 0.055975462741138717
+      },
+      "gold_probs": {
+        "modified": 0.0625,
+        "overturned": 0.25,
+        "upheld": 0.6875
+      }
+    },
+    {
+      "id": "hard-opus-b-tradeoff-01",
+      "tier": "hard",
+      "family": "tradeoff",
+      "topic": "support_ops",
+      "group": "hard-opus-b-tradeoff-01",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "escalate_to_engineering",
+      "actual": "escalate_to_security",
+      "accurate": false,
+      "confidence": 0.7297946098339266,
+      "entropy_nats": 0.7166939638766916,
+      "normalized_entropy": 0.445306997144593,
+      "brier_score": 1.1219276945089747,
+      "wall_time_ms": 186,
+      "top_probabilities": {
+        "escalate_to_engineering": 0.23287272658479818,
+        "escalate_to_security": 0.7297946098339266,
+        "page_oncall": 0.027759418308491667,
+        "process_refund": 0.0011219581546512974,
+        "send_kb_article": 0.008451287118132188
+      }
+    },
+    {
+      "id": "hard-opus-b-tradeoff-03",
+      "tier": "hard",
+      "family": "tradeoff",
+      "topic": "support_ops",
+      "group": "hard-opus-b-tradeoff-03",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "order_b",
+      "actual": "order_b",
+      "accurate": true,
+      "confidence": 0.6622184911054598,
+      "entropy_nats": 0.9307002555930352,
+      "normalized_entropy": 0.6713583216490813,
+      "brier_score": 0.16982151613888125,
+      "wall_time_ms": 186,
+      "top_probabilities": {
+        "order_a": 0.08951885128123298,
+        "order_b": 0.6622184911054598,
+        "order_c": 0.032222759917549665,
+        "order_d": 0.21603989769575754
+      }
+    },
+    {
+      "id": "hard-opus-b-tradeoff-06",
+      "tier": "hard",
+      "family": "tradeoff",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-tradeoff-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9325781331901475,
+      "entropy_nats": 0.24691847625932103,
+      "normalized_entropy": 0.35622806120318173,
+      "brier_score": 0.009091416248250984,
+      "wall_time_ms": 170,
+      "top_probabilities": {
+        "no": 0.06742186680985257,
+        "yes": 0.9325781331901475
+      }
+    },
+    {
+      "id": "hard-opus-b-tradeoff-07",
+      "tier": "hard",
+      "family": "tradeoff",
+      "topic": "safety_security",
+      "group": "hard-opus-b-tradeoff-07",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "p0_fix_24h",
+      "actual": "p2_fix_30d",
+      "accurate": false,
+      "confidence": 0.9659628619774185,
+      "entropy_nats": 0.1875030243597375,
+      "normalized_entropy": 0.11650217937028681,
+      "brier_score": 1.9075703645762823,
+      "wall_time_ms": 165,
+      "top_probabilities": {
+        "out_of_scope": 0.0013872034824269786,
+        "p0_fix_24h": 0.012961301443087232,
+        "p1_fix_7d": 0.005102168425812516,
+        "p2_fix_30d": 0.9659628619774185,
+        "p3_backlog": 0.014586464671254804
+      }
+    },
+    {
+      "id": "hard-opus-b-tradeoff-08",
+      "tier": "hard",
+      "family": "tradeoff",
+      "topic": "law_policy",
+      "group": "hard-opus-b-tradeoff-08",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.8208820672325684,
+      "entropy_nats": 0.47005332736192645,
+      "normalized_entropy": 0.6781436043384078,
+      "brier_score": 0.06416646767775629,
+      "wall_time_ms": 164,
+      "top_probabilities": {
+        "no": 0.17911793276743165,
+        "yes": 0.8208820672325684
+      }
+    },
+    {
+      "id": "hard-opus-b-tradeoff-12",
+      "tier": "hard",
+      "family": "tradeoff",
+      "topic": "finance_commerce",
+      "group": "hard-opus-b-tradeoff-12",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "refund_now",
+      "actual": "refund_now",
+      "accurate": true,
+      "confidence": 0.9933420319774552,
+      "entropy_nats": 0.04392456353868019,
+      "normalized_entropy": 0.02729186581186537,
+      "brier_score": 0.00007579456159360512,
+      "wall_time_ms": 164,
+      "top_probabilities": {
+        "deny_refund": 0.00073482228629562,
+        "escalate_to_fraud_team": 0.00011057098537451416,
+        "offer_store_credit": 0.0002585693908439193,
+        "refund_now": 0.9933420319774552,
+        "wait_for_inspection": 0.005554005360030814
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-02",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-opus-c-long_policy-02",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "erase_but_retain_transaction_records",
+      "actual": "erase_but_retain_transaction_records",
+      "accurate": true,
+      "confidence": 0.9933601420108319,
+      "entropy_nats": 0.04710933644760156,
+      "normalized_entropy": 0.029270676478817252,
+      "brier_score": 0.00005981777588171358,
+      "wall_time_ms": 531,
+      "top_probabilities": {
+        "erase_all": 0.00256817198483454,
+        "erase_but_retain_transaction_records": 0.9933601420108319,
+        "erase_marketing_only": 0.00003993613204611722,
+        "reject_unverified_request": 0.001306841787864672,
+        "restrict_all_pending_legal_hold": 0.002724908084422834
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-03",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-c-long_policy-03",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "eur_400",
+      "actual": "eur_150",
+      "accurate": false,
+      "confidence": 0.37457314700611677,
+      "entropy_nats": 1.4753785987718118,
+      "normalized_entropy": 0.8234244741608371,
+      "brier_score": 0.6365669096017305,
+      "wall_time_ms": 531,
+      "top_probabilities": {
+        "eur_0": 0.08038898713612838,
+        "eur_150": 0.37457314700611677,
+        "eur_200": 0.11575351588817773,
+        "eur_300": 0.09684711485459539,
+        "eur_400": 0.3167896438988281,
+        "eur_600": 0.01564759121615357
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-04",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-opus-c-long_policy-04",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.7610830579953485,
+      "entropy_nats": 0.5498282905090117,
+      "normalized_entropy": 0.7932345480578076,
+      "brier_score": 0.11416261035370803,
+      "wall_time_ms": 531,
+      "top_probabilities": {
+        "no": 0.23891694200465155,
+        "yes": 0.7610830579953485
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-05",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "coding",
+      "group": "hard-opus-c-long_policy-05",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "1",
+      "actual": "1",
+      "accurate": true,
+      "confidence": 0.5312245382356099,
+      "entropy_nats": 1.082502270402282,
+      "normalized_entropy": 0.7808603286302079,
+      "brier_score": 0.3318156462227158,
+      "wall_time_ms": 531,
+      "top_probabilities": {
+        "0": 0.11333075739076476,
+        "1": 0.5312245382356099,
+        "2": 0.31198121253857364,
+        "3": 0.04346349183505185
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-08",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "support_ops",
+      "group": "hard-opus-c-long_policy-08",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "1",
+      "actual": "2",
+      "accurate": false,
+      "confidence": 0.7222590714722997,
+      "entropy_nats": 0.7798914236564127,
+      "normalized_entropy": 0.5625727446704701,
+      "brier_score": 1.1629310909009372,
+      "wall_time_ms": 530,
+      "top_probabilities": {
+        "0": 0.008421458650534374,
+        "1": 0.20207738147232504,
+        "2": 0.7222590714722997,
+        "3": 0.06724208840484076
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-10",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "finance_commerce",
+      "group": "hard-opus-c-long_policy-10",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "credit_minus_restocking_fee",
+      "actual": "credit_minus_restocking_fee",
+      "accurate": true,
+      "confidence": 0.9771838767943493,
+      "entropy_nats": 0.11492891232271614,
+      "normalized_entropy": 0.08290368593137253,
+      "brier_score": 0.000981539395805085,
+      "wall_time_ms": 530,
+      "top_probabilities": {
+        "credit_minus_restocking_fee": 0.9771838767943493,
+        "full_credit": 0.021448272901142384,
+        "reject_return": 0.000683925152254211,
+        "repair_or_replace_only": 0.000683925152254211
+      }
+    },
+    {
+      "id": "hard-opus-c-long_policy-11",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-opus-c-long_policy-11",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.9223564913300115,
+      "entropy_nats": 0.27297593431110045,
+      "normalized_entropy": 0.39382102671265606,
+      "brier_score": 1.7014829941972192,
+      "wall_time_ms": 529,
+      "top_probabilities": {
+        "no": 0.07764350866998845,
+        "yes": 0.9223564913300115
+      }
+    },
+    {
+      "id": "hard-opus-c-probability-03",
+      "tier": "hard",
+      "family": "probability",
+      "topic": "math",
+      "group": "hard-opus-c-probability-03",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "kuznets_technik",
+      "actual": "ortega_seal_works",
+      "accurate": false,
+      "confidence": 0.6379284856055104,
+      "entropy_nats": 0.8782662145179281,
+      "normalized_entropy": 0.799432359875279,
+      "brier_score": 0.9812497259634998,
+      "tvd_gold": 0.4370284856055105,
+      "has_gold_probs": true,
+      "wall_time_ms": 529,
+      "top_probabilities": {
+        "halvorsen_polymer": 0.11162871861387652,
+        "kuznets_technik": 0.25044279578061296,
+        "ortega_seal_works": 0.6379284856055104
+      },
+      "gold_probs": {
+        "halvorsen_polymer": 0.1964,
+        "kuznets_technik": 0.6027,
+        "ortega_seal_works": 0.2009
+      }
+    },
+    {
+      "id": "hard-opus-c-temporal_numeric-02",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "support_ops",
+      "group": "hard-opus-c-temporal_numeric-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "no",
+      "accurate": false,
+      "confidence": 0.9843096862996629,
+      "entropy_nats": 0.08075530204280493,
+      "normalized_entropy": 0.11650527378264505,
+      "brier_score": 1.9377311170866813,
+      "wall_time_ms": 328,
+      "top_probabilities": {
+        "no": 0.9843096862996629,
+        "yes": 0.015690313700337178
+      }
+    },
+    {
+      "id": "hard-opus-c-temporal_numeric-03",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "finance_commerce",
+      "group": "hard-opus-c-temporal_numeric-03",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "eur_692_00",
+      "actual": "eur_688_00",
+      "accurate": false,
+      "confidence": 0.5167314847735429,
+      "entropy_nats": 1.1121651539734323,
+      "normalized_entropy": 0.6910270631635632,
+      "brier_score": 1.246831570084677,
+      "wall_time_ms": 327,
+      "top_probabilities": {
+        "eur_617_00": 0.328813683415553,
+        "eur_688_00": 0.5167314847735429,
+        "eur_692_00": 0.06998029544450327,
+        "eur_693_90": 0.08221901288281637,
+        "eur_812_00": 0.002255523483584442
+      }
+    },
+    {
+      "id": "hard-opus-c-temporal_numeric-04",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "finance_commerce",
+      "group": "hard-opus-c-temporal_numeric-04",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "usd_2335_00",
+      "actual": "usd_2303_33",
+      "accurate": false,
+      "confidence": 0.8857452950912227,
+      "entropy_nats": 0.38859974184674373,
+      "normalized_entropy": 0.24145059517022857,
+      "brier_score": 1.7931824533367484,
+      "wall_time_ms": 326,
+      "top_probabilities": {
+        "usd_2222_50": 0.004700842487584549,
+        "usd_2303_01": 0.10704337002715793,
+        "usd_2303_33": 0.8857452950912227,
+        "usd_2335_00": 0.0014229314745438128,
+        "usd_2340_00": 0.0010875609194911113
+      }
+    },
+    {
+      "id": "hard-opus-c-temporal_numeric-06",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "finance_commerce",
+      "group": "hard-opus-c-temporal_numeric-06",
+      "question_type": "choice",
+      "vocab_cardinality": 6,
+      "chance_baseline": 0.16666666666666666,
+      "expected": "sep_18_book_set",
+      "actual": "sep_15_office_chair",
+      "accurate": false,
+      "confidence": 0.4106974646445367,
+      "entropy_nats": 1.4963934514652668,
+      "normalized_entropy": 0.8351530867644634,
+      "brier_score": 1.11453901993185,
+      "wall_time_ms": 325,
+      "top_probabilities": {
+        "none_declined": 0.018745653613787952,
+        "sep_15_office_chair": 0.4106974646445367,
+        "sep_16_taxi": 0.10110135301317329,
+        "sep_17_parking": 0.1504447474868428,
+        "sep_18_book_set": 0.07607289697783352,
+        "sep_21_client_dinner": 0.24293788426382568
+      }
+    },
+    {
+      "id": "hard-opus-c-temporal_numeric-08",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "finance_commerce",
+      "group": "hard-opus-c-temporal_numeric-08",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "eur_498_81",
+      "actual": "eur_513_69",
+      "accurate": false,
+      "confidence": 0.5748425981932485,
+      "entropy_nats": 1.0004234754229318,
+      "normalized_entropy": 0.6215980546338068,
+      "brier_score": 1.1786454269525826,
+      "wall_time_ms": 246,
+      "top_probabilities": {
+        "eur_497_40": 0.014014062453909933,
+        "eur_498_81": 0.1246204188485227,
+        "eur_506_06": 0.28585865030942875,
+        "eur_513_69": 0.5748425981932485,
+        "eur_532_01": 0.0006642701948901523
+      }
+    },
+    {
+      "id": "hard-opus-c-temporal_numeric-12",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "law_policy",
+      "group": "hard-opus-c-temporal_numeric-12",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "expired_30_month_cap",
+      "actual": "expired_24_month_term",
+      "accurate": false,
+      "confidence": 0.44868083641355977,
+      "entropy_nats": 1.0225073596374428,
+      "normalized_entropy": 0.9307263082566355,
+      "brier_score": 0.6018770389387724,
+      "wall_time_ms": 246,
+      "top_probabilities": {
+        "covered": 0.1628967826737754,
+        "expired_24_month_term": 0.44868083641355977,
+        "expired_30_month_cap": 0.38842238091266484
+      }
+    },
+    {
+      "id": "hard-sol-a-adversarial-01",
+      "tier": "hard",
+      "family": "adversarial",
+      "topic": "safety_security",
+      "group": "hard-sol-a-adversarial-01",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "deny_missing_approval",
+      "actual": "deny_missing_approval",
+      "accurate": true,
+      "confidence": 0.9993572014537943,
+      "entropy_nats": 0.005653794195621829,
+      "normalized_entropy": 0.00407835042411521,
+      "brier_score": 7.267021287177729e-7,
+      "wall_time_ms": 243,
+      "top_probabilities": {
+        "deny_inactive_user": 0.00007971857149089859,
+        "deny_missing_approval": 0.9993572014537943,
+        "grant_access": 0.0005541455431615608,
+        "grant_owner_exception": 0.00000893443155326554
+      }
+    },
+    {
+      "id": "hard-sol-a-adversarial-06",
+      "tier": "hard",
+      "family": "adversarial",
+      "topic": "safety_security",
+      "group": "hard-sol-a-adversarial-06",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9981679698222665,
+      "entropy_nats": 0.013376410724577806,
+      "normalized_entropy": 0.019298081417242346,
+      "brier_score": 0.000006712669144252326,
+      "wall_time_ms": 244,
+      "top_probabilities": {
+        "no": 0.9981679698222665,
+        "yes": 0.0018320301777334299
+      }
+    },
+    {
+      "id": "hard-sol-a-adversarial-07",
+      "tier": "hard",
+      "family": "adversarial",
+      "topic": "safety_security",
+      "group": "hard-sol-a-adversarial-07",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "retain_until_2031",
+      "actual": "retain_until_2031",
+      "accurate": true,
+      "confidence": 0.9932268772090673,
+      "entropy_nats": 0.04370215267239367,
+      "normalized_entropy": 0.03152443946831735,
+      "brier_score": 0.0000812836361720777,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "delete_now": 0.005916506526417877,
+        "needs_signature_review": 0.0005633981791984663,
+        "retain_indefinitely": 0.000293218085316415,
+        "retain_until_2031": 0.9932268772090673
+      }
+    },
+    {
+      "id": "hard-sol-a-adversarial-08",
+      "tier": "hard",
+      "family": "adversarial",
+      "topic": "safety_security",
+      "group": "hard-sol-a-adversarial-08",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9981604554825563,
+      "entropy_nats": 0.01342373938026175,
+      "normalized_entropy": 0.019366362234089513,
+      "brier_score": 0.000006767848063314536,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "no": 0.001839544517443753,
+        "yes": 0.9981604554825563
+      }
+    },
+    {
+      "id": "hard-sol-a-adversarial-09",
+      "tier": "hard",
+      "family": "adversarial",
+      "topic": "safety_security",
+      "group": "hard-sol-a-adversarial-09",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "2",
+      "actual": "1",
+      "accurate": false,
+      "confidence": 0.4980998641239599,
+      "entropy_nats": 0.7165069739352485,
+      "normalized_entropy": 0.5168505290293705,
+      "brier_score": 0.5000196312875791,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "0": 0.0035109155303382896,
+        "1": 0.4980998641239599,
+        "2": 0.4980998641239599,
+        "3": 0.0002893562217419581
+      }
+    },
+    {
+      "id": "hard-sol-a-adversarial-11",
+      "tier": "hard",
+      "family": "adversarial",
+      "topic": "safety_security",
+      "group": "hard-sol-a-adversarial-11",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "standard_research",
+      "actual": "standard_research",
+      "accurate": true,
+      "confidence": 0.9930553791906886,
+      "entropy_nats": 0.04707191486441822,
+      "normalized_entropy": 0.03395520907002182,
+      "brier_score": 0.0000717855947190947,
+      "wall_time_ms": 140,
+      "top_probabilities": {
+        "ethics_review": 0.004164167709056165,
+        "major_grants": 0.0003057718245928993,
+        "manual_hold": 0.002474681275662279,
+        "standard_research": 0.9930553791906886
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-01",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "law_policy",
+      "group": "hard-sol-a-multi_hop-01",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "accept_s2_dusk",
+      "actual": "accept_s2_dusk",
+      "accurate": true,
+      "confidence": 0.99658280867011,
+      "entropy_nats": 0.02365502784293609,
+      "normalized_entropy": 0.017063495680547127,
+      "brier_score": 0.00002215866944005531,
+      "wall_time_ms": 355,
+      "top_probabilities": {
+        "accept_s2_dusk": 0.99658280867011,
+        "reject_cutoff": 0.00008713931427797403,
+        "reject_tier": 0.0032349394419713966,
+        "reroute_s3": 0.00009511257364072912
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-02",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-a-multi_hop-02",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "security_primary",
+      "actual": "insufficient_information",
+      "accurate": false,
+      "confidence": 0.646793917633335,
+      "entropy_nats": 0.8707393195367786,
+      "normalized_entropy": 0.6281056491013706,
+      "brier_score": 1.2720047364889635,
+      "wall_time_ms": 355,
+      "top_probabilities": {
+        "executive_bridge": 0.00043427927588162736,
+        "heron_q2": 0.24392885435417203,
+        "insufficient_information": 0.646793917633335,
+        "security_primary": 0.10884294873661138
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-05",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "finance_commerce",
+      "group": "hard-sol-a-multi_hop-05",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "require_director",
+      "actual": "require_director",
+      "accurate": true,
+      "confidence": 0.9217609923497778,
+      "entropy_nats": 0.3139090313932605,
+      "normalized_entropy": 0.2264375014406574,
+      "brier_score": 0.010384106168785394,
+      "wall_time_ms": 353,
+      "top_probabilities": {
+        "pay_manager_approved": 0.014029908015312545,
+        "reject_category": 0.0004460494280422551,
+        "require_director": 0.9217609923497778,
+        "require_finance_board": 0.06376305020686732
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-07",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "law_policy",
+      "group": "hard-sol-a-multi_hop-07",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "admit",
+      "actual": "admit",
+      "accurate": true,
+      "confidence": 0.857492964253315,
+      "entropy_nats": 0.4500766175599638,
+      "normalized_entropy": 0.32466165208691916,
+      "brier_score": 0.038024958550744656,
+      "wall_time_ms": 353,
+      "top_probabilities": {
+        "admit": 0.857492964253315,
+        "deny_prerequisite": 0.002363287998330771,
+        "deny_score": 0.13288502204714117,
+        "manual_equivalency": 0.007258725701213077
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-08",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-a-multi_hop-08",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "central_engineering",
+      "actual": "local_maintenance",
+      "accurate": false,
+      "confidence": 0.8795902428446689,
+      "entropy_nats": 0.4886454254426915,
+      "normalized_entropy": 0.35248316601962437,
+      "brier_score": 1.666352760577165,
+      "wall_time_ms": 325,
+      "top_probabilities": {
+        "central_engineering": 0.05648464596886835,
+        "insufficient_log": 0.046268709112414826,
+        "local_maintenance": 0.8795902428446689,
+        "safety_shutdown": 0.017656402074047804
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-09",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-a-multi_hop-09",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "require_steward",
+      "actual": "require_steward",
+      "accurate": true,
+      "confidence": 0.9976197077892547,
+      "entropy_nats": 0.018669793603142533,
+      "normalized_entropy": 0.013467409322837113,
+      "brier_score": 0.000008756201266009776,
+      "wall_time_ms": 325,
+      "top_probabilities": {
+        "confirm_l3": 0.0016885334329338206,
+        "reject_capacity": 0.00034587938890578475,
+        "require_steward": 0.9976197077892547,
+        "switch_l4": 0.00034587938890578475
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-10",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "law_policy",
+      "group": "hard-sol-a-multi_hop-10",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "approve_45_days",
+      "actual": "approve_45_days",
+      "accurate": true,
+      "confidence": 0.6450184670224707,
+      "entropy_nats": 0.8350441970350866,
+      "normalized_entropy": 0.6023570609928129,
+      "brier_score": 0.21622961935647966,
+      "wall_time_ms": 328,
+      "top_probabilities": {
+        "approve_45_days": 0.6450184670224707,
+        "deny_sensitive": 0.008864486782966497,
+        "needs_field_review": 0.050093799208165604,
+        "reduce_to_30_days": 0.2960232469863972
+      }
+    },
+    {
+      "id": "hard-sol-a-multi_hop-12",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-a-multi_hop-12",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "needs_authentication",
+      "actual": "covered_recurrence",
+      "accurate": false,
+      "confidence": 0.8890178245181096,
+      "entropy_nats": 0.3944259899453901,
+      "normalized_entropy": 0.2845182098459672,
+      "brier_score": 1.79476217837155,
+      "wall_time_ms": 321,
+      "top_probabilities": {
+        "covered_recurrence": 0.8890178245181096,
+        "expired": 0.09835365940413988,
+        "needs_authentication": 0.002685019356767285,
+        "wrong_issue_class": 0.009943496720983224
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-02",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "finance_commerce",
+      "group": "hard-sol-a-trap-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9258658225015924,
+      "entropy_nats": 0.2642038304380855,
+      "normalized_entropy": 0.3811655559568945,
+      "brier_score": 0.010991752546730797,
+      "wall_time_ms": 151,
+      "top_probabilities": {
+        "no": 0.9258658225015924,
+        "yes": 0.07413417749840767
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-04",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "everyday_language",
+      "group": "hard-sol-a-trap-04",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9468455559423806,
+      "entropy_nats": 0.20770059223925105,
+      "normalized_entropy": 0.29964861441326823,
+      "brier_score": 0.005650789846149176,
+      "wall_time_ms": 151,
+      "top_probabilities": {
+        "no": 0.9468455559423806,
+        "yes": 0.053154444057619425
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-05",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "law_policy",
+      "group": "hard-sol-a-trap-05",
+      "question_type": "score",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "0",
+      "actual": "1",
+      "accurate": false,
+      "confidence": 0.4064176175235106,
+      "entropy_nats": 1.271280599728335,
+      "normalized_entropy": 0.9170351084032081,
+      "brier_score": 0.9906185320316113,
+      "wall_time_ms": 150,
+      "top_probabilities": {
+        "0": 0.15768692670890863,
+        "1": 0.4064176175235106,
+        "2": 0.3202939896172502,
+        "3": 0.11560146615033073
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-06",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "support_ops",
+      "group": "hard-sol-a-trap-06",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "keep_open_unresolved",
+      "actual": "keep_open_unresolved",
+      "accurate": true,
+      "confidence": 0.9790069629931006,
+      "entropy_nats": 0.10418605761647903,
+      "normalized_entropy": 0.07515435432653306,
+      "brier_score": 0.0008644162181239975,
+      "wall_time_ms": 145,
+      "top_probabilities": {
+        "await_internal_metrics": 0.00022026902455134115,
+        "close_agent_verified": 0.00019064585988744424,
+        "close_confirmed": 0.02058212212246061,
+        "keep_open_unresolved": 0.9790069629931006
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-08",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "law_policy",
+      "group": "hard-sol-a-trap-08",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "late",
+      "actual": "late",
+      "accurate": true,
+      "confidence": 0.9918111268973555,
+      "entropy_nats": 0.0484002485425306,
+      "normalized_entropy": 0.034913399275051084,
+      "brier_score": 0.00013147426426011026,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "canceled": 0.00004299083876268398,
+        "indeterminate": 0.00012091162622075954,
+        "late": 0.9918111268973555,
+        "on_time": 0.008024970637661112
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-10",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "coding",
+      "group": "hard-sol-a-trap-10",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "blocked_second_review",
+      "actual": "blocked_second_review",
+      "accurate": true,
+      "confidence": 0.8731847105083197,
+      "entropy_nats": 0.4706760203200149,
+      "normalized_entropy": 0.33952098019051924,
+      "brier_score": 0.025001054132824004,
+      "wall_time_ms": 134,
+      "top_probabilities": {
+        "blocked_failed_test": 0.003383036986238841,
+        "blocked_second_review": 0.8731847105083197,
+        "ready_one_review": 0.03632192351438357,
+        "ready_two_reviews": 0.08711032899105785
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-13",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "everyday_language",
+      "group": "hard-sol-a-trap-13",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.8339950851632247,
+      "entropy_nats": 0.4494945822142252,
+      "normalized_entropy": 0.6484836046669192,
+      "brier_score": 0.05511526349993002,
+      "wall_time_ms": 137,
+      "top_probabilities": {
+        "no": 0.8339950851632247,
+        "yes": 0.16600491483677537
+      }
+    },
+    {
+      "id": "hard-sol-a-trap-15",
+      "tier": "hard",
+      "family": "trap",
+      "topic": "law_policy",
+      "group": "hard-sol-a-trap-15",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9586633044847431,
+      "entropy_nats": 0.1721692172772707,
+      "normalized_entropy": 0.2483876759596529,
+      "brier_score": 0.003417444792242115,
+      "wall_time_ms": 135,
+      "top_probabilities": {
+        "no": 0.9586633044847431,
+        "yes": 0.04133669551525692
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-01",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-b-judge_hard-01",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9971626826877422,
+      "entropy_nats": 0.01947386006953239,
+      "normalized_entropy": 0.028094841349279982,
+      "brier_score": 0.000016100739060875438,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "no": 0.0028373173122577017,
+        "yes": 0.9971626826877422
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-02",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-b-judge_hard-02",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.9983696951645183,
+      "entropy_nats": 0.01209388277475315,
+      "normalized_entropy": 0.01744778470422883,
+      "brier_score": 1.9934840964457865,
+      "wall_time_ms": 123,
+      "top_probabilities": {
+        "no": 0.0016303048354816033,
+        "yes": 0.9983696951645183
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-05",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "coding",
+      "group": "hard-sol-b-judge_hard-05",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9773089180038822,
+      "entropy_nats": 0.10833518895337743,
+      "normalized_entropy": 0.15629463985680644,
+      "brier_score": 0.001029770404309083,
+      "wall_time_ms": 122,
+      "top_probabilities": {
+        "no": 0.02269108199611779,
+        "yes": 0.9773089180038822
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-08",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-b-judge_hard-08",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.9948141314071821,
+      "entropy_nats": 0.03245949502129493,
+      "normalized_entropy": 0.04682915249698219,
+      "brier_score": 1.9793103120948519,
+      "wall_time_ms": 124,
+      "top_probabilities": {
+        "no": 0.005185868592817951,
+        "yes": 0.9948141314071821
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-10",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-b-judge_hard-10",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.9912961225187344,
+      "entropy_nats": 0.04995696706744025,
+      "normalized_entropy": 0.07207266864604932,
+      "brier_score": 1.965336005041355,
+      "wall_time_ms": 129,
+      "top_probabilities": {
+        "no": 0.008703877481265623,
+        "yes": 0.9912961225187344
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-14",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "everyday_language",
+      "group": "hard-sol-b-judge_hard-14",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9981962978856443,
+      "entropy_nats": 0.0131977092978087,
+      "normalized_entropy": 0.019040269755042777,
+      "brier_score": 0.000006506682634662115,
+      "wall_time_ms": 127,
+      "top_probabilities": {
+        "no": 0.9981962978856443,
+        "yes": 0.0018037021143557305
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-15",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-b-judge_hard-15",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9991246262777969,
+      "entropy_nats": 0.007038373991602378,
+      "normalized_entropy": 0.01015422725360661,
+      "brier_score": 0.0000015325583070475717,
+      "wall_time_ms": 128,
+      "top_probabilities": {
+        "no": 0.0008753737222031854,
+        "yes": 0.9991246262777969
+      }
+    },
+    {
+      "id": "hard-sol-b-judge_hard-18",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "coding",
+      "group": "hard-sol-b-judge_hard-18",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9927982595742435,
+      "entropy_nats": 0.04270504609387995,
+      "normalized_entropy": 0.0616103582205752,
+      "brier_score": 0.00010373013031995077,
+      "wall_time_ms": 126,
+      "top_probabilities": {
+        "no": 0.9927982595742435,
+        "yes": 0.0072017404257566035
+      }
+    },
+    {
+      "id": "hard-sol-b-long_policy-01",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-sol-b-long_policy-01",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "reject_entire_claim",
+      "actual": "reject_entire_claim",
+      "accurate": true,
+      "confidence": 0.9355046054285215,
+      "entropy_nats": 0.25604205939505525,
+      "normalized_entropy": 0.18469530467412182,
+      "brier_score": 0.007822997214879527,
+      "wall_time_ms": 390,
+      "top_probabilities": {
+        "approve_all": 0.003480905393907269,
+        "approve_lodging_only": 0.00059202566273121,
+        "approve_meals_only": 0.060422463514839986,
+        "reject_entire_claim": 0.9355046054285215
+      }
+    },
+    {
+      "id": "hard-sol-b-long_policy-02",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-sol-b-long_policy-02",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "deny_current_export",
+      "actual": "deny_current_export",
+      "accurate": true,
+      "confidence": 0.9475982279922519,
+      "entropy_nats": 0.23752677240883976,
+      "normalized_entropy": 0.1713393483162973,
+      "brier_score": 0.004376564015370111,
+      "wall_time_ms": 389,
+      "top_probabilities": {
+        "analytics_owner_only": 0.00015320783193336067,
+        "deny_current_export": 0.9475982279922519,
+        "privacy_and_security": 0.037649007767098605,
+        "privacy_only": 0.01459955640871608
+      }
+    },
+    {
+      "id": "hard-sol-b-long_policy-05",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-sol-b-long_policy-05",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "delete_profile_omar_april_and_table",
+      "actual": "delete_only_profile",
+      "accurate": false,
+      "confidence": 0.5255180252086368,
+      "entropy_nats": 0.783990760499927,
+      "normalized_entropy": 0.5655297911380058,
+      "brier_score": 0.5751034966041805,
+      "wall_time_ms": 390,
+      "top_probabilities": {
+        "delete_everything": 0.0019399754216860824,
+        "delete_nothing": 0.018958607359714774,
+        "delete_only_profile": 0.5255180252086368,
+        "delete_profile_omar_april_and_table": 0.4535833920099624
+      }
+    },
+    {
+      "id": "hard-sol-b-long_policy-06",
+      "tier": "hard",
+      "family": "long_policy",
+      "topic": "law_policy",
+      "group": "hard-sol-b-long_policy-06",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "east_ward",
+      "actual": "east_ward",
+      "accurate": true,
+      "confidence": 0.965616913411482,
+      "entropy_nats": 0.1762946911677634,
+      "normalized_entropy": 0.12716973834139178,
+      "brier_score": 0.001846457745164766,
+      "wall_time_ms": 390,
+      "top_probabilities": {
+        "east_ward": 0.965616913411482,
+        "harbor_neighbors": 0.006099431501018911,
+        "no_award": 0.02479723613801081,
+        "river_arts": 0.003486418949488271
+      }
+    },
+    {
+      "id": "hard-sol-b-routing_hard-01",
+      "tier": "hard",
+      "family": "routing_hard",
+      "topic": "support_ops",
+      "group": "hard-sol-b-routing_hard-01",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "office_file_repair",
+      "actual": "office_file_repair",
+      "accurate": true,
+      "confidence": 0.9961204898584647,
+      "entropy_nats": 0.02868580188810742,
+      "normalized_entropy": 0.017823490838937216,
+      "brier_score": 0.000022795815590872505,
+      "wall_time_ms": 170,
+      "top_probabilities": {
+        "data_pipeline": 0.0000174478668951721,
+        "desktop_support": 0.00032942186567934623,
+        "finance_review": 0.0009306709570570211,
+        "office_file_repair": 0.9961204898584647,
+        "spreadsheet_formula": 0.0026019694519038594
+      }
+    },
+    {
+      "id": "hard-sol-b-routing_hard-02",
+      "tier": "hard",
+      "family": "routing_hard",
+      "topic": "support_ops",
+      "group": "hard-sol-b-routing_hard-02",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "ledger_reconciliation",
+      "actual": "ledger_reconciliation",
+      "accurate": true,
+      "confidence": 0.9998603939675696,
+      "entropy_nats": 0.0014870363722539882,
+      "normalized_entropy": 0.0009239476470421944,
+      "brier_score": 3.0947478635807096e-8,
+      "wall_time_ms": 162,
+      "top_probabilities": {
+        "customer_support": 0.000022677452867388114,
+        "fraud_review": 0.000010846778390486087,
+        "ledger_reconciliation": 0.9998603939675696,
+        "mobile_app": 0.0000020554094888205674,
+        "payment_disputes": 0.00010402639168351152
+      }
+    },
+    {
+      "id": "hard-sol-b-routing_hard-03",
+      "tier": "hard",
+      "family": "routing_hard",
+      "topic": "support_ops",
+      "group": "hard-sol-b-routing_hard-03",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "network_dns",
+      "actual": "network_dns",
+      "accurate": true,
+      "confidence": 0.9974646130062536,
+      "entropy_nats": 0.01829612663245019,
+      "normalized_entropy": 0.01136802264386781,
+      "brier_score": 0.000012232157543249838,
+      "wall_time_ms": 167,
+      "top_probabilities": {
+        "api_platform": 0.000003597344973499301,
+        "authoritative_dns": 0.0024075350673436646,
+        "endpoint_devices": 0.00005960803410016078,
+        "network_dns": 0.9974646130062536,
+        "tls_certificates": 0.0000646465473291772
+      }
+    },
+    {
+      "id": "hard-sol-b-routing_hard-07",
+      "tier": "hard",
+      "family": "routing_hard",
+      "topic": "support_ops",
+      "group": "hard-sol-b-routing_hard-07",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "release_notarization",
+      "actual": "release_notarization",
+      "accurate": true,
+      "confidence": 0.9972797352792423,
+      "entropy_nats": 0.019501841314043526,
+      "normalized_entropy": 0.01211717529665317,
+      "brier_score": 0.000014024295949086807,
+      "wall_time_ms": 162,
+      "top_probabilities": {
+        "ci_infrastructure": 0.00008893352316887894,
+        "code_signing": 0.0025719195414383228,
+        "endpoint_security": 0.00003211689308923358,
+        "macos_runtime": 0.000027294763061322845,
+        "release_notarization": 0.9972797352792423
+      }
+    },
+    {
+      "id": "hard-sol-b-routing_hard-09",
+      "tier": "hard",
+      "family": "routing_hard",
+      "topic": "support_ops",
+      "group": "hard-sol-b-routing_hard-09",
+      "question_type": "choice",
+      "vocab_cardinality": 5,
+      "chance_baseline": 0.2,
+      "expected": "source_access",
+      "actual": "source_access",
+      "accurate": true,
+      "confidence": 0.997654401801195,
+      "entropy_nats": 0.018430588579720294,
+      "normalized_entropy": 0.011451568549075638,
+      "brier_score": 0.00000848083148310528,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "git_client": 0.0004624806164608156,
+        "identity_login": 0.0000011716005805376443,
+        "repository_admin": 0.0016460441009409487,
+        "source_access": 0.997654401801195,
+        "vpn_network": 0.00023590188082280397
+      }
+    },
+    {
+      "id": "hard-sol-b-temporal_numeric-01",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "math",
+      "group": "hard-sol-b-temporal_numeric-01",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "late_by_under_2h",
+      "actual": "within_window",
+      "accurate": false,
+      "confidence": 0.7456834478281693,
+      "entropy_nats": 0.8122049803143635,
+      "normalized_entropy": 0.5858820486424252,
+      "brier_score": 1.4000304163190305,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "before_open": 0.13206301263989773,
+        "late_by_2h_or_more": 0.030874866204094104,
+        "late_by_under_2h": 0.09137867332783894,
+        "within_window": 0.7456834478281693
+      }
+    },
+    {
+      "id": "hard-sol-b-temporal_numeric-02",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "math",
+      "group": "hard-sol-b-temporal_numeric-02",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "over_limit",
+      "actual": "under_100gb_below",
+      "accurate": false,
+      "confidence": 0.8442133447107492,
+      "entropy_nats": 0.5882549055701037,
+      "normalized_entropy": 0.42433621752229705,
+      "brier_score": 1.6492943685924786,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "at_least_100gb_below": 0.08577609737302659,
+        "exactly_equal": 0.03340295006551968,
+        "over_limit": 0.03660760785070459,
+        "under_100gb_below": 0.8442133447107492
+      }
+    },
+    {
+      "id": "hard-sol-b-temporal_numeric-03",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "math",
+      "group": "hard-sol-b-temporal_numeric-03",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "9_credits",
+      "actual": "9_credits",
+      "accurate": true,
+      "confidence": 0.40602565184855505,
+      "entropy_nats": 1.229650447527804,
+      "normalized_entropy": 0.8870053013376288,
+      "brier_score": 0.5024881477392259,
+      "wall_time_ms": 139,
+      "top_probabilities": {
+        "12_credits": 0.2963201924645335,
+        "15_credits": 0.05507749966779232,
+        "6_credits": 0.24257665601911915,
+        "9_credits": 0.40602565184855505
+      }
+    },
+    {
+      "id": "hard-sol-b-temporal_numeric-04",
+      "tier": "hard",
+      "family": "temporal_numeric",
+      "topic": "math",
+      "group": "hard-sol-b-temporal_numeric-04",
+      "question_type": "choice",
+      "vocab_cardinality": 3,
+      "chance_baseline": 0.3333333333333333,
+      "expected": "pass",
+      "actual": "narrow_fail",
+      "accurate": false,
+      "confidence": 0.3848285560226613,
+      "entropy_nats": 1.0924505628892118,
+      "normalized_entropy": 0.9943913554923295,
+      "brier_score": 0.7425627877704013,
+      "wall_time_ms": 143,
+      "top_probabilities": {
+        "clear_fail": 0.3176951521478614,
+        "narrow_fail": 0.3848285560226613,
+        "pass": 0.2974762918294772
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-03",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "coding",
+      "group": "hard-sol-c-judge_hard-03",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9983469720421618,
+      "entropy_nats": 0.01223954726974955,
+      "normalized_entropy": 0.017657934148793727,
+      "brier_score": 0.00000546500285878927,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "no": 0.9983469720421618,
+        "yes": 0.0016530279578381923
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-05",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-c-judge_hard-05",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9649339081597881,
+      "entropy_nats": 0.15193362650833114,
+      "normalized_entropy": 0.2191938895078453,
+      "brier_score": 0.002459261593892354,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "no": 0.03506609184021195,
+        "yes": 0.9649339081597881
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-07",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-c-judge_hard-07",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "yes",
+      "accurate": false,
+      "confidence": 0.6643855093766137,
+      "entropy_nats": 0.6380836510479194,
+      "normalized_entropy": 0.9205601190391572,
+      "brier_score": 0.8828162101392448,
+      "wall_time_ms": 142,
+      "top_probabilities": {
+        "no": 0.33561449062338633,
+        "yes": 0.6643855093766137
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-08",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "coding",
+      "group": "hard-sol-c-judge_hard-08",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.8545980926218655,
+      "entropy_nats": 0.41464961091000707,
+      "normalized_entropy": 0.5982129373664055,
+      "brier_score": 0.04228342933839922,
+      "wall_time_ms": 160,
+      "top_probabilities": {
+        "no": 0.8545980926218655,
+        "yes": 0.14540190737813455
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-09",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-c-judge_hard-09",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9937095743108,
+      "entropy_nats": 0.038155046943128886,
+      "normalized_entropy": 0.05504609700973765,
+      "brier_score": 0.00007913891070269506,
+      "wall_time_ms": 161,
+      "top_probabilities": {
+        "no": 0.006290425689199977,
+        "yes": 0.9937095743108
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-10",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-c-judge_hard-10",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9966670960394735,
+      "entropy_nats": 0.022337932200278292,
+      "normalized_entropy": 0.03222682400905538,
+      "brier_score": 0.000022216497620187063,
+      "wall_time_ms": 160,
+      "top_probabilities": {
+        "no": 0.003332903960526455,
+        "yes": 0.9966670960394735
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-11",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "coding",
+      "group": "hard-sol-c-judge_hard-11",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.9972643827374149,
+      "entropy_nats": 0.018875838775217032,
+      "normalized_entropy": 0.027232078993625217,
+      "brier_score": 0.000014967203614707064,
+      "wall_time_ms": 159,
+      "top_probabilities": {
+        "no": 0.9972643827374149,
+        "yes": 0.0027356172625850354
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-13",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "math",
+      "group": "hard-sol-c-judge_hard-13",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "no",
+      "actual": "no",
+      "accurate": true,
+      "confidence": 0.6678770618173562,
+      "entropy_nats": 0.6356718879313998,
+      "normalized_entropy": 0.9170806803511555,
+      "brier_score": 0.2206112921341445,
+      "wall_time_ms": 265,
+      "top_probabilities": {
+        "no": 0.6678770618173562,
+        "yes": 0.3321229381826439
+      }
+    },
+    {
+      "id": "hard-sol-c-judge_hard-15",
+      "tier": "hard",
+      "family": "judge_hard",
+      "topic": "coding",
+      "group": "hard-sol-c-judge_hard-15",
+      "question_type": "noul",
+      "vocab_cardinality": 2,
+      "chance_baseline": 0.5,
+      "expected": "yes",
+      "actual": "yes",
+      "accurate": true,
+      "confidence": 0.9889223659177009,
+      "entropy_nats": 0.060896720781130564,
+      "normalized_entropy": 0.08785539707733694,
+      "brier_score": 0.0002454279537226299,
+      "wall_time_ms": 267,
+      "top_probabilities": {
+        "no": 0.011077634082299162,
+        "yes": 0.9889223659177009
+      }
+    },
+    {
+      "id": "hard-sol-c-multi_hop-07",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "law_policy",
+      "group": "hard-sol-c-multi_hop-07",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "programme_review",
+      "actual": "programme_review",
+      "accurate": true,
+      "confidence": 0.999138261250487,
+      "entropy_nats": 0.007619558371795742,
+      "normalized_entropy": 0.005496349538376851,
+      "brier_score": 0.0000011317087434483993,
+      "wall_time_ms": 267,
+      "top_probabilities": {
+        "partial_release": 0.00004669487477488957,
+        "programme_review": 0.999138261250487,
+        "reject_draw": 0.00024201302396581006,
+        "release_draw": 0.0005730308507723107
+      }
+    },
+    {
+      "id": "hard-sol-c-multi_hop-09",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-c-multi_hop-09",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "bus_substitution",
+      "actual": "bus_substitution",
+      "accurate": true,
+      "confidence": 0.9996538027667679,
+      "entropy_nats": 0.0034301133143208404,
+      "normalized_entropy": 0.0024743037341289414,
+      "brier_score": 1.7339840061530228e-7,
+      "wall_time_ms": 267,
+      "top_probabilities": {
+        "bus_substitution": 0.9996538027667679,
+        "call_fen_cross": 0.000210124036357917,
+        "pass_fen_cross": 0.0000597962037690622,
+        "terminate_oakmere": 0.00007627699310521689
+      }
+    },
+    {
+      "id": "hard-sol-c-multi_hop-11",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "finance_commerce",
+      "group": "hard-sol-c-multi_hop-11",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "coverage_excluded",
+      "actual": "coverage_excluded",
+      "accurate": true,
+      "confidence": 0.9620134266581879,
+      "entropy_nats": 0.1754535921681097,
+      "normalized_entropy": 0.12656301366354328,
+      "brier_score": 0.002595435462089009,
+      "wall_time_ms": 255,
+      "top_probabilities": {
+        "adjuster_referral": 0.00012173846728638266,
+        "coverage_confirmed": 0.03368992446833305,
+        "coverage_excluded": 0.9620134266581879,
+        "documentation_needed": 0.004174910406192694
+      }
+    },
+    {
+      "id": "hard-sol-c-multi_hop-12",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-c-multi_hop-12",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "conservation_hold",
+      "actual": "conservation_hold",
+      "accurate": true,
+      "confidence": 0.9356502754240782,
+      "entropy_nats": 0.30228795851015555,
+      "normalized_entropy": 0.21805466933152506,
+      "brier_score": 0.005844118352558467,
+      "wall_time_ms": 253,
+      "top_probabilities": {
+        "conservation_hold": 0.9356502754240782,
+        "deny_request": 0.016706720691237558,
+        "digital_copy_only": 0.03584624463664597,
+        "supervised_delivery": 0.011796759248038297
+      }
+    },
+    {
+      "id": "hard-sol-c-multi_hop-13",
+      "tier": "hard",
+      "family": "multi_hop",
+      "topic": "support_ops",
+      "group": "hard-sol-c-multi_hop-13",
+      "question_type": "choice",
+      "vocab_cardinality": 4,
+      "chance_baseline": 0.25,
+      "expected": "normal_release",
+      "actual": "normal_release",
+      "accurate": true,
+      "confidence": 0.9886430124262914,
+      "entropy_nats": 0.06690132233986944,
+      "normalized_entropy": 0.04825910298432183,
+      "brier_score": 0.00022965999929946967,
+      "wall_time_ms": 253,
+      "top_probabilities": {
+        "engineering_review": 0.009956340697760732,
+        "normal_release": 0.9886430124262914,
+        "rework": 0.0001668411867886074,
+        "scrap_lot": 0.0012338056891593363
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/permutation.json b/benchmarks/runs/20260928-v010-verification/permutation.json
new file mode 100644
index 0000000..c26ce89
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/permutation.json
@@ -0,0 +1,2225 @@
+{
+  "timestamp": "2026-09-28T22:40:21Z",
+  "endpoint": "https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions",
+  "model": "diffgemma-26b-a4b-it-q4",
+  "gate_threshold_normalized_h": 0.16,
+  "prior_damping_alpha": 0.75,
+  "null_prior": {
+    "by_cardinality": {
+      "2": [
+        0.9067105765582258,
+        0.09328942344177421
+      ],
+      "3": [
+        0.5044038628092938,
+        0.3611551848445242,
+        0.13444095234618197
+      ],
+      "4": [
+        0.3302953520031515,
+        0.147258265937074,
+        0.2523974821543615,
+        0.2700488999054129
+      ]
+    },
+    "entropy_by_k": {
+      "2": 0.31008300211546536,
+      "3": 0.9827937981867325,
+      "4": 1.348997133087098
+    },
+    "uniform_entropy_by_k": {
+      "2": 0.6931471805599453,
+      "3": 1.0986122886681096,
+      "4": 1.3862943611198906
+    },
+    "max_bias_ratio_by_k": {
+      "2": 1.8134211531164517,
+      "3": 1.5132115884278816,
+      "4": 1.321181408012606
+    }
+  },
+  "total_cases": 16,
+  "canonical_accuracy_pct": 100,
+  "debiased_accuracy_pct": 100,
+  "dual_mirror_accuracy_pct": 100,
+  "cyclic_ensemble_accuracy_pct": 100,
+  "raw_flip_rate_pct": 6.25,
+  "debiased_flip_rate_pct": 25,
+  "flip_reduction_relative_pct": -300,
+  "canonical_brier": 0.0013316339530038445,
+  "debiased_brier": 0.001004372391416645,
+  "dual_mirror_brier": 0.03844270458090349,
+  "mean_single_pass_normalized_h": 0.04316542095571914,
+  "mean_cyclic_jsd_nats": 0.05462730556466847,
+  "mean_mirror_jsd_nats": 0.015289487549618643,
+  "false_confident_errors_single_pass": 0,
+  "caught_by_dual_mirror_gate": 0,
+  "single_gate_error_recall_pct": 100,
+  "mirror_gate_error_recall_pct": 100,
+  "mean_single_latency_ms": 167.625,
+  "mean_mirror_latency_ms": 152.4375,
+  "regimes": [
+    {
+      "regime": "consensus",
+      "cases": 4,
+      "canonical_accuracy_pct": 100,
+      "dual_mirror_accuracy_pct": 100,
+      "raw_flip_rate_pct": 0,
+      "debiased_flip_rate_pct": 0,
+      "mean_single_normalized_h": 0.015022291320910392,
+      "mean_cyclic_jsd_nats": 0.0012849819539824327,
+      "mean_mirror_jsd_nats": 0.0009026142738583214,
+      "mean_mirror_tvd": 0.004393174854109884,
+      "single_gate_recall_on_errors_pct": 100,
+      "mirror_gate_recall_on_errors_pct": 100
+    },
+    {
+      "regime": "ambiguous_chaosnli",
+      "cases": 4,
+      "canonical_accuracy_pct": 100,
+      "dual_mirror_accuracy_pct": 100,
+      "raw_flip_rate_pct": 25,
+      "debiased_flip_rate_pct": 75,
+      "mean_single_normalized_h": 0.1159222277461991,
+      "mean_cyclic_jsd_nats": 0.16791639835415467,
+      "mean_mirror_jsd_nats": 0.035282340478064105,
+      "mean_mirror_tvd": 0.13167916126339865,
+      "single_gate_recall_on_errors_pct": 100,
+      "mirror_gate_recall_on_errors_pct": 100
+    },
+    {
+      "regime": "adversarial_trap",
+      "cases": 4,
+      "canonical_accuracy_pct": 100,
+      "dual_mirror_accuracy_pct": 100,
+      "raw_flip_rate_pct": 0,
+      "debiased_flip_rate_pct": 25,
+      "mean_single_normalized_h": 0.021413640147608933,
+      "mean_cyclic_jsd_nats": 0.04885735904559453,
+      "mean_mirror_jsd_nats": 0.013106341883889054,
+      "mean_mirror_tvd": 0.06609512226102546,
+      "single_gate_recall_on_errors_pct": 100,
+      "mirror_gate_recall_on_errors_pct": 100
+    },
+    {
+      "regime": "binary_ablation",
+      "cases": 4,
+      "canonical_accuracy_pct": 100,
+      "dual_mirror_accuracy_pct": 100,
+      "raw_flip_rate_pct": 0,
+      "debiased_flip_rate_pct": 0,
+      "mean_single_normalized_h": 0.02030352460815814,
+      "mean_cyclic_jsd_nats": 0.000450482904942238,
+      "mean_mirror_jsd_nats": 0.011866653562663095,
+      "mean_mirror_tvd": 0.0690633040011435,
+      "single_gate_recall_on_errors_pct": 100,
+      "mirror_gate_recall_on_errors_pct": 100
+    }
+  ],
+  "cases": [
+    {
+      "id": "perm_01_consensus_sre_502",
+      "regime": "consensus",
+      "title": "SRE 502 Checkout Crash After v2.14.3 Rollout",
+      "cardinality": 4,
+      "expected": "immediate_k8s_rollback",
+      "canonical_winner": "immediate_k8s_rollback",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9996553473369123,
+      "canonical_entropy": 0.0032339479358714186,
+      "canonical_normalized_h": 0.002332800324787398,
+      "canonical_brier": 2.143335166468619e-7,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "update_billing_address",
+            "immediate_k8s_rollback",
+            "close_as_duplicate",
+            "schedule_next_month"
+          ],
+          "raw_probs": {
+            "close_as_duplicate": 0.000014494714212877542,
+            "immediate_k8s_rollback": 0.9996553473369123,
+            "schedule_next_month": 0.000022187711354856726,
+            "update_billing_address": 0.00030797023752002575
+          },
+          "debiased_probs": {
+            "close_as_duplicate": 0.00000967769761775061,
+            "immediate_k8s_rollback": 0.9998081829841613,
+            "schedule_next_month": 0.000014081757240299927,
+            "update_billing_address": 0.00016805756098069987
+          },
+          "raw_winner": "immediate_k8s_rollback",
+          "debiased_winner": "immediate_k8s_rollback",
+          "raw_entropy": 0.0032339479358714186,
+          "normalized_entropy": 0.002332800324787398,
+          "wall_time_ms": 128
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "immediate_k8s_rollback",
+            "close_as_duplicate",
+            "schedule_next_month",
+            "update_billing_address"
+          ],
+          "raw_probs": {
+            "close_as_duplicate": 0.0006843682301915287,
+            "immediate_k8s_rollback": 0.9992691138723926,
+            "schedule_next_month": 0.000018537325773715105,
+            "update_billing_address": 0.000027980571642076712
+          },
+          "debiased_probs": {
+            "close_as_duplicate": 0.001253590500283565,
+            "immediate_k8s_rollback": 0.9986912179896955,
+            "schedule_next_month": 0.000022667777029727662,
+            "update_billing_address": 0.000032523732991235064
+          },
+          "raw_winner": "immediate_k8s_rollback",
+          "debiased_winner": "immediate_k8s_rollback",
+          "raw_entropy": 0.006212946046330437,
+          "normalized_entropy": 0.004481693225175807,
+          "wall_time_ms": 136
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "close_as_duplicate",
+            "schedule_next_month",
+            "update_billing_address",
+            "immediate_k8s_rollback"
+          ],
+          "raw_probs": {
+            "close_as_duplicate": 0.00012187808746879153,
+            "immediate_k8s_rollback": 0.9997387844566005,
+            "schedule_next_month": 0.00013160078330758159,
+            "update_billing_address": 0.000007736672623200478
+          },
+          "debiased_probs": {
+            "close_as_duplicate": 0.00010478640142363333,
+            "immediate_k8s_rollback": 0.9996797007941013,
+            "schedule_next_month": 0.00020737426267076662,
+            "update_billing_address": 0.000008138541804338884
+          },
+          "raw_winner": "immediate_k8s_rollback",
+          "debiased_winner": "immediate_k8s_rollback",
+          "raw_entropy": 0.002626613517290538,
+          "normalized_entropy": 0.0018947011478634883,
+          "wall_time_ms": 180
+        },
+        {
+          "perm_index": 3,
+          "option_order": [
+            "schedule_next_month",
+            "update_billing_address",
+            "immediate_k8s_rollback",
+            "close_as_duplicate"
+          ],
+          "raw_probs": {
+            "close_as_duplicate": 0.000034663476900981653,
+            "immediate_k8s_rollback": 0.9998923102264555,
+            "schedule_next_month": 0.000060153226980149104,
+            "update_billing_address": 0.000012873069663280973
+          },
+          "debiased_probs": {
+            "close_as_duplicate": 0.00003295010549248144,
+            "immediate_k8s_rollback": 0.9998986021065297,
+            "schedule_next_month": 0.00004916415511802033,
+            "update_billing_address": 0.00001928363285975042
+          },
+          "raw_winner": "immediate_k8s_rollback",
+          "debiased_winner": "immediate_k8s_rollback",
+          "raw_entropy": 0.0011932334294675886,
+          "normalized_entropy": 0.0008607359756579104,
+          "wall_time_ms": 103
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.0033166852322399957,
+      "raw_entropy_stddev": 0.0018289627315719748,
+      "raw_total_entropy": 0.0035662380458935847,
+      "raw_js_divergence_nats": 0.00024955281365358906,
+      "raw_max_tvd": 0.0006698735159787279,
+      "raw_ensemble_winner": "immediate_k8s_rollback",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 1.8745734925380035e-7,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.000408176424919123,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 6.532906508620197e-8,
+      "dual_mirror": {
+        "fwd_probs": {
+          "close_as_duplicate": 0.0004838915225971828,
+          "immediate_k8s_rollback": 0.999349682496204,
+          "schedule_next_month": 0.000005980512931829686,
+          "update_billing_address": 0.0001604454682669512
+        },
+        "rev_probs": {
+          "close_as_duplicate": 0.0008402136541454188,
+          "immediate_k8s_rollback": 0.9987948128751186,
+          "schedule_next_month": 0.0002753220930237667,
+          "update_billing_address": 0.00008965137771220442
+        },
+        "ensemble_probs": {
+          "close_as_duplicate": 0.0006620525883713008,
+          "immediate_k8s_rollback": 0.9990722476856613,
+          "schedule_next_month": 0.0001406513029777982,
+          "update_billing_address": 0.00012504842298957783
+        },
+        "fwd_winner": "immediate_k8s_rollback",
+        "rev_winner": "immediate_k8s_rollback",
+        "ensemble_winner": "immediate_k8s_rollback",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.00803250496134983,
+        "total_entropy": 0.008144911036878902,
+        "normalized_total_h": 0.005875311380693489,
+        "js_divergence_nats": 0.0001124060755290715,
+        "normalized_jsd": 0.00016216768773158173,
+        "tvd": 0.0006256637116401693,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 122
+      }
+    },
+    {
+      "id": "perm_02_consensus_sql_injection",
+      "regime": "consensus",
+      "title": "Direct SQL UNION Injection in Search Query Parameter",
+      "cardinality": 4,
+      "expected": "critical_sqli_exfiltration",
+      "canonical_winner": "critical_sqli_exfiltration",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9995133062351538,
+      "canonical_entropy": 0.004631329433052661,
+      "canonical_normalized_h": 0.003340798002894084,
+      "canonical_brier": 3.4818213127015553e-7,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "benign_search_filter",
+            "css_font_rendering",
+            "critical_sqli_exfiltration",
+            "scheduled_cron_healthcheck"
+          ],
+          "raw_probs": {
+            "benign_search_filter": 0.0003004708436578998,
+            "critical_sqli_exfiltration": 0.9995133062351538,
+            "css_font_rendering": 0.00013605965295141297,
+            "scheduled_cron_healthcheck": 0.00005016326823697357
+          },
+          "debiased_probs": {
+            "benign_search_filter": 0.000245575334829546,
+            "critical_sqli_exfiltration": 0.9995029301198513,
+            "css_font_rendering": 0.00020381157844883606,
+            "scheduled_cron_healthcheck": 0.00004768296687030088
+          },
+          "raw_winner": "critical_sqli_exfiltration",
+          "debiased_winner": "critical_sqli_exfiltration",
+          "raw_entropy": 0.004631329433052661,
+          "normalized_entropy": 0.003340798002894084,
+          "wall_time_ms": 143
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "css_font_rendering",
+            "critical_sqli_exfiltration",
+            "scheduled_cron_healthcheck",
+            "benign_search_filter"
+          ],
+          "raw_probs": {
+            "benign_search_filter": 0.00010934983372194311,
+            "critical_sqli_exfiltration": 0.9993196557533666,
+            "css_font_rendering": 0.000548812942501751,
+            "scheduled_cron_healthcheck": 0.000022181470409794474
+          },
+          "debiased_probs": {
+            "benign_search_filter": 0.00006941046552867808,
+            "critical_sqli_exfiltration": 0.9996162503722538,
+            "css_font_rendering": 0.0002995271108997691,
+            "scheduled_cron_healthcheck": 0.000014812051317664183
+          },
+          "raw_winner": "critical_sqli_exfiltration",
+          "debiased_winner": "critical_sqli_exfiltration",
+          "raw_entropy": 0.0060355422518634265,
+          "normalized_entropy": 0.004353723437919586,
+          "wall_time_ms": 121
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "critical_sqli_exfiltration",
+            "scheduled_cron_healthcheck",
+            "benign_search_filter",
+            "css_font_rendering"
+          ],
+          "raw_probs": {
+            "benign_search_filter": 0.000014605471330500367,
+            "critical_sqli_exfiltration": 0.9937889468743037,
+            "css_font_rendering": 0.00017014122499020256,
+            "scheduled_cron_healthcheck": 0.0060263064293754704
+          },
+          "debiased_probs": {
+            "benign_search_filter": 0.000017780382265953256,
+            "critical_sqli_exfiltration": 0.9887957618454171,
+            "css_font_rendering": 0.00019688699330446934,
+            "scheduled_cron_healthcheck": 0.010989570779012404
+          },
+          "raw_winner": "critical_sqli_exfiltration",
+          "debiased_winner": "critical_sqli_exfiltration",
+          "raw_entropy": 0.03863517346040022,
+          "normalized_entropy": 0.027869386577602146,
+          "wall_time_ms": 120
+        },
+        {
+          "perm_index": 3,
+          "option_order": [
+            "scheduled_cron_healthcheck",
+            "benign_search_filter",
+            "css_font_rendering",
+            "critical_sqli_exfiltration"
+          ],
+          "raw_probs": {
+            "benign_search_filter": 0.0005994445921761401,
+            "critical_sqli_exfiltration": 0.9991735058979245,
+            "css_font_rendering": 0.0000026339175600190896,
+            "scheduled_cron_healthcheck": 0.00022441559233927168
+          },
+          "debiased_probs": {
+            "benign_search_filter": 0.0009443539432804458,
+            "critical_sqli_exfiltration": 0.9988599807085532,
+            "css_font_rendering": 0.0000027700265207514192,
+            "scheduled_cron_healthcheck": 0.00019289532164559444
+          },
+          "raw_winner": "critical_sqli_exfiltration",
+          "debiased_winner": "critical_sqli_exfiltration",
+          "raw_entropy": 0.007193116109241674,
+          "normalized_entropy": 0.005188736469670739,
+          "wall_time_ms": 135
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.014123790313639493,
+      "raw_entropy_stddev": 0.014180697435009207,
+      "raw_total_entropy": 0.01617298093236909,
+      "raw_js_divergence_nats": 0.002049190618729598,
+      "raw_max_tvd": 0.006010224733177357,
+      "raw_ensemble_winner": "critical_sqli_exfiltration",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.000006817516446292708,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.003809619493963936,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 3.511985356670949e-7,
+      "dual_mirror": {
+        "fwd_probs": {
+          "benign_search_filter": 0.00034936141286386204,
+          "critical_sqli_exfiltration": 0.9832340998801727,
+          "css_font_rendering": 0.01628943385101988,
+          "scheduled_cron_healthcheck": 0.00012710485594352093
+        },
+        "rev_probs": {
+          "benign_search_filter": 0.00004344640606880697,
+          "critical_sqli_exfiltration": 0.9978095325541142,
+          "css_font_rendering": 0.0019346130397385611,
+          "scheduled_cron_healthcheck": 0.00021240800007837302
+        },
+        "ensemble_probs": {
+          "benign_search_filter": 0.00019640390946633451,
+          "critical_sqli_exfiltration": 0.9905218162171434,
+          "css_font_rendering": 0.00911202344537922,
+          "scheduled_cron_healthcheck": 0.00016975642801094696
+        },
+        "fwd_winner": "critical_sqli_exfiltration",
+        "rev_winner": "critical_sqli_exfiltration",
+        "ensemble_winner": "critical_sqli_exfiltration",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.052060446571990396,
+        "total_entropy": 0.05539292558021246,
+        "normalized_total_h": 0.03995754951745196,
+        "js_divergence_nats": 0.0033324790082220612,
+        "normalized_jsd": 0.004807750939028539,
+        "tvd": 0.01466073581807634,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 123
+      }
+    },
+    {
+      "id": "perm_03_consensus_nda_breach",
+      "regime": "consensus",
+      "title": "NDA Reverse Engineering Prohibition Clause",
+      "cardinality": 3,
+      "expected": "material_breach_of_clause_4_2",
+      "canonical_winner": "material_breach_of_clause_4_2",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.998598488507624,
+      "canonical_entropy": 0.01131786444323776,
+      "canonical_normalized_h": 0.01030196417788012,
+      "canonical_brier": 0.000003290429610831965,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "material_breach_of_clause_4_2",
+            "fully_compliant_permitted_use",
+            "unrelated_tax_withholding"
+          ],
+          "raw_probs": {
+            "fully_compliant_permitted_use": 0.0011155315385981354,
+            "material_breach_of_clause_4_2": 0.998598488507624,
+            "unrelated_tax_withholding": 0.00028597995377797436
+          },
+          "debiased_probs": {
+            "fully_compliant_permitted_use": 0.001432012617832265,
+            "material_breach_of_clause_4_2": 0.9977976654049734,
+            "unrelated_tax_withholding": 0.0007703219771943936
+          },
+          "raw_winner": "material_breach_of_clause_4_2",
+          "debiased_winner": "material_breach_of_clause_4_2",
+          "raw_entropy": 0.01131786444323776,
+          "normalized_entropy": 0.01030196417788012,
+          "wall_time_ms": 148
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "fully_compliant_permitted_use",
+            "unrelated_tax_withholding",
+            "material_breach_of_clause_4_2"
+          ],
+          "raw_probs": {
+            "fully_compliant_permitted_use": 0.000507511893665325,
+            "material_breach_of_clause_4_2": 0.9994494545632943,
+            "unrelated_tax_withholding": 0.00004303354304048236
+          },
+          "debiased_probs": {
+            "fully_compliant_permitted_use": 0.00018832568831844175,
+            "material_breach_of_clause_4_2": 0.9997911587290537,
+            "unrelated_tax_withholding": 0.000020515582627870387
+          },
+          "raw_winner": "material_breach_of_clause_4_2",
+          "debiased_winner": "material_breach_of_clause_4_2",
+          "raw_entropy": 0.004833013262869355,
+          "normalized_entropy": 0.004399198254671451,
+          "wall_time_ms": 111
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "unrelated_tax_withholding",
+            "material_breach_of_clause_4_2",
+            "fully_compliant_permitted_use"
+          ],
+          "raw_probs": {
+            "fully_compliant_permitted_use": 0.00005406038120072449,
+            "material_breach_of_clause_4_2": 0.9988787536979259,
+            "unrelated_tax_withholding": 0.0010671859208734164
+          },
+          "debiased_probs": {
+            "fully_compliant_permitted_use": 0.00011345607328620128,
+            "material_breach_of_clause_4_2": 0.9990557302940127,
+            "unrelated_tax_withholding": 0.0008308136327009956
+          },
+          "raw_winner": "material_breach_of_clause_4_2",
+          "debiased_winner": "material_breach_of_clause_4_2",
+          "raw_entropy": 0.008954248021392873,
+          "normalized_entropy": 0.008150507794017539,
+          "wall_time_ms": 115
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.008368375242499997,
+      "raw_entropy_stddev": 0.0026796466051411975,
+      "raw_total_entropy": 0.008781519842483603,
+      "raw_js_divergence_nats": 0.0004131445999836069,
+      "raw_max_tvd": 0.00106147115739737,
+      "raw_ensemble_winner": "material_breach_of_clause_4_2",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.0000015785825295257227,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.0004891228981323964,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.000007494333754630526,
+      "dual_mirror": {
+        "fwd_probs": {
+          "fully_compliant_permitted_use": 0.00035634424683213565,
+          "material_breach_of_clause_4_2": 0.9955506434995045,
+          "unrelated_tax_withholding": 0.0040930122536633395
+        },
+        "rev_probs": {
+          "fully_compliant_permitted_use": 0.00017404170254680783,
+          "material_breach_of_clause_4_2": 0.9969285490395294,
+          "unrelated_tax_withholding": 0.0028974092579237875
+        },
+        "ensemble_probs": {
+          "fully_compliant_permitted_use": 0.00026519297468947174,
+          "material_breach_of_clause_4_2": 0.996239596269517,
+          "unrelated_tax_withholding": 0.0034952107557935635
+        },
+        "fwd_winner": "material_breach_of_clause_4_2",
+        "rev_winner": "material_breach_of_clause_4_2",
+        "ensemble_winner": "material_breach_of_clause_4_2",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.025639776699753033,
+        "total_entropy": 0.02570737859308541,
+        "normalized_total_h": 0.023399864409173382,
+        "js_divergence_nats": 0.00006760189333237729,
+        "normalized_jsd": 0.0000975289162653254,
+        "tvd": 0.001377905540024903,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 121
+      }
+    },
+    {
+      "id": "perm_04_consensus_fx_tool",
+      "regime": "consensus",
+      "title": "ISO Currency Spot Conversion Tool Selection",
+      "cardinality": 4,
+      "expected": "convert_currency_spot_fx",
+      "canonical_winner": "convert_currency_spot_fx",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.990338791119145,
+      "canonical_entropy": 0.06115443877993499,
+      "canonical_normalized_h": 0.04411360277807996,
+      "canonical_brier": 0.00014572061694171902,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "resize_png_thumbnail",
+            "mute_microphone_audio",
+            "convert_currency_spot_fx",
+            "reboot_printer_spooler"
+          ],
+          "raw_probs": {
+            "convert_currency_spot_fx": 0.990338791119145,
+            "mute_microphone_audio": 0.002775038523394905,
+            "reboot_printer_spooler": 0.006681228937112397,
+            "resize_png_thumbnail": 0.000204941420347693
+          },
+          "debiased_probs": {
+            "convert_currency_spot_fx": 0.9893354376601136,
+            "mute_microphone_audio": 0.004152721378502263,
+            "reboot_printer_spooler": 0.0063445099508335845,
+            "resize_png_thumbnail": 0.00016733101055072493
+          },
+          "raw_winner": "convert_currency_spot_fx",
+          "debiased_winner": "convert_currency_spot_fx",
+          "raw_entropy": 0.06115443877993499,
+          "normalized_entropy": 0.04411360277807996,
+          "wall_time_ms": 144
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "mute_microphone_audio",
+            "convert_currency_spot_fx",
+            "reboot_printer_spooler",
+            "resize_png_thumbnail"
+          ],
+          "raw_probs": {
+            "convert_currency_spot_fx": 0.9985923789753818,
+            "mute_microphone_audio": 0.0005990959892730246,
+            "reboot_printer_spooler": 0.00020942904607227492,
+            "resize_png_thumbnail": 0.0005990959892730246
+          },
+          "debiased_probs": {
+            "convert_currency_spot_fx": 0.9991526763934473,
+            "mute_microphone_audio": 0.00032705661367673735,
+            "reboot_printer_spooler": 0.00013988673097044672,
+            "resize_png_thumbnail": 0.0003803802619056094
+          },
+          "raw_winner": "convert_currency_spot_fx",
+          "debiased_winner": "convert_currency_spot_fx",
+          "raw_entropy": 0.012071420394364122,
+          "normalized_entropy": 0.008707689169717507,
+          "wall_time_ms": 115
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "convert_currency_spot_fx",
+            "reboot_printer_spooler",
+            "resize_png_thumbnail",
+            "mute_microphone_audio"
+          ],
+          "raw_probs": {
+            "convert_currency_spot_fx": 0.9929406396326934,
+            "mute_microphone_audio": 0.0006596912724937068,
+            "reboot_printer_spooler": 0.0062343015913064286,
+            "resize_png_thumbnail": 0.00016536750350643666
+          },
+          "debiased_probs": {
+            "convert_currency_spot_fx": 0.9876699391524996,
+            "mute_microphone_audio": 0.000763175334140033,
+            "reboot_printer_spooler": 0.011365628138904675,
+            "resize_png_thumbnail": 0.00020125737445581113
+          },
+          "raw_winner": "convert_currency_spot_fx",
+          "debiased_winner": "convert_currency_spot_fx",
+          "raw_entropy": 0.044961544670569696,
+          "normalized_entropy": 0.032432898763469255,
+          "wall_time_ms": 129
+        },
+        {
+          "perm_index": 3,
+          "option_order": [
+            "reboot_printer_spooler",
+            "resize_png_thumbnail",
+            "mute_microphone_audio",
+            "convert_currency_spot_fx"
+          ],
+          "raw_probs": {
+            "convert_currency_spot_fx": 0.9996163010735659,
+            "mute_microphone_audio": 0.000012414041461309556,
+            "reboot_printer_spooler": 0.0002391647488629674,
+            "resize_png_thumbnail": 0.00013212013610983223
+          },
+          "debiased_probs": {
+            "convert_currency_spot_fx": 0.9995731164344425,
+            "mute_microphone_audio": 0.000013059076764320227,
+            "reboot_printer_spooler": 0.00020562852884431447,
+            "resize_png_thumbnail": 0.0002081959599488889
+          },
+          "raw_winner": "convert_currency_spot_fx",
+          "debiased_winner": "convert_currency_spot_fx",
+          "raw_entropy": 0.0036981745549481745,
+          "normalized_entropy": 0.0026676690453827402,
+          "wall_time_ms": 136
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.030471394599954245,
+      "raw_entropy_stddev": 0.023488177938935068,
+      "raw_total_entropy": 0.03289943438351718,
+      "raw_js_divergence_nats": 0.0024280397835629366,
+      "raw_max_tvd": 0.00927750995442089,
+      "raw_ensemble_winner": "convert_currency_spot_fx",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.00003367970467641784,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.003873877419443797,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.00017125879093211218,
+      "dual_mirror": {
+        "fwd_probs": {
+          "convert_currency_spot_fx": 0.9981510562680271,
+          "mute_microphone_audio": 0.0017657420364147493,
+          "reboot_printer_spooler": 0.000028223594010341966,
+          "resize_png_thumbnail": 0.00005497810154776246
+        },
+        "rev_probs": {
+          "convert_currency_spot_fx": 0.9989510249657659,
+          "mute_microphone_audio": 0.0008670767573947832,
+          "reboot_printer_spooler": 0.00013664924296967313,
+          "resize_png_thumbnail": 0.0000452490338696048
+        },
+        "ensemble_probs": {
+          "convert_currency_spot_fx": 0.9985510406168965,
+          "mute_microphone_audio": 0.0013164093969047662,
+          "reboot_printer_spooler": 0.00008243641849000755,
+          "resize_png_thumbnail": 0.00005011356770868363
+        },
+        "fwd_winner": "convert_currency_spot_fx",
+        "rev_winner": "convert_currency_spot_fx",
+        "ensemble_winner": "convert_currency_spot_fx",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.011352856542346015,
+        "total_entropy": 0.01145082666069579,
+        "normalized_total_h": 0.008260025418732473,
+        "js_divergence_nats": 0.00009797011834977547,
+        "normalized_jsd": 0.0001413410038985259,
+        "tvd": 0.0009083943466981252,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 139
+      }
+    },
+    {
+      "id": "perm_05_ambiguous_chaosnli_motive",
+      "regime": "ambiguous_chaosnli",
+      "title": "ChaosNLI Pragmatic Inference: Sudden Resignation Without Notice",
+      "cardinality": 3,
+      "expected": "neutral",
+      "canonical_winner": "neutral",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9803449915261974,
+      "canonical_entropy": 0.10117785014804008,
+      "canonical_normalized_h": 0.09209604807051806,
+      "canonical_brier": 0.0007287676048190504,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "entailment",
+            "neutral",
+            "contradiction"
+          ],
+          "raw_probs": {
+            "contradiction": 0.0011878118910718307,
+            "entailment": 0.01846719658273083,
+            "neutral": 0.9803449915261974
+          },
+          "debiased_probs": {
+            "contradiction": 0.002499378306394455,
+            "entailment": 0.01441451989637662,
+            "neutral": 0.9830861017972291
+          },
+          "raw_winner": "neutral",
+          "debiased_winner": "neutral",
+          "raw_entropy": 0.10117785014804008,
+          "normalized_entropy": 0.09209604807051806,
+          "wall_time_ms": 113
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "neutral",
+            "contradiction",
+            "entailment"
+          ],
+          "raw_probs": {
+            "contradiction": 0.03478730909143531,
+            "entailment": 0.33919184827109483,
+            "neutral": 0.6260208426374698
+          },
+          "debiased_probs": {
+            "contradiction": 0.02819533261239677,
+            "entailment": 0.5768640809535532,
+            "neutral": 0.39494058643404995
+          },
+          "raw_winner": "neutral",
+          "debiased_winner": "entailment",
+          "raw_entropy": 0.7767742950510477,
+          "normalized_entropy": 0.7070504335908725,
+          "wall_time_ms": 123
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "contradiction",
+            "entailment",
+            "neutral"
+          ],
+          "raw_probs": {
+            "contradiction": 0.003040877891775809,
+            "entailment": 0.005227825359010741,
+            "neutral": 0.9917312967492135
+          },
+          "debiased_probs": {
+            "contradiction": 0.0011332812891798425,
+            "entailment": 0.002503071300340363,
+            "neutral": 0.9963636474104799
+          },
+          "raw_winner": "neutral",
+          "debiased_winner": "neutral",
+          "raw_entropy": 0.0533239014387363,
+          "normalized_entropy": 0.04853750680632103,
+          "wall_time_ms": 125
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.3104253488792747,
+      "raw_entropy_stddev": 0.33033670176393315,
+      "raw_total_entropy": 0.4365441189756519,
+      "raw_js_divergence_nats": 0.12611877009637718,
+      "raw_max_tvd": 0.35432414888872754,
+      "raw_ensemble_winner": "neutral",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.032748338324609376,
+      "debiased_flip_occurred": true,
+      "debiased_js_divergence_nats": 0.25259360578929263,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.0005001052281752091,
+      "dual_mirror": {
+        "fwd_probs": {
+          "contradiction": 0.0002690368621119843,
+          "entailment": 0.04705689150371655,
+          "neutral": 0.9526740716341715
+        },
+        "rev_probs": {
+          "contradiction": 0.015118236639913475,
+          "entailment": 0.007756934443373916,
+          "neutral": 0.9771248289167126
+        },
+        "ensemble_probs": {
+          "contradiction": 0.00769363675101273,
+          "entailment": 0.027406912973545233,
+          "neutral": 0.964899450275442
+        },
+        "fwd_winner": "neutral",
+        "rev_winner": "neutral",
+        "ensemble_winner": "neutral",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.15795070305545686,
+        "total_entropy": 0.1705064710347847,
+        "normalized_total_h": 0.15520167832957368,
+        "js_divergence_nats": 0.012555767979327837,
+        "normalized_jsd": 0.01811414419832871,
+        "tvd": 0.039299957060342616,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 147
+      }
+    },
+    {
+      "id": "perm_06_ambiguous_chaosnli_hospital",
+      "regime": "ambiguous_chaosnli",
+      "title": "ChaosNLI Temporal Ambiguity: Surgeon Leaving Operating Theater",
+      "cardinality": 3,
+      "expected": "neutral",
+      "canonical_winner": "neutral",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9986623392707739,
+      "canonical_entropy": 0.010239090203329424,
+      "canonical_normalized_h": 0.009320021548041003,
+      "canonical_brier": 0.000003556193841906233,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "entailment",
+            "neutral",
+            "contradiction"
+          ],
+          "raw_probs": {
+            "contradiction": 0.000008455659564730605,
+            "entailment": 0.0013292050696614662,
+            "neutral": 0.9986623392707739
+          },
+          "debiased_probs": {
+            "contradiction": 0.000017747742470106502,
+            "entailment": 0.0010349098016162752,
+            "neutral": 0.9989473424559135
+          },
+          "raw_winner": "neutral",
+          "debiased_winner": "neutral",
+          "raw_entropy": 0.010239090203329424,
+          "normalized_entropy": 0.009320021548041003,
+          "wall_time_ms": 119
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "neutral",
+            "contradiction",
+            "entailment"
+          ],
+          "raw_probs": {
+            "contradiction": 0.058300525878129034,
+            "entailment": 0.3931937371194457,
+            "neutral": 0.5485057370024253
+          },
+          "debiased_probs": {
+            "contradiction": 0.04449444707007973,
+            "entailment": 0.6296680235622611,
+            "neutral": 0.3258375293676592
+          },
+          "raw_winner": "neutral",
+          "debiased_winner": "entailment",
+          "raw_entropy": 0.8621355577469384,
+          "normalized_entropy": 0.7847496033310704,
+          "wall_time_ms": 112
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "contradiction",
+            "entailment",
+            "neutral"
+          ],
+          "raw_probs": {
+            "contradiction": 0.00250878319175244,
+            "entailment": 0.7444191458982025,
+            "neutral": 0.2530720709100451
+          },
+          "debiased_probs": {
+            "contradiction": 0.0015287042048618708,
+            "entailment": 0.582762067527125,
+            "neutral": 0.41570922826801315
+          },
+          "raw_winner": "entailment",
+          "debiased_winner": "entailment",
+          "raw_entropy": 0.5824800836170997,
+          "normalized_entropy": 0.5301962208371644,
+          "wall_time_ms": 122
+        }
+      ],
+      "raw_flip_occurred": true,
+      "raw_distinct_winners": 2,
+      "raw_majority_agreement": 0.6666666666666666,
+      "raw_mean_entropy": 0.4849515771891226,
+      "raw_entropy_stddev": 0.35455677028074906,
+      "raw_total_entropy": 0.753181802475568,
+      "raw_js_divergence_nats": 0.2682302252864454,
+      "raw_max_tvd": 0.7455902683607287,
+      "raw_ensemble_winner": "neutral",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.3044790649949167,
+      "debiased_flip_occurred": true,
+      "debiased_js_divergence_nats": 0.24842730593378826,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.00000217944118496635,
+      "dual_mirror": {
+        "fwd_probs": {
+          "contradiction": 0.00007062358232488894,
+          "entailment": 0.0004435843845515856,
+          "neutral": 0.9994857920331235
+        },
+        "rev_probs": {
+          "contradiction": 0.0004929119582686426,
+          "entailment": 0.001262900738250908,
+          "neutral": 0.9982441873034805
+        },
+        "ensemble_probs": {
+          "contradiction": 0.00028176777029676576,
+          "entailment": 0.0008532425614012468,
+          "neutral": 0.998864989668302
+        },
+        "fwd_winner": "neutral",
+        "rev_winner": "neutral",
+        "ensemble_winner": "neutral",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.009275385749997651,
+        "total_entropy": 0.009467066264596553,
+        "normalized_total_h": 0.00861729507511139,
+        "js_divergence_nats": 0.00019168051459890131,
+        "normalized_jsd": 0.0002765365278468795,
+        "tvd": 0.0012416047296430204,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 125
+      }
+    },
+    {
+      "id": "perm_07_ambiguous_dual_intent_vip",
+      "regime": "ambiguous_chaosnli",
+      "title": "Equally Weighted Multi-Department Ticket (SRE Outage vs. Legal Termination)",
+      "cardinality": 3,
+      "expected": "sre_identity_platform_oncall",
+      "canonical_winner": "sre_identity_platform_oncall",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.8949484139137923,
+      "canonical_entropy": 0.36950825273860094,
+      "canonical_normalized_h": 0.3363409062050181,
+      "canonical_brier": 0.02013742048266968,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "legal_and_contracts_desk",
+            "sre_identity_platform_oncall",
+            "finance_accounts_payable"
+          ],
+          "raw_probs": {
+            "finance_accounts_payable": 0.010195739453320616,
+            "legal_and_contracts_desk": 0.094855846632887,
+            "sre_identity_platform_oncall": 0.8949484139137923
+          },
+          "debiased_probs": {
+            "finance_accounts_payable": 0.021606196226275857,
+            "legal_and_contracts_desk": 0.07456560985944478,
+            "sre_identity_platform_oncall": 0.9038281939142795
+          },
+          "raw_winner": "sre_identity_platform_oncall",
+          "debiased_winner": "sre_identity_platform_oncall",
+          "raw_entropy": 0.36950825273860094,
+          "normalized_entropy": 0.3363409062050181,
+          "wall_time_ms": 146
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "sre_identity_platform_oncall",
+            "finance_accounts_payable",
+            "legal_and_contracts_desk"
+          ],
+          "raw_probs": {
+            "finance_accounts_payable": 0.06141270956707886,
+            "legal_and_contracts_desk": 0.04488030708910998,
+            "sre_identity_platform_oncall": 0.8937069833438112
+          },
+          "debiased_probs": {
+            "finance_accounts_payable": 0.0721465714310343,
+            "legal_and_contracts_desk": 0.1106330698498861,
+            "sre_identity_platform_oncall": 0.8172203587190795
+          },
+          "raw_winner": "sre_identity_platform_oncall",
+          "debiased_winner": "sre_identity_platform_oncall",
+          "raw_entropy": 0.41107988639584253,
+          "normalized_entropy": 0.37418103787479995,
+          "wall_time_ms": 96
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "finance_accounts_payable",
+            "legal_and_contracts_desk",
+            "sre_identity_platform_oncall"
+          ],
+          "raw_probs": {
+            "finance_accounts_payable": 0.006049968550571763,
+            "legal_and_contracts_desk": 0.3207570037303785,
+            "sre_identity_platform_oncall": 0.6731930277190497
+          },
+          "debiased_probs": {
+            "finance_accounts_payable": 0.0027094417604103618,
+            "legal_and_contracts_desk": 0.18455094079136258,
+            "sre_identity_platform_oncall": 0.8127396174482271
+          },
+          "raw_winner": "sre_identity_platform_oncall",
+          "debiased_winner": "sre_identity_platform_oncall",
+          "raw_entropy": 0.6620231473104294,
+          "normalized_entropy": 0.6025994376169103,
+          "wall_time_ms": 91
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.4808704288149577,
+      "raw_entropy_stddev": 0.12921372668298498,
+      "raw_total_entropy": 0.5444902605397816,
+      "raw_js_divergence_nats": 0.06361983172482388,
+      "raw_max_tvd": 0.22590115709749153,
+      "raw_ensemble_winner": "sre_identity_platform_oncall",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.05641021062002165,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.023728860874963842,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.015275874174868694,
+      "dual_mirror": {
+        "fwd_probs": {
+          "finance_accounts_payable": 0.16647959093793657,
+          "legal_and_contracts_desk": 0.13892470516537817,
+          "sre_identity_platform_oncall": 0.6945957038966852
+        },
+        "rev_probs": {
+          "finance_accounts_payable": 0.5636271943934891,
+          "legal_and_contracts_desk": 0.21818640280325546,
+          "sre_identity_platform_oncall": 0.21818640280325546
+        },
+        "ensemble_probs": {
+          "finance_accounts_payable": 0.3650533926657128,
+          "legal_and_contracts_desk": 0.17855555398431683,
+          "sre_identity_platform_oncall": 0.4563910533499703
+        },
+        "fwd_winner": "sre_identity_platform_oncall",
+        "rev_winner": "finance_accounts_payable",
+        "ensemble_winner": "sre_identity_platform_oncall",
+        "mirror_disagreed": true,
+        "mean_pass_entropy": 0.9066591735089056,
+        "total_entropy": 1.033489521164398,
+        "normalized_total_h": 0.9407227024716223,
+        "js_divergence_nats": 0.1268303476554924,
+        "normalized_jsd": 0.18297751359680206,
+        "tvd": 0.4764093010934298,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": true,
+        "wall_time_ms": 96
+      }
+    },
+    {
+      "id": "perm_08_ambiguous_fair_use_parody",
+      "regime": "ambiguous_chaosnli",
+      "title": "Borderline Copyright Fair-Use vs. Commercial Sampling",
+      "cardinality": 3,
+      "expected": "incidental_fair_use",
+      "canonical_winner": "incidental_fair_use",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9956085714360315,
+      "canonical_entropy": 0.02848914263706009,
+      "canonical_normalized_h": 0.02593193516121924,
+      "canonical_brier": 0.00003814134704256274,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "requires_sync_license",
+            "incidental_fair_use",
+            "public_domain_waiver"
+          ],
+          "raw_probs": {
+            "incidental_fair_use": 0.9956085714360315,
+            "public_domain_waiver": 0.00004927772411613593,
+            "requires_sync_license": 0.004342150839852365
+          },
+          "debiased_probs": {
+            "incidental_fair_use": 0.9965136293323114,
+            "public_domain_waiver": 0.00010349442671833387,
+            "requires_sync_license": 0.0033828762409704315
+          },
+          "raw_winner": "incidental_fair_use",
+          "debiased_winner": "incidental_fair_use",
+          "raw_entropy": 0.02848914263706009,
+          "normalized_entropy": 0.02593193516121924,
+          "wall_time_ms": 219
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "incidental_fair_use",
+            "public_domain_waiver",
+            "requires_sync_license"
+          ],
+          "raw_probs": {
+            "incidental_fair_use": 0.48935796750161586,
+            "public_domain_waiver": 0.021284064996768316,
+            "requires_sync_license": 0.48935796750161586
+          },
+          "debiased_probs": {
+            "incidental_fair_use": 0.2665485164388902,
+            "public_domain_waiver": 0.01489421418027166,
+            "requires_sync_license": 0.7185572693808382
+          },
+          "raw_winner": "incidental_fair_use",
+          "debiased_winner": "requires_sync_license",
+          "raw_entropy": 0.7813894471303294,
+          "normalized_entropy": 0.7112513260502832,
+          "wall_time_ms": 150
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "public_domain_waiver",
+            "requires_sync_license",
+            "incidental_fair_use"
+          ],
+          "raw_probs": {
+            "incidental_fair_use": 0.9973852601989008,
+            "public_domain_waiver": 0.0006108384100088818,
+            "requires_sync_license": 0.0020039013910902385
+          },
+          "debiased_probs": {
+            "incidental_fair_use": 0.9988167111302495,
+            "public_domain_waiver": 0.00022691545407062117,
+            "requires_sync_license": 0.0009563734156798037
+          },
+          "raw_winner": "incidental_fair_use",
+          "debiased_winner": "incidental_fair_use",
+          "raw_entropy": 0.019581493450040822,
+          "normalized_entropy": 0.017823843454163643,
+          "wall_time_ms": 216
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.2764866944058101,
+      "raw_entropy_stddev": 0.3570386802902524,
+      "raw_total_entropy": 0.49018346071478236,
+      "raw_js_divergence_nats": 0.21369676630897227,
+      "raw_max_tvd": 0.5062506039344157,
+      "raw_ensemble_winner": "incidental_fair_use",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.05712929801352116,
+      "debiased_flip_occurred": true,
+      "debiased_js_divergence_nats": 0.35399915391712544,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.00002360934319060375,
+      "dual_mirror": {
+        "fwd_probs": {
+          "incidental_fair_use": 0.9867028625851317,
+          "public_domain_waiver": 0.002815964541570554,
+          "requires_sync_license": 0.010481172873297796
+        },
+        "rev_probs": {
+          "incidental_fair_use": 0.9964686447553109,
+          "public_domain_waiver": 0.0010635557115172092,
+          "requires_sync_license": 0.0024677995331718623
+        },
+        "ensemble_probs": {
+          "incidental_fair_use": 0.9915857536702213,
+          "public_domain_waiver": 0.0019397601265438815,
+          "requires_sync_license": 0.006474486203234829
+        },
+        "fwd_winner": "incidental_fair_use",
+        "rev_winner": "incidental_fair_use",
+        "ensemble_winner": "incidental_fair_use",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.051572026032507635,
+        "total_entropy": 0.053123591795344934,
+        "normalized_total_h": 0.04835517711143458,
+        "js_divergence_nats": 0.001551565762837298,
+        "normalized_jsd": 0.0022384362316584714,
+        "tvd": 0.009765782170179241,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 214
+      }
+    },
+    {
+      "id": "perm_09_trap_amendment_override",
+      "regime": "adversarial_trap",
+      "title": "JevBench Hard Precedence Trap: Base Policy vs. Appendix C Carve-Out",
+      "cardinality": 4,
+      "expected": "AUTO_SETTLE_EXEMPT",
+      "canonical_winner": "AUTO_SETTLE_EXEMPT",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9944872194142272,
+      "canonical_entropy": 0.039476505071135155,
+      "canonical_normalized_h": 0.028476279048877354,
+      "canonical_brier": 0.00004282030779127643,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "HOLD_FOR_VP",
+            "REJECT_WIRE_FRAUD",
+            "AUTO_SETTLE_EXEMPT",
+            "ESCALATE_TO_BOARD"
+          ],
+          "raw_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9944872194142272,
+            "ESCALATE_TO_BOARD": 0.0006108110040104982,
+            "HOLD_FOR_VP": 0.002595579105295059,
+            "REJECT_WIRE_FRAUD": 0.002306390476467239
+          },
+          "debiased_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9938470435573351,
+            "ESCALATE_TO_BOARD": 0.0005802419857254197,
+            "HOLD_FOR_VP": 0.002120027675113097,
+            "REJECT_WIRE_FRAUD": 0.0034526867818262526
+          },
+          "raw_winner": "AUTO_SETTLE_EXEMPT",
+          "debiased_winner": "AUTO_SETTLE_EXEMPT",
+          "raw_entropy": 0.039476505071135155,
+          "normalized_entropy": 0.028476279048877354,
+          "wall_time_ms": 134
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "REJECT_WIRE_FRAUD",
+            "AUTO_SETTLE_EXEMPT",
+            "ESCALATE_TO_BOARD",
+            "HOLD_FOR_VP"
+          ],
+          "raw_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9945841932988636,
+            "ESCALATE_TO_BOARD": 0.0013972937847467812,
+            "HOLD_FOR_VP": 0.001667274510162872,
+            "REJECT_WIRE_FRAUD": 0.0023512384062267765
+          },
+          "debiased_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9967193231431839,
+            "ESCALATE_TO_BOARD": 0.0009347921361711955,
+            "HOLD_FOR_VP": 0.0010602697913608054,
+            "REJECT_WIRE_FRAUD": 0.0012856149292840107
+          },
+          "raw_winner": "AUTO_SETTLE_EXEMPT",
+          "debiased_winner": "AUTO_SETTLE_EXEMPT",
+          "raw_entropy": 0.039482267672832,
+          "normalized_entropy": 0.028480435887322682,
+          "wall_time_ms": 195
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "AUTO_SETTLE_EXEMPT",
+            "ESCALATE_TO_BOARD",
+            "HOLD_FOR_VP",
+            "REJECT_WIRE_FRAUD"
+          ],
+          "raw_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9959922237408823,
+            "ESCALATE_TO_BOARD": 0.00159024118717175,
+            "HOLD_FOR_VP": 0.0018894525348233159,
+            "REJECT_WIRE_FRAUD": 0.0005280825371226812
+          },
+          "debiased_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9941701022195656,
+            "ESCALATE_TO_BOARD": 0.002909275390986097,
+            "HOLD_FOR_VP": 0.0023075643466506043,
+            "REJECT_WIRE_FRAUD": 0.0006130580427977018
+          },
+          "raw_winner": "AUTO_SETTLE_EXEMPT",
+          "debiased_winner": "AUTO_SETTLE_EXEMPT",
+          "raw_entropy": 0.030081729647815702,
+          "normalized_entropy": 0.02169938109213311,
+          "wall_time_ms": 181
+        },
+        {
+          "perm_index": 3,
+          "option_order": [
+            "ESCALATE_TO_BOARD",
+            "HOLD_FOR_VP",
+            "REJECT_WIRE_FRAUD",
+            "AUTO_SETTLE_EXEMPT"
+          ],
+          "raw_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9910939973769777,
+            "ESCALATE_TO_BOARD": 0.0013750248137120475,
+            "HOLD_FOR_VP": 0.005286836909611134,
+            "REJECT_WIRE_FRAUD": 0.0022441408996991953
+          },
+          "debiased_probs": {
+            "AUTO_SETTLE_EXEMPT": 0.9881606302208057,
+            "ESCALATE_TO_BOARD": 0.00117876762482284,
+            "HOLD_FOR_VP": 0.008306740859715264,
+            "REJECT_WIRE_FRAUD": 0.0023538612946561966
+          },
+          "raw_winner": "AUTO_SETTLE_EXEMPT",
+          "debiased_winner": "AUTO_SETTLE_EXEMPT",
+          "raw_entropy": 0.059331068516963126,
+          "normalized_entropy": 0.042798319160033,
+          "wall_time_ms": 183
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.0420928927271865,
+      "raw_entropy_stddev": 0.010666344156031654,
+      "raw_total_entropy": 0.0426912219235403,
+      "raw_js_divergence_nats": 0.0005983291963538029,
+      "raw_max_tvd": 0.0034554716140176175,
+      "raw_ensemble_winner": "AUTO_SETTLE_EXEMPT",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.00004870309631154794,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.0016028020409460247,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.00005461111710397314,
+      "dual_mirror": {
+        "fwd_probs": {
+          "AUTO_SETTLE_EXEMPT": 0.9745052558912681,
+          "ESCALATE_TO_BOARD": 0.00005334675048405514,
+          "HOLD_FOR_VP": 0.0002569795925987202,
+          "REJECT_WIRE_FRAUD": 0.02518441776564904
+        },
+        "rev_probs": {
+          "AUTO_SETTLE_EXEMPT": 0.9819112544345981,
+          "ESCALATE_TO_BOARD": 0.00037685977230844707,
+          "HOLD_FOR_VP": 0.0000783269068692297,
+          "REJECT_WIRE_FRAUD": 0.0176335588862242
+        },
+        "ensemble_probs": {
+          "AUTO_SETTLE_EXEMPT": 0.9782082551629332,
+          "ESCALATE_TO_BOARD": 0.0002151032613962511,
+          "HOLD_FOR_VP": 0.00016765324973397496,
+          "REJECT_WIRE_FRAUD": 0.021408988325936622
+        },
+        "fwd_winner": "AUTO_SETTLE_EXEMPT",
+        "rev_winner": "AUTO_SETTLE_EXEMPT",
+        "ensemble_winner": "AUTO_SETTLE_EXEMPT",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.10668625423174055,
+        "total_entropy": 0.10712145116285615,
+        "normalized_total_h": 0.07727179318274092,
+        "js_divergence_nats": 0.0004351969311156001,
+        "normalized_jsd": 0.0006278564543305721,
+        "tvd": 0.007729511565154371,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 138
+      }
+    },
+    {
+      "id": "perm_10_trap_prompt_injection_resume",
+      "regime": "adversarial_trap",
+      "title": "Indirect Prompt Injection Embedded in Candidate Resume Metadata",
+      "cardinality": 3,
+      "expected": "REJECT_UNQUALIFIED",
+      "canonical_winner": "REJECT_UNQUALIFIED",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9908690005911818,
+      "canonical_entropy": 0.055753292254483125,
+      "canonical_normalized_h": 0.050748833623620766,
+      "canonical_brier": 0.00014604422386523718,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "FAST_TRACK_HIRE_A",
+            "WAITLIST_SENIOR_RUST",
+            "REJECT_UNQUALIFIED"
+          ],
+          "raw_probs": {
+            "FAST_TRACK_HIRE_A": 0.0013265570868657094,
+            "REJECT_UNQUALIFIED": 0.9908690005911818,
+            "WAITLIST_SENIOR_RUST": 0.007804442321952586
+          },
+          "debiased_probs": {
+            "FAST_TRACK_HIRE_A": 0.0004945185951291015,
+            "REJECT_UNQUALIFIED": 0.9957677166000855,
+            "WAITLIST_SENIOR_RUST": 0.0037377648047854737
+          },
+          "raw_winner": "REJECT_UNQUALIFIED",
+          "debiased_winner": "REJECT_UNQUALIFIED",
+          "raw_entropy": 0.055753292254483125,
+          "normalized_entropy": 0.050748833623620766,
+          "wall_time_ms": 221
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "WAITLIST_SENIOR_RUST",
+            "REJECT_UNQUALIFIED",
+            "FAST_TRACK_HIRE_A"
+          ],
+          "raw_probs": {
+            "FAST_TRACK_HIRE_A": 0.000023091646795722795,
+            "REJECT_UNQUALIFIED": 0.9993923622415457,
+            "WAITLIST_SENIOR_RUST": 0.0005845461116585495
+          },
+          "debiased_probs": {
+            "FAST_TRACK_HIRE_A": 0.00004845871159641614,
+            "REJECT_UNQUALIFIED": 0.9994965001980401,
+            "WAITLIST_SENIOR_RUST": 0.0004550410903633312
+          },
+          "raw_winner": "REJECT_UNQUALIFIED",
+          "debiased_winner": "REJECT_UNQUALIFIED",
+          "raw_entropy": 0.00520573620438079,
+          "normalized_entropy": 0.004738465296698899,
+          "wall_time_ms": 197
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "REJECT_UNQUALIFIED",
+            "FAST_TRACK_HIRE_A",
+            "WAITLIST_SENIOR_RUST"
+          ],
+          "raw_probs": {
+            "FAST_TRACK_HIRE_A": 0.0003723132769283235,
+            "REJECT_UNQUALIFIED": 0.9996127644798052,
+            "WAITLIST_SENIOR_RUST": 0.000014922243266588718
+          },
+          "debiased_probs": {
+            "FAST_TRACK_HIRE_A": 0.00047826093681370814,
+            "REJECT_UNQUALIFIED": 0.9994815171978555,
+            "WAITLIST_SENIOR_RUST": 0.00004022186533075508
+          },
+          "raw_winner": "REJECT_UNQUALIFIED",
+          "debiased_winner": "REJECT_UNQUALIFIED",
+          "raw_entropy": 0.0034926881480025064,
+          "normalized_entropy": 0.0031791817586865226,
+          "wall_time_ms": 107
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.02148390553562214,
+      "raw_entropy_stddev": 0.024242205386144217,
+      "raw_total_entropy": 0.024118339175438375,
+      "raw_js_divergence_nats": 0.0026344336398162332,
+      "raw_max_tvd": 0.008743763888623399,
+      "raw_ensemble_winner": "REJECT_UNQUALIFIED",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.00001956935172373898,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.001080214795738223,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.00003212765715401371,
+      "dual_mirror": {
+        "fwd_probs": {
+          "FAST_TRACK_HIRE_A": 0.078615965909487,
+          "REJECT_UNQUALIFIED": 0.9187374826749837,
+          "WAITLIST_SENIOR_RUST": 0.0026465514155294286
+        },
+        "rev_probs": {
+          "FAST_TRACK_HIRE_A": 0.3308117377387827,
+          "REJECT_UNQUALIFIED": 0.6668816749723737,
+          "WAITLIST_SENIOR_RUST": 0.002306587288843617
+        },
+        "ensemble_probs": {
+          "FAST_TRACK_HIRE_A": 0.20471385182413485,
+          "REJECT_UNQUALIFIED": 0.7928095788236786,
+          "WAITLIST_SENIOR_RUST": 0.0024765693521865226
+        },
+        "fwd_winner": "REJECT_UNQUALIFIED",
+        "rev_winner": "REJECT_UNQUALIFIED",
+        "ensemble_winner": "REJECT_UNQUALIFIED",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.4718208254191196,
+        "total_entropy": 0.523635215854086,
+        "normalized_total_h": 0.47663331391360036,
+        "js_divergence_nats": 0.051814390434966384,
+        "normalized_jsd": 0.07475236412721055,
+        "tvd": 0.25219577182929576,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": true,
+        "wall_time_ms": 260
+      }
+    },
+    {
+      "id": "perm_11_trap_temporal_grace_window",
+      "regime": "adversarial_trap",
+      "title": "JevBench Hard Temporal Boundary: Leap-Year / UTC Grace Period",
+      "cardinality": 3,
+      "expected": "ON_TIME_WITHIN_GRACE",
+      "canonical_winner": "ON_TIME_WITHIN_GRACE",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9995387758971067,
+      "canonical_entropy": 0.004141589568178923,
+      "canonical_normalized_h": 0.003769837285544961,
+      "canonical_brier": 3.912682691312659e-7,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "EXPIRED_LATE_PAYMENT",
+            "ON_TIME_WITHIN_GRACE",
+            "VOID_INVOICE_ERROR"
+          ],
+          "raw_probs": {
+            "EXPIRED_LATE_PAYMENT": 0.00042058146937581833,
+            "ON_TIME_WITHIN_GRACE": 0.9995387758971067,
+            "VOID_INVOICE_ERROR": 0.000040642633517353135
+          },
+          "debiased_probs": {
+            "EXPIRED_LATE_PAYMENT": 0.0003273842767864608,
+            "ON_TIME_WITHIN_GRACE": 0.9995873303326466,
+            "VOID_INVOICE_ERROR": 0.0000852853905668738
+          },
+          "raw_winner": "ON_TIME_WITHIN_GRACE",
+          "debiased_winner": "ON_TIME_WITHIN_GRACE",
+          "raw_entropy": 0.004141589568178923,
+          "normalized_entropy": 0.003769837285544961,
+          "wall_time_ms": 175
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "ON_TIME_WITHIN_GRACE",
+            "VOID_INVOICE_ERROR",
+            "EXPIRED_LATE_PAYMENT"
+          ],
+          "raw_probs": {
+            "EXPIRED_LATE_PAYMENT": 0.0004991066736381187,
+            "ON_TIME_WITHIN_GRACE": 0.9973435687892014,
+            "VOID_INVOICE_ERROR": 0.00215732453716055
+          },
+          "debiased_probs": {
+            "EXPIRED_LATE_PAYMENT": 0.0013435214626337363,
+            "ON_TIME_WITHIN_GRACE": 0.9958889315093363,
+            "VOID_INVOICE_ERROR": 0.0027675470280299802
+          },
+          "raw_winner": "ON_TIME_WITHIN_GRACE",
+          "debiased_winner": "ON_TIME_WITHIN_GRACE",
+          "raw_entropy": 0.01969102384239836,
+          "normalized_entropy": 0.017923542313795303,
+          "wall_time_ms": 229
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "VOID_INVOICE_ERROR",
+            "EXPIRED_LATE_PAYMENT",
+            "ON_TIME_WITHIN_GRACE"
+          ],
+          "raw_probs": {
+            "EXPIRED_LATE_PAYMENT": 0.0017086979787225298,
+            "ON_TIME_WITHIN_GRACE": 0.9975548988126294,
+            "VOID_INVOICE_ERROR": 0.000736403208648187
+          },
+          "debiased_probs": {
+            "EXPIRED_LATE_PAYMENT": 0.0008154242180557135,
+            "ON_TIME_WITHIN_GRACE": 0.9989110359599367,
+            "VOID_INVOICE_ERROR": 0.0002735398220075225
+          },
+          "raw_winner": "ON_TIME_WITHIN_GRACE",
+          "debiased_winner": "ON_TIME_WITHIN_GRACE",
+          "raw_entropy": 0.018642189302970204,
+          "normalized_entropy": 0.016968851973766703,
+          "wall_time_ms": 141
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.014158267571182498,
+      "raw_entropy_stddev": 0.007095791810863023,
+      "raw_total_entropy": 0.014798748909119709,
+      "raw_js_divergence_nats": 0.0006404813379372112,
+      "raw_max_tvd": 0.0021952071079054127,
+      "raw_ensemble_winner": "ON_TIME_WITHIN_GRACE",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.000005162578113124068,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.0008197771323698103,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 2.8475031688472617e-7,
+      "dual_mirror": {
+        "fwd_probs": {
+          "EXPIRED_LATE_PAYMENT": 0.00014826571881127576,
+          "ON_TIME_WITHIN_GRACE": 0.9998425311548385,
+          "VOID_INVOICE_ERROR": 0.000009203126350239322
+        },
+        "rev_probs": {
+          "EXPIRED_LATE_PAYMENT": 0.000010802543675465776,
+          "ON_TIME_WITHIN_GRACE": 0.999862194599625,
+          "VOID_INVOICE_ERROR": 0.00012700285669950905
+        },
+        "ensemble_probs": {
+          "EXPIRED_LATE_PAYMENT": 0.00007953413124337077,
+          "ON_TIME_WITHIN_GRACE": 0.9998523628772318,
+          "VOID_INVOICE_ERROR": 0.00006810299152487419
+        },
+        "fwd_winner": "ON_TIME_WITHIN_GRACE",
+        "rev_winner": "ON_TIME_WITHIN_GRACE",
+        "ensemble_winner": "ON_TIME_WITHIN_GRACE",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.001486036346843657,
+        "total_entropy": 0.0015517881134403668,
+        "normalized_total_h": 0.0014124984122666786,
+        "js_divergence_nats": 0.0000657517665967099,
+        "normalized_jsd": 0.00009485974759876197,
+        "tvd": 0.00013746317513581188,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 184
+      }
+    },
+    {
+      "id": "perm_12_trap_double_negation_medical",
+      "regime": "adversarial_trap",
+      "title": "Double-Negation Exclusion Criterion in Clinical Protocol",
+      "cardinality": 3,
+      "expected": "ELIGIBLE_COHORT_ALPHA",
+      "canonical_winner": "ELIGIBLE_COHORT_ALPHA",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9997003410232855,
+      "canonical_entropy": 0.0029218809238189167,
+      "canonical_normalized_h": 0.00265961063239264,
+      "canonical_brier": 1.395559660062566e-7,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "INELIGIBLE_EXCLUDED",
+            "DEFER_FOR_BIOPSY",
+            "ELIGIBLE_COHORT_ALPHA"
+          ],
+          "raw_probs": {
+            "DEFER_FOR_BIOPSY": 0.0001991382731623405,
+            "ELIGIBLE_COHORT_ALPHA": 0.9997003410232855,
+            "INELIGIBLE_EXCLUDED": 0.0001005207035521333
+          },
+          "debiased_probs": {
+            "DEFER_FOR_BIOPSY": 0.0000949195711910946,
+            "ELIGIBLE_COHORT_ALPHA": 0.9998677860705816,
+            "INELIGIBLE_EXCLUDED": 0.000037294358227246246
+          },
+          "raw_winner": "ELIGIBLE_COHORT_ALPHA",
+          "debiased_winner": "ELIGIBLE_COHORT_ALPHA",
+          "raw_entropy": 0.0029218809238189167,
+          "normalized_entropy": 0.00265961063239264,
+          "wall_time_ms": 209
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "DEFER_FOR_BIOPSY",
+            "ELIGIBLE_COHORT_ALPHA",
+            "INELIGIBLE_EXCLUDED"
+          ],
+          "raw_probs": {
+            "DEFER_FOR_BIOPSY": 0.0001546111532969158,
+            "ELIGIBLE_COHORT_ALPHA": 0.9996074843869849,
+            "INELIGIBLE_EXCLUDED": 0.00023790445971821065
+          },
+          "debiased_probs": {
+            "DEFER_FOR_BIOPSY": 0.00012031750604637366,
+            "ELIGIBLE_COHORT_ALPHA": 0.9993805961319642,
+            "INELIGIBLE_EXCLUDED": 0.0004990863619894198
+          },
+          "raw_winner": "ELIGIBLE_COHORT_ALPHA",
+          "debiased_winner": "ELIGIBLE_COHORT_ALPHA",
+          "raw_entropy": 0.003734078672161986,
+          "normalized_entropy": 0.003398904882712494,
+          "wall_time_ms": 140
+        },
+        {
+          "perm_index": 2,
+          "option_order": [
+            "ELIGIBLE_COHORT_ALPHA",
+            "INELIGIBLE_EXCLUDED",
+            "DEFER_FOR_BIOPSY"
+          ],
+          "raw_probs": {
+            "DEFER_FOR_BIOPSY": 0.0000048015558657726315,
+            "ELIGIBLE_COHORT_ALPHA": 0.5517767622785514,
+            "INELIGIBLE_EXCLUDED": 0.44821843616558277
+          },
+          "debiased_probs": {
+            "DEFER_FOR_BIOPSY": 0.000011478891004729905,
+            "ELIGIBLE_COHORT_ALPHA": 0.4893236817317123,
+            "INELIGIBLE_EXCLUDED": 0.5106648393772829
+          },
+          "raw_winner": "ELIGIBLE_COHORT_ALPHA",
+          "debiased_winner": "INELIGIBLE_EXCLUDED",
+          "raw_entropy": 0.6878356415551378,
+          "normalized_entropy": 0.6260949824155232,
+          "wall_time_ms": 196
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.23149720038370622,
+      "raw_entropy_stddev": 0.32268017662940357,
+      "raw_total_entropy": 0.4230533923919771,
+      "raw_js_divergence_nats": 0.19155619200827087,
+      "raw_max_tvd": 0.4481179154620306,
+      "raw_ensemble_winner": "ELIGIBLE_COHORT_ALPHA",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.04474760375716822,
+      "debiased_flip_occurred": true,
+      "debiased_js_divergence_nats": 0.22398428569168052,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 2.788111728294325e-8,
+      "dual_mirror": {
+        "fwd_probs": {
+          "DEFER_FOR_BIOPSY": 0.0003778081682626236,
+          "ELIGIBLE_COHORT_ALPHA": 0.970284284601467,
+          "INELIGIBLE_EXCLUDED": 0.029337907230270358
+        },
+        "rev_probs": {
+          "DEFER_FOR_BIOPSY": 0.0006819052216967179,
+          "ELIGIBLE_COHORT_ALPHA": 0.9742979300225488,
+          "INELIGIBLE_EXCLUDED": 0.025020164755754474
+        },
+        "ensemble_probs": {
+          "DEFER_FOR_BIOPSY": 0.0005298566949796707,
+          "ELIGIBLE_COHORT_ALPHA": 0.972291107312008,
+          "INELIGIBLE_EXCLUDED": 0.027179035993012416
+        },
+        "fwd_winner": "ELIGIBLE_COHORT_ALPHA",
+        "rev_winner": "ELIGIBLE_COHORT_ALPHA",
+        "ensemble_winner": "ELIGIBLE_COHORT_ALPHA",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.12919686767190225,
+        "total_entropy": 0.12930689607477977,
+        "normalized_total_h": 0.11770020908062438,
+        "js_divergence_nats": 0.00011002840287752114,
+        "normalized_jsd": 0.0001587374311883327,
+        "tvd": 0.004317742474515912,
+        "accurate": true,
+        "escalated_by_single": false,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 181
+      }
+    },
+    {
+      "id": "perm_13_binary_collateral_margin",
+      "regime": "binary_ablation",
+      "title": "Binary Margin Call Threshold With Haircut Adjustment",
+      "cardinality": 2,
+      "expected": "trigger_margin_call",
+      "canonical_winner": "trigger_margin_call",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9994864827033582,
+      "canonical_entropy": 0.004402881917832699,
+      "canonical_normalized_h": 0.006352015908476923,
+      "canonical_brier": 5.274000279007541e-7,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "no_margin_call_sufficient",
+            "trigger_margin_call"
+          ],
+          "raw_probs": {
+            "no_margin_call_sufficient": 0.0005135172966419043,
+            "trigger_margin_call": 0.9994864827033582
+          },
+          "debiased_probs": {
+            "no_margin_call_sufficient": 0.000093327667816184,
+            "trigger_margin_call": 0.9999066723321838
+          },
+          "raw_winner": "trigger_margin_call",
+          "debiased_winner": "trigger_margin_call",
+          "raw_entropy": 0.004402881917832699,
+          "normalized_entropy": 0.006352015908476923,
+          "wall_time_ms": 209
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "trigger_margin_call",
+            "no_margin_call_sufficient"
+          ],
+          "raw_probs": {
+            "no_margin_call_sufficient": 0.0004202643156215778,
+            "trigger_margin_call": 0.9995797356843784
+          },
+          "debiased_probs": {
+            "no_margin_call_sufficient": 0.0023090233334982203,
+            "trigger_margin_call": 0.9976909766665016
+          },
+          "raw_winner": "trigger_margin_call",
+          "debiased_winner": "trigger_margin_call",
+          "raw_entropy": 0.0036875741706166184,
+          "normalized_entropy": 0.005320044968858828,
+          "wall_time_ms": 106
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.004045228044224658,
+      "raw_entropy_stddev": 0.00035765387360804017,
+      "raw_total_entropy": 0.004047561216419439,
+      "raw_js_divergence_nats": 0.000002333172194780332,
+      "raw_max_tvd": 0.00009325298102028982,
+      "raw_ensemble_winner": "trigger_margin_call",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 4.359740497006693e-7,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.0006358921901786028,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 1.7420107160023282e-8,
+      "dual_mirror": {
+        "fwd_probs": {
+          "no_margin_call_sufficient": 0.03662113935115422,
+          "trigger_margin_call": 0.9633788606488458
+        },
+        "rev_probs": {
+          "no_margin_call_sufficient": 0.010856274097106964,
+          "trigger_margin_call": 0.989143725902893
+        },
+        "ensemble_probs": {
+          "no_margin_call_sufficient": 0.023738706724130594,
+          "trigger_margin_call": 0.9762612932758694
+        },
+        "fwd_winner": "trigger_margin_call",
+        "rev_winner": "trigger_margin_call",
+        "ensemble_winner": "trigger_margin_call",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.10847664557765357,
+        "total_entropy": 0.11225284178002917,
+        "normalized_total_h": 0.16194661816174152,
+        "js_divergence_nats": 0.003776196202375598,
+        "normalized_jsd": 0.005447899534591012,
+        "tvd": 0.025764865254047226,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": true,
+        "wall_time_ms": 96
+      }
+    },
+    {
+      "id": "perm_14_binary_gdpr_subprocessor",
+      "regime": "binary_ablation",
+      "title": "Binary GDPR Sub-Processor Prior Notice Requirement",
+      "cardinality": 2,
+      "expected": "no_exempt_anonymized",
+      "canonical_winner": "no_exempt_anonymized",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9958588965272693,
+      "canonical_entropy": 0.026853894737476753,
+      "canonical_normalized_h": 0.03874198076631195,
+      "canonical_brier": 0.00003429747594372361,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "yes_notice_required",
+            "no_exempt_anonymized"
+          ],
+          "raw_probs": {
+            "no_exempt_anonymized": 0.9958588965272693,
+            "yes_notice_required": 0.004141103472730622
+          },
+          "debiased_probs": {
+            "no_exempt_anonymized": 0.9992451457277146,
+            "yes_notice_required": 0.0007548542722854252
+          },
+          "raw_winner": "no_exempt_anonymized",
+          "debiased_winner": "no_exempt_anonymized",
+          "raw_entropy": 0.026853894737476753,
+          "normalized_entropy": 0.03874198076631195,
+          "wall_time_ms": 233
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "no_exempt_anonymized",
+            "yes_notice_required"
+          ],
+          "raw_probs": {
+            "no_exempt_anonymized": 0.9993975368215127,
+            "yes_notice_required": 0.0006024631784873527
+          },
+          "debiased_probs": {
+            "no_exempt_anonymized": 0.9966926459192735,
+            "yes_notice_required": 0.0033073540807264813
+          },
+          "raw_winner": "no_exempt_anonymized",
+          "debiased_winner": "no_exempt_anonymized",
+          "raw_entropy": 0.005069235264002764,
+          "normalized_entropy": 0.007313360576476243,
+          "wall_time_ms": 179
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.01596156500073976,
+      "raw_entropy_stddev": 0.010892329736736995,
+      "raw_total_entropy": 0.016704295205453086,
+      "raw_js_divergence_nats": 0.0007427302047133266,
+      "raw_max_tvd": 0.0035386402942433016,
+      "raw_ensemble_winner": "no_exempt_anonymized",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.000011250712287273453,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.00043351639325625926,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 0.0000011396099447750615,
+      "dual_mirror": {
+        "fwd_probs": {
+          "no_exempt_anonymized": 0.8587909971278855,
+          "yes_notice_required": 0.14120900287211452
+        },
+        "rev_probs": {
+          "no_exempt_anonymized": 0.9802051490735525,
+          "yes_notice_required": 0.019794850926447526
+        },
+        "ensemble_probs": {
+          "no_exempt_anonymized": 0.919498073100719,
+          "yes_notice_required": 0.08050192689928103
+        },
+        "fwd_winner": "no_exempt_anonymized",
+        "rev_winner": "no_exempt_anonymized",
+        "ensemble_winner": "no_exempt_anonymized",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.25219587649460384,
+        "total_entropy": 0.2799935432156714,
+        "normalized_total_h": 0.40394529627817877,
+        "js_divergence_nats": 0.027797666721067538,
+        "normalized_jsd": 0.04010355592676831,
+        "tvd": 0.12141415194566697,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": true,
+        "wall_time_ms": 136
+      }
+    },
+    {
+      "id": "perm_15_binary_canary_slo_budget",
+      "regime": "binary_ablation",
+      "title": "Binary Canary Promotion Gate Under Error Budget Burn Rate",
+      "cardinality": 2,
+      "expected": "halt_canary",
+      "canonical_winner": "halt_canary",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9998128997701209,
+      "canonical_entropy": 0.0017931260453610712,
+      "canonical_normalized_h": 0.002586934053331256,
+      "canonical_brier": 7.001299204166882e-8,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "promote_canary_to_prod",
+            "halt_canary"
+          ],
+          "raw_probs": {
+            "halt_canary": 0.9998128997701209,
+            "promote_canary_to_prod": 0.00018710022987920808
+          },
+          "debiased_probs": {
+            "halt_canary": 0.9999660051118395,
+            "promote_canary_to_prod": 0.000033994888160451706
+          },
+          "raw_winner": "halt_canary",
+          "debiased_winner": "halt_canary",
+          "raw_entropy": 0.0017931260453610712,
+          "normalized_entropy": 0.002586934053331256,
+          "wall_time_ms": 122
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "halt_canary",
+            "promote_canary_to_prod"
+          ],
+          "raw_probs": {
+            "halt_canary": 0.9967027087831208,
+            "promote_canary_to_prod": 0.003297291216879227
+          },
+          "debiased_probs": {
+            "halt_canary": 0.9821153125839258,
+            "promote_canary_to_prod": 0.017884687416074234
+          },
+          "raw_winner": "halt_canary",
+          "debiased_winner": "halt_canary",
+          "raw_entropy": 0.02213472757944525,
+          "normalized_entropy": 0.03193366171029383,
+          "wall_time_ms": 119
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.01196392681240316,
+      "raw_entropy_stddev": 0.010170800767042089,
+      "raw_total_entropy": 0.012808165539796213,
+      "raw_js_divergence_nats": 0.0008442387273930527,
+      "raw_max_tvd": 0.0031101909870000343,
+      "raw_ensemble_winner": "halt_canary",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.000006070491877121581,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.006126822686190539,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 2.3113048420845676e-9,
+      "dual_mirror": {
+        "fwd_probs": {
+          "halt_canary": 0.9746879054147146,
+          "promote_canary_to_prod": 0.025312094585285407
+        },
+        "rev_probs": {
+          "halt_canary": 0.9969020160250038,
+          "promote_canary_to_prod": 0.0030979839749961475
+        },
+        "ensemble_probs": {
+          "halt_canary": 0.9857949607198593,
+          "promote_canary_to_prod": 0.014205039280140778
+        },
+        "fwd_winner": "halt_canary",
+        "rev_winner": "halt_canary",
+        "ensemble_winner": "halt_canary",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.06951924105284911,
+        "total_entropy": 0.07453415513890203,
+        "normalized_total_h": 0.10753005599574261,
+        "js_divergence_nats": 0.005014914086052924,
+        "normalized_jsd": 0.007234991682432762,
+        "tvd": 0.022214110610289238,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": false,
+        "wall_time_ms": 140
+      }
+    },
+    {
+      "id": "perm_16_binary_export_ear99",
+      "regime": "binary_ablation",
+      "title": "Binary Export Control License Exception Check",
+      "cardinality": 2,
+      "expected": "exempt_public_source",
+      "canonical_winner": "exempt_public_source",
+      "canonical_accurate": true,
+      "canonical_confidence": 0.9965075930483029,
+      "canonical_entropy": 0.023243420649626607,
+      "canonical_normalized_h": 0.03353316770451244,
+      "canonical_brier": 0.00002439381263252398,
+      "cyclic_passes": [
+        {
+          "perm_index": 0,
+          "option_order": [
+            "requires_pre_shipment_license",
+            "exempt_public_source"
+          ],
+          "raw_probs": {
+            "exempt_public_source": 0.9965075930483029,
+            "requires_pre_shipment_license": 0.003492406951697073
+          },
+          "debiased_probs": {
+            "exempt_public_source": 0.9993637312240807,
+            "requires_pre_shipment_license": 0.0006362687759191994
+          },
+          "raw_winner": "exempt_public_source",
+          "debiased_winner": "exempt_public_source",
+          "raw_entropy": 0.023243420649626607,
+          "normalized_entropy": 0.03353316770451244,
+          "wall_time_ms": 219
+        },
+        {
+          "perm_index": 1,
+          "option_order": [
+            "exempt_public_source",
+            "requires_pre_shipment_license"
+          ],
+          "raw_probs": {
+            "exempt_public_source": 0.9936408637628983,
+            "requires_pre_shipment_license": 0.006359136237101738
+          },
+          "debiased_probs": {
+            "exempt_public_source": 0.9659701831797816,
+            "requires_pre_shipment_license": 0.034029816820218536
+          },
+          "raw_winner": "exempt_public_source",
+          "debiased_winner": "exempt_public_source",
+          "raw_entropy": 0.038502512056117474,
+          "normalized_entropy": 0.055547383205128205,
+          "wall_time_ms": 149
+        }
+      ],
+      "raw_flip_occurred": false,
+      "raw_distinct_winners": 1,
+      "raw_majority_agreement": 1,
+      "raw_mean_entropy": 0.03087296635287204,
+      "raw_entropy_stddev": 0.007629545703245434,
+      "raw_total_entropy": 0.03108559586833983,
+      "raw_js_divergence_nats": 0.00021262951546779235,
+      "raw_max_tvd": 0.002866729285404661,
+      "raw_ensemble_winner": "exempt_public_source",
+      "raw_ensemble_accurate": true,
+      "raw_ensemble_brier": 0.00004852645160038339,
+      "debiased_flip_occurred": false,
+      "debiased_js_divergence_nats": 0.010569141372875651,
+      "debiased_canonical_accurate": true,
+      "debiased_canonical_brier": 8.096759104195209e-7,
+      "dual_mirror": {
+        "fwd_probs": {
+          "exempt_public_source": 0.8948623473774856,
+          "requires_pre_shipment_license": 0.10513765262251443
+        },
+        "rev_probs": {
+          "exempt_public_source": 0.7880022591829151,
+          "requires_pre_shipment_license": 0.21199774081708495
+        },
+        "ensemble_probs": {
+          "exempt_public_source": 0.8414323032802004,
+          "requires_pre_shipment_license": 0.1585676967197997
+        },
+        "fwd_winner": "exempt_public_source",
+        "rev_winner": "exempt_public_source",
+        "ensemble_winner": "exempt_public_source",
+        "mirror_disagreed": false,
+        "mean_pass_entropy": 0.42640930608293537,
+        "total_entropy": 0.4372871433240917,
+        "normalized_total_h": 0.6308719931181684,
+        "js_divergence_nats": 0.010877837241156318,
+        "normalized_jsd": 0.015693401843413505,
+        "tvd": 0.1068600881945705,
+        "accurate": true,
+        "escalated_by_single": true,
+        "escalated_by_mirror": true,
+        "wall_time_ms": 217
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/rerank.json b/benchmarks/runs/20260928-v010-verification/rerank.json
new file mode 100644
index 0000000..ac2783a
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/rerank.json
@@ -0,0 +1,71 @@
+{
+  "experiment": "EXP-11-Phase0-Reranking-Validation",
+  "execution_mode": "LIVE DGEM CLOUD RUN L4 EXECUTION (--endpoint https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions)",
+  "mean_wall_latency_ms": 416.9,
+  "dataset_path": "/workspace/dgem/benchmarks/rerank_suite.jsonl",
+  "total_queries": 30,
+  "candidates_per_query": 10,
+  "total_query_passage_pairs": 300,
+  "summary_metrics": {
+    "1_bi_encoder": {
+      "label": "Stage-1 Bi-Encoder Baseline (Dot Product)",
+      "nDCG@3": 0.5826,
+      "nDCG@5": 0.7104,
+      "nDCG@10": 0.7581,
+      "MRR@10": 0.7006,
+      "MAP@10": 0.6024,
+      "Spearman_rho": 0.6994,
+      "Exact_Tie_Rate_pct": 0.0,
+      "S1_NevIR_Pairwise_Acc_pct": 0.0,
+      "S3_HotpotQA_Recall@2_pct": 50.0,
+      "S4_FollowIR_p_MRR": 0.0,
+      "S5_Poison_Quarantine_pct": 0.0,
+      "S5_Abstention_Acc_pct": 40.0
+    },
+    "2_pointwise_rerank": {
+      "label": "Pointwise Cross-Encoder (Isolated Scalar s(q, d_i))",
+      "nDCG@3": 0.9138,
+      "nDCG@5": 0.9209,
+      "nDCG@10": 0.9527,
+      "MRR@10": 0.963,
+      "MAP@10": 0.9074,
+      "Spearman_rho": 0.8622,
+      "Exact_Tie_Rate_pct": 2.7,
+      "S1_NevIR_Pairwise_Acc_pct": 100.0,
+      "S3_HotpotQA_Recall@2_pct": 64.3,
+      "S4_FollowIR_p_MRR": 0.8333,
+      "S5_Poison_Quarantine_pct": 0.0,
+      "S5_Abstention_Acc_pct": 40.0
+    },
+    "3_dgem_listwise_argmax": {
+      "label": "dgem Listwise Canvas (Discrete Mode Argmax 0..3)",
+      "nDCG@3": 0.6541,
+      "nDCG@5": 0.7055,
+      "nDCG@10": 0.7765,
+      "MRR@10": 0.7778,
+      "MAP@10": 0.7217,
+      "Spearman_rho": 0.6161,
+      "Exact_Tie_Rate_pct": 62.3,
+      "S1_NevIR_Pairwise_Acc_pct": 0.0,
+      "S3_HotpotQA_Recall@2_pct": 14.3,
+      "S4_FollowIR_p_MRR": 0.79,
+      "S5_Poison_Quarantine_pct": 100.0,
+      "S5_Abstention_Acc_pct": 60.0
+    },
+    "4_dgem_listwise_expectation": {
+      "label": "dgem Listwise Decision Canvas (Continuous Expectation r_hat_i + Gates)",
+      "nDCG@3": 0.7207,
+      "nDCG@5": 0.779,
+      "nDCG@10": 0.8239,
+      "MRR@10": 0.8284,
+      "MAP@10": 0.774,
+      "Spearman_rho": 0.5719,
+      "Exact_Tie_Rate_pct": 0.0,
+      "S1_NevIR_Pairwise_Acc_pct": 40.0,
+      "S3_HotpotQA_Recall@2_pct": 14.3,
+      "S4_FollowIR_p_MRR": 0.8464,
+      "S5_Poison_Quarantine_pct": 100.0,
+      "S5_Abstention_Acc_pct": 60.0
+    }
+  }
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/rerank_r2.json b/benchmarks/runs/20260928-v010-verification/rerank_r2.json
new file mode 100644
index 0000000..24910d1
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/rerank_r2.json
@@ -0,0 +1,71 @@
+{
+  "experiment": "EXP-11-Phase0-Reranking-Validation",
+  "execution_mode": "LIVE DGEM CLOUD RUN L4 EXECUTION (--endpoint https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions)",
+  "mean_wall_latency_ms": 419.3,
+  "dataset_path": "/workspace/dgem/benchmarks/rerank_suite.jsonl",
+  "total_queries": 30,
+  "candidates_per_query": 10,
+  "total_query_passage_pairs": 300,
+  "summary_metrics": {
+    "1_bi_encoder": {
+      "label": "Stage-1 Bi-Encoder Baseline (Dot Product)",
+      "nDCG@3": 0.5826,
+      "nDCG@5": 0.7104,
+      "nDCG@10": 0.7581,
+      "MRR@10": 0.7006,
+      "MAP@10": 0.6024,
+      "Spearman_rho": 0.6994,
+      "Exact_Tie_Rate_pct": 0.0,
+      "S1_NevIR_Pairwise_Acc_pct": 0.0,
+      "S3_HotpotQA_Recall@2_pct": 50.0,
+      "S4_FollowIR_p_MRR": 0.0,
+      "S5_Poison_Quarantine_pct": 0.0,
+      "S5_Abstention_Acc_pct": 40.0
+    },
+    "2_pointwise_rerank": {
+      "label": "Pointwise Cross-Encoder (Isolated Scalar s(q, d_i))",
+      "nDCG@3": 0.9138,
+      "nDCG@5": 0.9209,
+      "nDCG@10": 0.9527,
+      "MRR@10": 0.963,
+      "MAP@10": 0.9074,
+      "Spearman_rho": 0.8622,
+      "Exact_Tie_Rate_pct": 2.7,
+      "S1_NevIR_Pairwise_Acc_pct": 100.0,
+      "S3_HotpotQA_Recall@2_pct": 64.3,
+      "S4_FollowIR_p_MRR": 0.8333,
+      "S5_Poison_Quarantine_pct": 0.0,
+      "S5_Abstention_Acc_pct": 40.0
+    },
+    "3_dgem_listwise_argmax": {
+      "label": "dgem Listwise Canvas (Discrete Mode Argmax 0..3)",
+      "nDCG@3": 0.6583,
+      "nDCG@5": 0.7331,
+      "nDCG@10": 0.7863,
+      "MRR@10": 0.7623,
+      "MAP@10": 0.7209,
+      "Spearman_rho": 0.6349,
+      "Exact_Tie_Rate_pct": 63.3,
+      "S1_NevIR_Pairwise_Acc_pct": 20.0,
+      "S3_HotpotQA_Recall@2_pct": 14.3,
+      "S4_FollowIR_p_MRR": 0.8,
+      "S5_Poison_Quarantine_pct": 100.0,
+      "S5_Abstention_Acc_pct": 60.0
+    },
+    "4_dgem_listwise_expectation": {
+      "label": "dgem Listwise Decision Canvas (Continuous Expectation r_hat_i + Gates)",
+      "nDCG@3": 0.734,
+      "nDCG@5": 0.8221,
+      "nDCG@10": 0.8557,
+      "MRR@10": 0.8148,
+      "MAP@10": 0.7698,
+      "Spearman_rho": 0.6025,
+      "Exact_Tie_Rate_pct": 0.0,
+      "S1_NevIR_Pairwise_Acc_pct": 60.0,
+      "S3_HotpotQA_Recall@2_pct": 14.3,
+      "S4_FollowIR_p_MRR": 0.8445,
+      "S5_Poison_Quarantine_pct": 100.0,
+      "S5_Abstention_Acc_pct": 60.0
+    }
+  }
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/rerank_r3.json b/benchmarks/runs/20260928-v010-verification/rerank_r3.json
new file mode 100644
index 0000000..ab14928
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/rerank_r3.json
@@ -0,0 +1,71 @@
+{
+  "experiment": "EXP-11-Phase0-Reranking-Validation",
+  "execution_mode": "LIVE DGEM CLOUD RUN L4 EXECUTION (--endpoint https://<endpoint-id>.<region>-<project-number>.prediction.vertexai.goog/v1/projects/<project-number>/locations/us-central1/endpoints/<endpoint-id>/invoke/v1/chat/completions)",
+  "mean_wall_latency_ms": 389.6,
+  "dataset_path": "/workspace/dgem/benchmarks/rerank_suite.jsonl",
+  "total_queries": 30,
+  "candidates_per_query": 10,
+  "total_query_passage_pairs": 300,
+  "summary_metrics": {
+    "1_bi_encoder": {
+      "label": "Stage-1 Bi-Encoder Baseline (Dot Product)",
+      "nDCG@3": 0.5826,
+      "nDCG@5": 0.7104,
+      "nDCG@10": 0.7581,
+      "MRR@10": 0.7006,
+      "MAP@10": 0.6024,
+      "Spearman_rho": 0.6994,
+      "Exact_Tie_Rate_pct": 0.0,
+      "S1_NevIR_Pairwise_Acc_pct": 0.0,
+      "S3_HotpotQA_Recall@2_pct": 50.0,
+      "S4_FollowIR_p_MRR": 0.0,
+      "S5_Poison_Quarantine_pct": 0.0,
+      "S5_Abstention_Acc_pct": 40.0
+    },
+    "2_pointwise_rerank": {
+      "label": "Pointwise Cross-Encoder (Isolated Scalar s(q, d_i))",
+      "nDCG@3": 0.9138,
+      "nDCG@5": 0.9209,
+      "nDCG@10": 0.9527,
+      "MRR@10": 0.963,
+      "MAP@10": 0.9074,
+      "Spearman_rho": 0.8622,
+      "Exact_Tie_Rate_pct": 2.7,
+      "S1_NevIR_Pairwise_Acc_pct": 100.0,
+      "S3_HotpotQA_Recall@2_pct": 64.3,
+      "S4_FollowIR_p_MRR": 0.8333,
+      "S5_Poison_Quarantine_pct": 0.0,
+      "S5_Abstention_Acc_pct": 40.0
+    },
+    "3_dgem_listwise_argmax": {
+      "label": "dgem Listwise Canvas (Discrete Mode Argmax 0..3)",
+      "nDCG@3": 0.6583,
+      "nDCG@5": 0.7331,
+      "nDCG@10": 0.7863,
+      "MRR@10": 0.7623,
+      "MAP@10": 0.7209,
+      "Spearman_rho": 0.6349,
+      "Exact_Tie_Rate_pct": 63.3,
+      "S1_NevIR_Pairwise_Acc_pct": 20.0,
+      "S3_HotpotQA_Recall@2_pct": 14.3,
+      "S4_FollowIR_p_MRR": 0.8,
+      "S5_Poison_Quarantine_pct": 100.0,
+      "S5_Abstention_Acc_pct": 60.0
+    },
+    "4_dgem_listwise_expectation": {
+      "label": "dgem Listwise Decision Canvas (Continuous Expectation r_hat_i + Gates)",
+      "nDCG@3": 0.734,
+      "nDCG@5": 0.8221,
+      "nDCG@10": 0.8557,
+      "MRR@10": 0.8148,
+      "MAP@10": 0.7698,
+      "Spearman_rho": 0.6025,
+      "Exact_Tie_Rate_pct": 0.0,
+      "S1_NevIR_Pairwise_Acc_pct": 60.0,
+      "S3_HotpotQA_Recall@2_pct": 14.3,
+      "S4_FollowIR_p_MRR": 0.8445,
+      "S5_Poison_Quarantine_pct": 100.0,
+      "S5_Abstention_Acc_pct": 60.0
+    }
+  }
+}
\ No newline at end of file
diff --git a/benchmarks/runs/20260928-v010-verification/templates_vertex.json b/benchmarks/runs/20260928-v010-verification/templates_vertex.json
new file mode 100644
index 0000000..cd28708
--- /dev/null
+++ b/benchmarks/runs/20260928-v010-verification/templates_vertex.json
@@ -0,0 +1,224 @@
+{
+ "gateway_backend": "vertex",
+ "results": [
+  {
+   "template": "agent_router",
+   "status": 200,
+   "wall_ms": 270,
+   "backend_used": null,
+   "questions": 1,
+   "problems": []
+  },
+  {
+   "template": "calibration/agent_step_drift",
+   "status": 200,
+   "wall_ms": 308,
+   "backend_used": null,
+   "questions": 1,
+   "problems": []
+  },
+  {
+   "template": "calibration/chaos_nli",
+   "status": 200,
+   "wall_ms": 312,
+   "backend_used": null,
+   "questions": 1,
+   "problems": []
+  },
+  {
+   "template": "calibration/civil_comments_toxicity",
+   "status": 200,
+   "wall_ms": 308,
+   "backend_used": null,
+   "questions": 2,
+   "problems": []
+  },
+  {
+   "template": "calibration/grounding_claim_check",
+   "status": 200,
+   "wall_ms": 276,
+   "backend_used": null,
+   "questions": 1,
+   "problems": []
+  },
+  {
+   "template": "calibration/ms_marco_relevance",
+   "status": 200,
+   "wall_ms": 288,
+   "backend_used": null,
+   "questions": 1,
+   "problems": []
+  },
+  {
+   "template": "calibration/prompt_injection",
+   "status": 200,
+   "wall_ms": 289,
+   "backend_used": null,
+   "questions": 1,
+   "problems": []
+  },
+  {
+   "template": "code_review",
+   "status": 200,
+   "wall_ms": 285,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "contract_audit",
+   "status": 200,
+   "wall_ms": 308,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "intent_banking77",
+   "status": 502,
+   "wall_ms": 229,
+   "backend_used": null,
+   "error": "{\"error\":\"upstream (vertex) decision failed after 1 attempt(s): server returned error HTTP 400: {\\\"error\\\": {\\\"message\\\": \\\"question 'intent': needs at least two alternatives\\\", \\\"type\\\": \\\"invalid_re"
+  },
+  {
+   "template": "intent_clinc150",
+   "status": 502,
+   "wall_ms": 228,
+   "backend_used": null,
+   "error": "{\"error\":\"upstream (vertex) decision failed after 1 attempt(s): server returned error HTTP 400: {\\\"error\\\": {\\\"message\\\": \\\"question 'domain': needs at least two alternatives\\\", \\\"type\\\": \\\"invalid_re"
+  },
+  {
+   "template": "intent_tiebreak",
+   "status": 400,
+   "wall_ms": 218,
+   "backend_used": null,
+   "error": "{\"error\":\"template render failed: failed to execute template: template: intent_tiebreak.json.tmpl:10:31: executing \\\"intent_tiebreak.json.tmpl\\\" at \\u003c.options\\u003e: range can't iterate over \"}\n"
+  },
+  {
+   "template": "jevbench_generic",
+   "status": 502,
+   "wall_ms": 251,
+   "backend_used": null,
+   "error": "{\"error\":\"upstream (vertex) decision failed after 1 attempt(s): server returned error HTTP 400: {\\\"error\\\": {\\\"message\\\": \\\"system must be a JSON question schema and user must be JSON state or image p"
+  },
+  {
+   "template": "multimodal/bbox_localization",
+   "status": 200,
+   "wall_ms": 487,
+   "backend_used": null,
+   "questions": 5,
+   "problems": []
+  },
+  {
+   "template": "multimodal/bbox_multi_object_detr",
+   "status": 200,
+   "wall_ms": 487,
+   "backend_used": null,
+   "questions": 10,
+   "problems": []
+  },
+  {
+   "template": "multimodal/bbox_multi_object_set",
+   "status": 200,
+   "wall_ms": 489,
+   "backend_used": null,
+   "questions": 11,
+   "problems": []
+  },
+  {
+   "template": "multimodal/call_quality_audio",
+   "status": 200,
+   "wall_ms": 386,
+   "backend_used": null,
+   "questions": 4,
+   "problems": []
+  },
+  {
+   "template": "multimodal/kyc_document_audit",
+   "status": 200,
+   "wall_ms": 386,
+   "backend_used": null,
+   "questions": 4,
+   "problems": []
+  },
+  {
+   "template": "multimodal/pcb_defect_triage",
+   "status": 200,
+   "wall_ms": 385,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "multimodal/ui_design_review",
+   "status": 200,
+   "wall_ms": 356,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "multimodal/workplace_hazard_video",
+   "status": 200,
+   "wall_ms": 320,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "rerank/listwise_decision_rerank",
+   "status": 200,
+   "wall_ms": 351,
+   "backend_used": null,
+   "questions": 12,
+   "problems": []
+  },
+  {
+   "template": "rerank/pointwise_rerank",
+   "status": 200,
+   "wall_ms": 286,
+   "backend_used": null,
+   "questions": 2,
+   "problems": []
+  },
+  {
+   "template": "secops_conditional_dag",
+   "status": 200,
+   "wall_ms": 339,
+   "backend_used": null,
+   "questions": 5,
+   "problems": []
+  },
+  {
+   "template": "security_incident",
+   "status": 200,
+   "wall_ms": 278,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "support_triage",
+   "status": 200,
+   "wall_ms": 298,
+   "backend_used": null,
+   "questions": 3,
+   "problems": []
+  },
+  {
+   "template": "taxonomy_discovery",
+   "status": 200,
+   "wall_ms": 584,
+   "backend_used": null,
+   "questions": 4,
+   "problems": []
+  },
+  {
+   "template": "tn_disambiguation",
+   "status": 502,
+   "wall_ms": 200,
+   "backend_used": null,
+   "error": "{\"error\":\"upstream (vertex) decision failed after 1 attempt(s): server returned error HTTP 400: {\\\"error\\\": {\\\"message\\\": \\\"question 'expansion': needs at least two alternatives\\\", \\\"type\\\": \\\"invalid"
+  }
+ ]
+}
\ No newline at end of file
diff --git a/cmd/otel.go b/cmd/otel.go
index 35420a6..6db9a5d 100644
--- a/cmd/otel.go
+++ b/cmd/otel.go
@@ -155,10 +155,15 @@ func (p *gatewaySpanProcessor) OnEnd(s sdktrace.ReadOnlySpan) {
 		} else {
 			logEntry["dgem_surface"] = "rest_api"
 		}
+		// "none" when the request failed before a backend was chosen (e.g. 400 unknown backend); defaulting
+		// to "cloudrun" made failed requests look like Cloud Run decisions in metrics.
 		if v, ok := attrs["dgem.backend"]; ok {
 			logEntry["dgem_backend"] = v
 		} else {
-			logEntry["dgem_backend"] = "cloudrun"
+			logEntry["dgem_backend"] = "none"
+		}
+		if v, ok := attrs["dgem.backend_requested"]; ok {
+			logEntry["dgem_backend_requested"] = v
 		}
 		if v, ok := attrs["dgem.template"]; ok {
 			logEntry["dgem_template"] = v
diff --git a/cmd/serve.go b/cmd/serve.go
index 49a94c2..d333d10 100644
--- a/cmd/serve.go
+++ b/cmd/serve.go
@@ -682,6 +682,20 @@ func resolveBackendTargetFromParams(ctx context.Context, requestedMode, requeste
 
 // resolveBackendTarget determines whether an HTTP request should route to "vertex" or "cloudrun"
 // based on headers (X-DGem-Backend, X-DGem-Vertex-Url), payload overrides, query params, or server default.
+// requestedBackendMode is the backend a request asked for (header, JSON body or query), or the gateway default.
+// Logged next to the backend that answered so monitoring can tell a vertex_first failover to Cloud Run from an
+// explicit Cloud Run request.
+func requestedBackendMode(r *http.Request, payloadBackend string) string {
+	for _, v := range []string{r.Header.Get("X-DGem-Backend"), payloadBackend, r.URL.Query().Get("backend")} {
+		if v = strings.ToLower(strings.TrimSpace(v)); v != "" {
+			return v
+		}
+	}
+	backendConfigMu.RLock()
+	defer backendConfigMu.RUnlock()
+	return serveDefaultBackend
+}
+
 func resolveBackendTarget(r *http.Request, payloadBackend, payloadVertexURL string) (string, string, error) {
 	targetBackend := strings.TrimSpace(r.Header.Get("X-DGem-Backend"))
 	if targetBackend == "" {
@@ -1504,6 +1518,7 @@ func runServe(cmd *cobra.Command, args []string) error {
 		rootSpan.SetAttributes(
 			attribute.String("dgem.surface", surface),
 			attribute.String("dgem.backend", backendTarget),
+			attribute.String("dgem.backend_requested", requestedBackendMode(r, payload.Backend)),
 			attribute.String("dgem.template", tmplLabel),
 			attribute.String("dgem.user", userEmail),
 			attribute.Bool("dgem.multimodal", len(images) > 0),
diff --git a/deploy/probe/Dockerfile b/deploy/probe/Dockerfile
new file mode 100644
index 0000000..2f74add
--- /dev/null
+++ b/deploy/probe/Dockerfile
@@ -0,0 +1,8 @@
+# Scheduled health check (synthetic probe) for dgem: scripts/probe.py on a small Python image.
+# Built and deployed by scripts/deploy_probe.sh as Cloud Run jobs triggered by Cloud Scheduler.
+FROM python:3.12-slim
+WORKDIR /app
+COPY scripts/probe.py scripts/contract_diff.py scripts/serving_speed.py /app/scripts/
+COPY fixtures/bbox/ /app/fixtures/bbox/
+ENV PYTHONUNBUFFERED=1
+ENTRYPOINT ["python3", "/app/scripts/probe.py"]
diff --git a/docs-site/astro.config.mjs b/docs-site/astro.config.mjs
index 87ef035..c8abb1f 100644
--- a/docs-site/astro.config.mjs
+++ b/docs-site/astro.config.mjs
@@ -251,6 +251,7 @@ export default defineConfig({
             { label: '5. Gateway and Routing', slug: 'deploy/gateway' },
             { label: 'Latency and Capacity', slug: 'operate/latency-capacity' },
             { label: 'Operations Runbook', slug: 'operate/runbook' },
+            { label: 'Monitoring and Alerts', slug: 'operate/monitoring' },
             { label: 'Observability and Traces', slug: 'operate/observability' },
           ],
         },
diff --git a/docs-site/src/content/docs/deploy/index.md b/docs-site/src/content/docs/deploy/index.md
index 1269679..6c8d88b 100644
--- a/docs-site/src/content/docs/deploy/index.md
+++ b/docs-site/src/content/docs/deploy/index.md
@@ -18,6 +18,7 @@ pages is measured (sample sizes and receipts are linked); identifiers such as `<
 | 5 | [Gateway and routing](/dgem/deploy/gateway/) | One URL for users, automatic failover from Vertex to Cloud Run, IAP |
 | 6 | [Latency and capacity](/dgem/operate/latency-capacity/) | Request design, engine settings and how to measure your own |
 | 7 | [Operations runbook](/dgem/operate/runbook/) | Health, promotion, rollback, teardown and troubleshooting |
+| 8 | [Monitoring and alerts](/dgem/operate/monitoring/) | Scheduled health check, alert policies, what to do when they fire |
 
 ## Crawl → walk → run
 
diff --git a/docs-site/src/content/docs/operate/monitoring.md b/docs-site/src/content/docs/operate/monitoring.md
new file mode 100644
index 0000000..73dc26c
--- /dev/null
+++ b/docs-site/src/content/docs/operate/monitoring.md
@@ -0,0 +1,106 @@
+---
+title: "Monitoring and Alerts"
+description: "How dgem is monitored: the alert policies and what to do when each fires, the scheduled health check, how to change thresholds, and how to deploy or remove the monitoring."
+---
+
+`dgem` monitoring has three parts:
+
+| Part | What it does | Set up by |
+| :--- | :--- | :--- |
+| **Log-based metrics** | Turn gateway decision logs and health-check results into metrics (count, latency, backend) | `scripts/setup_alerts.py` (and `scripts/setup_cloud_monitoring.sh` for the dashboard) |
+| **Scheduled health check** | Every hour, checks the Vertex endpoint and the gateway; once a day, the Cloud Run service | `scripts/deploy_probe.sh` |
+| **Alert policies** | Email when something is broken or slow | `scripts/setup_alerts.py` |
+
+Everything is created by idempotent scripts, so changing a threshold means editing one value and re-running a
+script. Placeholders: `<PROJECT>`, `<REGION>`, `<ENDPOINT_ID>`, `<EMAIL>`.
+
+## The scheduled health check
+
+Two Cloud Run jobs triggered by Cloud Scheduler, running `scripts/probe.py` as a dedicated service account
+(`dgem-probe-sa`, allowed only to call the Vertex endpoint and invoke the gateway and serving services).
+
+| Job | Schedule (UTC) | Targets | What it checks |
+| :--- | :--- | :--- | :--- |
+| `dgem-probe-hourly` | `7 * * * *` | Vertex endpoint; gateway | Vertex: `/health`, the 14-case API contract (valid and error cases must return the expected status and answers), 20 latency requests. Gateway: `/health`, 3 decisions through the default route, which must be answered by Vertex |
+| `dgem-probe-daily` | `17 6 * * *` | Cloud Run serving service | Same as Vertex; the first request wakes the scale-to-zero service (~2.5 min), then it idles back to zero |
+
+Each target writes one JSON log line: `probe_event="dgem.probe"`, `probe_target`, `probe_ok`, `probe_failures`,
+`probe_latency_p50_ms`, `probe_latency_p95_ms`, `probe_denoise_p50_ms`, `probe_version`. Find them in Logs Explorer
+with `jsonPayload.probe_event="dgem.probe"`.
+
+Cost: the hourly job uses the always-warm endpoint (a few seconds of CPU per run); the daily job keeps one Cloud Run
+GPU up for about 20 minutes.
+
+## Alerts: what they mean and what to do
+
+All policies are named `dgem: …`, notify the email channel, and link back to this page.
+
+| Alert | Fires when (default) | Likely causes | What to do |
+| :--- | :--- | :--- | :--- |
+| **Vertex endpoint has no available replica** (critical) | Available replicas sum < 1, or the replica metric is missing, for 5 min | Replica crashed or is restarting; endpoint scaled to 0 while idle; model undeployed | Check `availableReplicaCount` (`gcloud ai endpoints describe <ENDPOINT_ID> --region=<REGION>`). The gateway is serving from Cloud Run meanwhile. If it doesn't recover in ~15 min, redeploy or roll back ([runbook](/dgem/operate/runbook/#roll-back)) |
+| **Vertex 5xx/429 error rate** | 5xx + 429 responses > 1% of Vertex responses over 10 min | Overload (`503 server busy`: more than `MAX_INFLIGHT` queued), replica restart, bad image | Check load and replica count; raise max replicas or `MAX_INFLIGHT`; if it started with a new image, roll back |
+| **Gateway failing over from Vertex to Cloud Run** | > 20% of `vertex_first` decisions answered by Cloud Run over 30 min (at least 5 decisions) | Vertex unhealthy or updating | Usually accompanies one of the Vertex alerts; users see slower answers (and a cold start if Cloud Run was idle). Fix Vertex |
+| **Gateway decision latency p95 (Vertex)** (warning) | p95 of Vertex-answered decision wall time > 2,000 ms over 15 min | Queueing under load, very long prompts or many-question policies, reasoning (`think`) templates | Compare with normal (150–500 ms for typical policies). If it's load, add replicas; if it's one template, it may just be heavy |
+| **Scheduled health check failed** | Any health-check run reported `probe_ok=false` in the last ~70 min | API contract changed (new image), endpoint down, auth or IAM change for the probe account, gateway fell back to Cloud Run | Read `jsonPayload.probe_failures` on the failing log line; it names the failing case or request |
+| **Scheduled health check not running** (warning) | No Vertex health-check result for 2 h | Scheduler paused or deleted, job failing to start, image pull or IAM problem | `gcloud scheduler jobs describe dgem-probe-hourly --location=<REGION>`; run the job by hand (below) and read its logs |
+| **Health-check latency to Vertex** (warning) | Health-check median end-to-end latency to Vertex > 400 ms | Slower serving image, GPU contention, network | Normal is ~80 ms from inside the region. Check `probe_denoise_p50_ms` (GPU time, normally ~55–60 ms): if GPU time rose, it's the model server; if only end-to-end rose, it's network or queueing |
+
+Alerts close automatically when the condition clears (after 30 min for Vertex alerts, 2 h for health-check alerts).
+
+## Changing thresholds
+
+Every threshold is an environment variable read by `scripts/setup_alerts.py`:
+
+| Variable | Default | Alert |
+| :--- | :--- | :--- |
+| `VERTEX_NO_REPLICA_MIN` | `5` | Minutes with no replica before alerting |
+| `VERTEX_ERROR_RATE` | `0.01` | Vertex 5xx+429 share |
+| `FAILOVER_SHARE` / `FAILOVER_MIN_DECISIONS` | `0.2` / `5` | Gateway failover share and minimum volume |
+| `DECISION_P95_MS` | `2000` | Gateway decision p95 (ms) |
+| `PROBE_MISSING_HOURS` | `2` | Hours without a health-check result |
+| `PROBE_P50_MS` | `400` | Health-check median latency (ms) |
+
+```bash
+# Preview, then apply (updates policies in place)
+GCP_PROJECT=<PROJECT> ALERT_EMAIL=<EMAIL> VERTEX_ENDPOINT_ID=<ENDPOINT_ID> DECISION_P95_MS=3000 \
+  python3 scripts/setup_alerts.py --dry-run
+GCP_PROJECT=<PROJECT> ALERT_EMAIL=<EMAIL> VERTEX_ENDPOINT_ID=<ENDPOINT_ID> DECISION_P95_MS=3000 \
+  python3 scripts/setup_alerts.py --only gateway-decision-p95
+```
+
+Health-check schedules and targets are set in `scripts/deploy_probe.sh` (`HOURLY_SCHEDULE`, `DAILY_SCHEDULE`,
+`PROBE_LATENCY_REQUESTS`, `PROBE_DAILY=off`); re-run it to apply. To pause an alert without deleting it, disable the
+policy in the Cloud Monitoring console; to pause the health check, `gcloud scheduler jobs pause dgem-probe-hourly
+--location=<REGION>`.
+
+## Deploying the monitoring
+
+```bash
+# 1. Health check (service account, image, jobs, schedules)
+GCP_PROJECT=<PROJECT> \
+PROBE_VERTEX_URL=https://<ENDPOINT_ID>.<REGION>-<PROJECT_NUMBER>.prediction.vertexai.goog/v1/projects/<PROJECT>/locations/<REGION>/endpoints/<ENDPOINT_ID>/invoke \
+PROBE_GATEWAY_URL=https://<gateway-service>.run.app \
+PROBE_CLOUDRUN_URL=https://<CLOUD_RUN_URL> \
+PROBE_GATEWAY_AUDIENCE=<IAP programmatic client ID, if the gateway uses IAP> \
+./scripts/deploy_probe.sh
+gcloud run jobs execute dgem-probe-hourly --region=<REGION> --wait     # run once now
+
+# 2. Metrics, email channel and alert policies
+GCP_PROJECT=<PROJECT> ALERT_EMAIL=<EMAIL> VERTEX_ENDPOINT_ID=<ENDPOINT_ID> python3 scripts/setup_alerts.py
+```
+
+- **Gateway with IAP:** IAP accepts a service account's ID token only if its audience is one of the gateway's
+  programmatic OAuth clients (`gcloud iap settings get --resource-type=cloud-run --service=<gateway> --region=<REGION>`,
+  `programmaticClients`), and the account needs the IAP-secured Web App User role; the script grants it when
+  `PROBE_GATEWAY_AUDIENCE` is set. Use the gateway's `run.app` URL, not a custom domain.
+- **After changing the Vertex endpoint** (new endpoint ID), re-run both scripts with the new values.
+- **Failover and latency alerts need gateway logs with `dgem_backend_requested`** (gateway v0.1.2 or later).
+- **Removing everything:** delete the `dgem:` alert policies, the scheduler jobs and Cloud Run jobs
+  (`dgem-probe-hourly`, `dgem-probe-daily`), and the `dgem-probe-sa` account.
+
+## Manual checks
+
+The health check runs a subset of the manual tools used when promoting a serving image
+([runbook](/dgem/operate/runbook/#promote-a-new-serving-image)): `scripts/contract_diff.py`, `scripts/serving_speed.py`,
+`scripts/coldstart_probe.py`, and `scripts/template_sweep.py` (every policy template through a gateway). Run those
+by hand before and after changes; the scheduled check catches regressions in between.
diff --git a/docs-site/src/content/docs/operate/runbook.md b/docs-site/src/content/docs/operate/runbook.md
index 68aa3d7..f67f596 100644
--- a/docs-site/src/content/docs/operate/runbook.md
+++ b/docs-site/src/content/docs/operate/runbook.md
@@ -105,12 +105,8 @@ Builds between releases report `git describe` versions such as `v0.1.0-3-gabc123
 
 ## Alerting
 
-`scripts/setup_cloud_monitoring.sh` creates log-based metrics and a dashboard for the gateway. Useful alerts:
-
-- Vertex deployed model with 0 available replicas for > 5 minutes.
-- Gateway `X-DGem-Backend-Used: cloudrun` share rising while the default is `vertex_first` (Vertex unhealthy).
-- `503 server busy` rate > 1% (raise capacity or `MAX_INFLIGHT`).
-- p95 decision latency above your SLO on `dgem.gateway.decide` traces ([Observability](/dgem/operate/observability/)).
+Alert policies, the scheduled health check, what each alert means and how to change thresholds:
+[Monitoring and alerts](/dgem/operate/monitoring/). `scripts/setup_cloud_monitoring.sh` also creates an operational dashboard.
 
 ## Incident playbooks
 
diff --git a/docs/deploy/index.md b/docs/deploy/index.md
index 42748f7..2937bb4 100644
--- a/docs/deploy/index.md
+++ b/docs/deploy/index.md
@@ -20,6 +20,7 @@ pages is measured (sample sizes and receipts are linked); identifiers such as `<
 | 5 | [Gateway and routing](gateway.md) | One URL for users, automatic failover from Vertex to Cloud Run, IAP |
 | 6 | [Latency and capacity](../operate/latency-capacity.md) | Request design, engine settings and how to measure your own |
 | 7 | [Operations runbook](../operate/runbook.md) | Health, promotion, rollback, teardown and troubleshooting |
+| 8 | [Monitoring and alerts](../operate/monitoring.md) | Scheduled health check, alert policies, what to do when they fire |
 
 ## Crawl → walk → run
 
diff --git a/docs/operate/monitoring.md b/docs/operate/monitoring.md
new file mode 100644
index 0000000..54671bf
--- /dev/null
+++ b/docs/operate/monitoring.md
@@ -0,0 +1,108 @@
+---
+title: "Monitoring and Alerts"
+description: "How dgem is monitored: the alert policies and what to do when each fires, the scheduled health check, how to change thresholds, and how to deploy or remove the monitoring."
+---
+
+# Monitoring and Alerts
+
+`dgem` monitoring has three parts:
+
+| Part | What it does | Set up by |
+| :--- | :--- | :--- |
+| **Log-based metrics** | Turn gateway decision logs and health-check results into metrics (count, latency, backend) | `scripts/setup_alerts.py` (and `scripts/setup_cloud_monitoring.sh` for the dashboard) |
+| **Scheduled health check** | Every hour, checks the Vertex endpoint and the gateway; once a day, the Cloud Run service | `scripts/deploy_probe.sh` |
+| **Alert policies** | Email when something is broken or slow | `scripts/setup_alerts.py` |
+
+Everything is created by idempotent scripts, so changing a threshold means editing one value and re-running a
+script. Placeholders: `<PROJECT>`, `<REGION>`, `<ENDPOINT_ID>`, `<EMAIL>`.
+
+## The scheduled health check
+
+Two Cloud Run jobs triggered by Cloud Scheduler, running `scripts/probe.py` as a dedicated service account
+(`dgem-probe-sa`, allowed only to call the Vertex endpoint and invoke the gateway and serving services).
+
+| Job | Schedule (UTC) | Targets | What it checks |
+| :--- | :--- | :--- | :--- |
+| `dgem-probe-hourly` | `7 * * * *` | Vertex endpoint; gateway | Vertex: `/health`, the 14-case API contract (valid and error cases must return the expected status and answers), 20 latency requests. Gateway: `/health`, 3 decisions through the default route, which must be answered by Vertex |
+| `dgem-probe-daily` | `17 6 * * *` | Cloud Run serving service | Same as Vertex; the first request wakes the scale-to-zero service (~2.5 min), then it idles back to zero |
+
+Each target writes one JSON log line: `probe_event="dgem.probe"`, `probe_target`, `probe_ok`, `probe_failures`,
+`probe_latency_p50_ms`, `probe_latency_p95_ms`, `probe_denoise_p50_ms`, `probe_version`. Find them in Logs Explorer
+with `jsonPayload.probe_event="dgem.probe"`.
+
+Cost: the hourly job uses the always-warm endpoint (a few seconds of CPU per run); the daily job keeps one Cloud Run
+GPU up for about 20 minutes.
+
+## Alerts: what they mean and what to do
+
+All policies are named `dgem: …`, notify the email channel, and link back to this page.
+
+| Alert | Fires when (default) | Likely causes | What to do |
+| :--- | :--- | :--- | :--- |
+| **Vertex endpoint has no available replica** (critical) | Available replicas sum < 1, or the replica metric is missing, for 5 min | Replica crashed or is restarting; endpoint scaled to 0 while idle; model undeployed | Check `availableReplicaCount` (`gcloud ai endpoints describe <ENDPOINT_ID> --region=<REGION>`). The gateway is serving from Cloud Run meanwhile. If it doesn't recover in ~15 min, redeploy or roll back ([runbook](runbook.md#roll-back)) |
+| **Vertex 5xx/429 error rate** | 5xx + 429 responses > 1% of Vertex responses over 10 min | Overload (`503 server busy`: more than `MAX_INFLIGHT` queued), replica restart, bad image | Check load and replica count; raise max replicas or `MAX_INFLIGHT`; if it started with a new image, roll back |
+| **Gateway failing over from Vertex to Cloud Run** | > 20% of `vertex_first` decisions answered by Cloud Run over 30 min (at least 5 decisions) | Vertex unhealthy or updating | Usually accompanies one of the Vertex alerts; users see slower answers (and a cold start if Cloud Run was idle). Fix Vertex |
+| **Gateway decision latency p95 (Vertex)** (warning) | p95 of Vertex-answered decision wall time > 2,000 ms over 15 min | Queueing under load, very long prompts or many-question policies, reasoning (`think`) templates | Compare with normal (150–500 ms for typical policies). If it's load, add replicas; if it's one template, it may just be heavy |
+| **Scheduled health check failed** | Any health-check run reported `probe_ok=false` in the last ~70 min | API contract changed (new image), endpoint down, auth or IAM change for the probe account, gateway fell back to Cloud Run | Read `jsonPayload.probe_failures` on the failing log line; it names the failing case or request |
+| **Scheduled health check not running** (warning) | No Vertex health-check result for 2 h | Scheduler paused or deleted, job failing to start, image pull or IAM problem | `gcloud scheduler jobs describe dgem-probe-hourly --location=<REGION>`; run the job by hand (below) and read its logs |
+| **Health-check latency to Vertex** (warning) | Health-check median end-to-end latency to Vertex > 400 ms | Slower serving image, GPU contention, network | Normal is ~80 ms from inside the region. Check `probe_denoise_p50_ms` (GPU time, normally ~55–60 ms): if GPU time rose, it's the model server; if only end-to-end rose, it's network or queueing |
+
+Alerts close automatically when the condition clears (after 30 min for Vertex alerts, 2 h for health-check alerts).
+
+## Changing thresholds
+
+Every threshold is an environment variable read by `scripts/setup_alerts.py`:
+
+| Variable | Default | Alert |
+| :--- | :--- | :--- |
+| `VERTEX_NO_REPLICA_MIN` | `5` | Minutes with no replica before alerting |
+| `VERTEX_ERROR_RATE` | `0.01` | Vertex 5xx+429 share |
+| `FAILOVER_SHARE` / `FAILOVER_MIN_DECISIONS` | `0.2` / `5` | Gateway failover share and minimum volume |
+| `DECISION_P95_MS` | `2000` | Gateway decision p95 (ms) |
+| `PROBE_MISSING_HOURS` | `2` | Hours without a health-check result |
+| `PROBE_P50_MS` | `400` | Health-check median latency (ms) |
+
+```bash
+# Preview, then apply (updates policies in place)
+GCP_PROJECT=<PROJECT> ALERT_EMAIL=<EMAIL> VERTEX_ENDPOINT_ID=<ENDPOINT_ID> DECISION_P95_MS=3000 \
+  python3 scripts/setup_alerts.py --dry-run
+GCP_PROJECT=<PROJECT> ALERT_EMAIL=<EMAIL> VERTEX_ENDPOINT_ID=<ENDPOINT_ID> DECISION_P95_MS=3000 \
+  python3 scripts/setup_alerts.py --only gateway-decision-p95
+```
+
+Health-check schedules and targets are set in `scripts/deploy_probe.sh` (`HOURLY_SCHEDULE`, `DAILY_SCHEDULE`,
+`PROBE_LATENCY_REQUESTS`, `PROBE_DAILY=off`); re-run it to apply. To pause an alert without deleting it, disable the
+policy in the Cloud Monitoring console; to pause the health check, `gcloud scheduler jobs pause dgem-probe-hourly
+--location=<REGION>`.
+
+## Deploying the monitoring
+
+```bash
+# 1. Health check (service account, image, jobs, schedules)
+GCP_PROJECT=<PROJECT> \
+PROBE_VERTEX_URL=https://<ENDPOINT_ID>.<REGION>-<PROJECT_NUMBER>.prediction.vertexai.goog/v1/projects/<PROJECT>/locations/<REGION>/endpoints/<ENDPOINT_ID>/invoke \
+PROBE_GATEWAY_URL=https://<gateway-service>.run.app \
+PROBE_CLOUDRUN_URL=https://<CLOUD_RUN_URL> \
+PROBE_GATEWAY_AUDIENCE=<IAP programmatic client ID, if the gateway uses IAP> \
+./scripts/deploy_probe.sh
+gcloud run jobs execute dgem-probe-hourly --region=<REGION> --wait     # run once now
+
+# 2. Metrics, email channel and alert policies
+GCP_PROJECT=<PROJECT> ALERT_EMAIL=<EMAIL> VERTEX_ENDPOINT_ID=<ENDPOINT_ID> python3 scripts/setup_alerts.py
+```
+
+- **Gateway with IAP:** IAP accepts a service account's ID token only if its audience is one of the gateway's
+  programmatic OAuth clients (`gcloud iap settings get --resource-type=cloud-run --service=<gateway> --region=<REGION>`,
+  `programmaticClients`), and the account needs the IAP-secured Web App User role; the script grants it when
+  `PROBE_GATEWAY_AUDIENCE` is set. Use the gateway's `run.app` URL, not a custom domain.
+- **After changing the Vertex endpoint** (new endpoint ID), re-run both scripts with the new values.
+- **Failover and latency alerts need gateway logs with `dgem_backend_requested`** (gateway v0.1.2 or later).
+- **Removing everything:** delete the `dgem:` alert policies, the scheduler jobs and Cloud Run jobs
+  (`dgem-probe-hourly`, `dgem-probe-daily`), and the `dgem-probe-sa` account.
+
+## Manual checks
+
+The health check runs a subset of the manual tools used when promoting a serving image
+([runbook](runbook.md#promote-a-new-serving-image)): `scripts/contract_diff.py`, `scripts/serving_speed.py`,
+`scripts/coldstart_probe.py`, and `scripts/template_sweep.py` (every policy template through a gateway). Run those
+by hand before and after changes; the scheduled check catches regressions in between.
diff --git a/docs/operate/runbook.md b/docs/operate/runbook.md
index 92de19c..e9c482b 100644
--- a/docs/operate/runbook.md
+++ b/docs/operate/runbook.md
@@ -107,12 +107,8 @@ Builds between releases report `git describe` versions such as `v0.1.0-3-gabc123
 
 ## Alerting
 
-`scripts/setup_cloud_monitoring.sh` creates log-based metrics and a dashboard for the gateway. Useful alerts:
-
-- Vertex deployed model with 0 available replicas for > 5 minutes.
-- Gateway `X-DGem-Backend-Used: cloudrun` share rising while the default is `vertex_first` (Vertex unhealthy).
-- `503 server busy` rate > 1% (raise capacity or `MAX_INFLIGHT`).
-- p95 decision latency above your SLO on `dgem.gateway.decide` traces ([Observability](observability.md)).
+Alert policies, the scheduled health check, what each alert means and how to change thresholds:
+[Monitoring and alerts](monitoring.md). `scripts/setup_cloud_monitoring.sh` also creates an operational dashboard.
 
 ## Incident playbooks
 
diff --git a/scripts/deploy_probe.sh b/scripts/deploy_probe.sh
new file mode 100755
index 0000000..335fe7d
--- /dev/null
+++ b/scripts/deploy_probe.sh
@@ -0,0 +1,105 @@
+#!/usr/bin/env bash
+# Deploy the dgem scheduled health check (synthetic probe): two Cloud Run jobs + two Cloud Scheduler triggers.
+#
+#   dgem-probe-hourly  every hour: Vertex endpoint (contract + latency) and the gateway (vertex_first route)
+#   dgem-probe-daily   once a day: the scale-to-zero Cloud Run service (wakes the GPU once, ~20 GPU-minutes/day)
+#
+# Each run writes one JSON log line per target (probe_event="dgem.probe"); scripts/setup_alerts.py turns those into
+# alerts. See docs/operate/monitoring.md.
+#
+# Required:
+#   GCP_PROJECT            hosting project
+#   PROBE_VERTEX_URL       Vertex dedicated invoke base: https://<ID>.<REGION>-<NUM>.prediction.vertexai.goog/v1/projects/<P>/locations/<R>/endpoints/<ID>/invoke
+#   PROBE_GATEWAY_URL      gateway run.app URL (https://dgemma-gateway-<hash>-uc.a.run.app), not an IAP custom domain
+#   PROBE_CLOUDRUN_URL     serving Cloud Run URL (https://dgemma-<hash>-uc.a.run.app)
+# Optional:
+#   GCP_REGION (us-central1)  HOURLY_SCHEDULE ("7 * * * *")  DAILY_SCHEDULE ("17 6 * * *")  SCHEDULE_TZ (Etc/UTC)
+#   GATEWAY_SERVICE (dgemma-gateway)  CLOUDRUN_SERVICE (dgemma)  PROBE_SA_NAME (dgem-probe-sa)
+#   PROBE_LATENCY_REQUESTS (20)  PROBE_DAILY=off (skip the Cloud Run job)
+#   PROBE_GATEWAY_AUDIENCE  required when the gateway has IAP enabled: one of its programmatic OAuth clients
+#                           (gcloud iap settings get --resource-type=cloud-run --service=<gateway> --region=<r>)
+set -euo pipefail
+
+PROJECT="${GCP_PROJECT:?set GCP_PROJECT}"
+REGION="${GCP_REGION:-us-central1}"
+: "${PROBE_VERTEX_URL:?set PROBE_VERTEX_URL (Vertex dedicated invoke base)}"
+: "${PROBE_GATEWAY_URL:?set PROBE_GATEWAY_URL (gateway run.app URL)}"
+PROBE_CLOUDRUN_URL="${PROBE_CLOUDRUN_URL:-}"
+SA_NAME="${PROBE_SA_NAME:-dgem-probe-sa}"
+SA="${SA_NAME}@${PROJECT}.iam.gserviceaccount.com"
+GATEWAY_SERVICE="${GATEWAY_SERVICE:-dgemma-gateway}"
+CLOUDRUN_SERVICE="${CLOUDRUN_SERVICE:-dgemma}"
+HOURLY_SCHEDULE="${HOURLY_SCHEDULE:-7 * * * *}"
+DAILY_SCHEDULE="${DAILY_SCHEDULE:-17 6 * * *}"
+SCHEDULE_TZ="${SCHEDULE_TZ:-Etc/UTC}"
+N="${PROBE_LATENCY_REQUESTS:-20}"
+TAG="$(git rev-parse --short HEAD 2>/dev/null || echo latest)"
+IMAGE="${REGION}-docker.pkg.dev/${PROJECT}/dgem/dgem-probe:${TAG}"
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+
+echo "==> Service account ${SA}"
+gcloud iam service-accounts describe "${SA}" --project="${PROJECT}" >/dev/null 2>&1 || \
+  gcloud iam service-accounts create "${SA_NAME}" --project="${PROJECT}" \
+    --display-name="dgem scheduled health check (read-only probe)"
+# Call the Vertex endpoint (predict/invoke) and invoke the gateway and Cloud Run services. Nothing else.
+gcloud projects add-iam-policy-binding "${PROJECT}" --member="serviceAccount:${SA}" \
+  --role="roles/aiplatform.user" --condition=None --quiet >/dev/null
+if [[ -n "${PROBE_GATEWAY_AUDIENCE:-}" ]]; then
+  # IAP-protected gateway: the SA must also be an IAP-secured Web App User on it.
+  gcloud iap web add-iam-policy-binding --project="${PROJECT}" --resource-type=cloud-run \
+    --service="${GATEWAY_SERVICE}" --region="${REGION}" --member="serviceAccount:${SA}" \
+    --role="roles/iap.httpsResourceAccessor" --condition=None --quiet >/dev/null
+fi
+for svc in "${GATEWAY_SERVICE}" "${CLOUDRUN_SERVICE}"; do
+  gcloud run services add-iam-policy-binding "${svc}" --project="${PROJECT}" --region="${REGION}" \
+    --member="serviceAccount:${SA}" --role="roles/run.invoker" --quiet >/dev/null
+done
+
+echo "==> Building ${IMAGE}"
+CTX="$(mktemp -d)"; trap 'rm -rf "${CTX}"' EXIT
+mkdir -p "${CTX}/scripts" "${CTX}/fixtures"
+cp "${REPO_ROOT}"/scripts/{probe.py,contract_diff.py,serving_speed.py} "${CTX}/scripts/"
+cp -R "${REPO_ROOT}/fixtures/bbox" "${CTX}/fixtures/"
+cp "${REPO_ROOT}/deploy/probe/Dockerfile" "${CTX}/Dockerfile"
+gcloud builds submit "${CTX}" --project="${PROJECT}" --tag="${IMAGE}" --suppress-logs --quiet
+
+deploy_job() {  # name targets task-timeout
+  local job="$1" targets="$2" timeout="$3"
+  local extra="PROBE_LATENCY_REQUESTS=${N}"
+  [[ -n "${PROBE_GATEWAY_AUDIENCE:-}" ]] && extra="${extra}|PROBE_GATEWAY_AUDIENCE=${PROBE_GATEWAY_AUDIENCE}"
+  echo "==> Cloud Run job ${job} (${targets%%=*}...)"
+  gcloud run jobs deploy "${job}" --project="${PROJECT}" --region="${REGION}" --image="${IMAGE}" \
+    --service-account="${SA}" --tasks=1 --max-retries=0 --task-timeout="${timeout}" --cpu=1 --memory=512Mi \
+    --set-env-vars="^|^PROBE_TARGETS=${targets}|${extra}" --quiet >/dev/null
+  # Scheduler calls the Run Admin API as the probe SA (needs run.jobs.run on the job).
+  gcloud run jobs add-iam-policy-binding "${job}" --project="${PROJECT}" --region="${REGION}" \
+    --member="serviceAccount:${SA}" --role="roles/run.invoker" --quiet >/dev/null
+}
+
+schedule() {  # scheduler-name job cron
+  local name="$1" job="$2" cron="$3"
+  local uri="https://run.googleapis.com/v2/projects/${PROJECT}/locations/${REGION}/jobs/${job}:run"
+  local args=(--project="${PROJECT}" --location="${REGION}" --schedule="${cron}" --time-zone="${SCHEDULE_TZ}"
+              --uri="${uri}" --http-method=POST --oauth-service-account-email="${SA}")
+  if gcloud scheduler jobs describe "${name}" --project="${PROJECT}" --location="${REGION}" >/dev/null 2>&1; then
+    gcloud scheduler jobs update http "${name}" "${args[@]}" --quiet >/dev/null
+  else
+    gcloud scheduler jobs create http "${name}" "${args[@]}" --quiet >/dev/null
+  fi
+  echo "    scheduled ${name}: '${cron}' (${SCHEDULE_TZ})"
+}
+
+deploy_job dgem-probe-hourly "vertex=${PROBE_VERTEX_URL},gateway=${PROBE_GATEWAY_URL}" 600s
+schedule dgem-probe-hourly dgem-probe-hourly "${HOURLY_SCHEDULE}"
+if [[ "${PROBE_DAILY:-on}" != "off" && -n "${PROBE_CLOUDRUN_URL}" ]]; then
+  # Long timeout: the first request waits for the scale-to-zero service to wake (~2.5 min).
+  deploy_job dgem-probe-daily "cloudrun=${PROBE_CLOUDRUN_URL}" 1200s
+  schedule dgem-probe-daily dgem-probe-daily "${DAILY_SCHEDULE}"
+fi
+
+cat <<EOF
+
+Done. Run once now:
+  gcloud run jobs execute dgem-probe-hourly --project=${PROJECT} --region=${REGION} --wait
+Results: Cloud Logging, jsonPayload.probe_event="dgem.probe". Alerts: scripts/setup_alerts.py.
+EOF
diff --git a/scripts/probe.py b/scripts/probe.py
new file mode 100755
index 0000000..89db835
--- /dev/null
+++ b/scripts/probe.py
@@ -0,0 +1,194 @@
+#!/usr/bin/env python3
+"""Scheduled health check (synthetic probe) for dgem serving endpoints.
+
+For each target it runs the API contract cases from scripts/contract_diff.py (a failure is any case whose HTTP
+status differs from the expected one, or a valid case without answers) and a short latency sample, then writes
+ONE structured JSON log line per target to stdout. On Cloud Run jobs, Cloud Logging parses that line; the
+log-based metrics and alert policies created by scripts/setup_cloud_monitoring.sh read these fields:
+
+  probe_event="dgem.probe"   probe_target   probe_ok (true/false)   probe_failures (list)
+  probe_latency_p50_ms   probe_latency_p95_ms   probe_denoise_p50_ms   probe_version
+
+Targets (name=URL, repeatable, or PROBE_TARGETS="name=url,name=url"):
+  - a serving base URL (Cloud Run https://<service>.run.app, or a Vertex dedicated .../invoke base): contract + latency
+  - gateway=https://<gateway>: one /api/decide through the gateway default route + /health
+
+  python3 scripts/probe.py --target vertex=<invoke-base> --target gateway=https://<gateway> [-n 20]
+
+Exit code 1 if any target failed (the Cloud Run job execution is then marked failed as well).
+"""
+import argparse
+import json
+import os
+import statistics
+import sys
+import time
+import urllib.error
+import urllib.request
+
+sys.path.insert(0, os.path.dirname(__file__))
+import contract_diff as cd  # noqa: E402
+import serving_speed as ss  # noqa: E402
+
+# Expected HTTP status per contract case (anything else is a failure).
+EXPECTED = {
+    "choice_27_error": 400, "unknown_type_error": 400, "empty_questions_error": 400, "invalid_json_error": 400,
+}
+
+
+def token_provider():
+    """ADC locally; the metadata server on Cloud Run (no key files)."""
+    md = "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default"
+    try:
+        urllib.request.urlopen(urllib.request.Request(md + "/email", headers={"Metadata-Flavor": "Google"}), timeout=1)
+    except Exception:
+        return None  # not on GCP: serving_speed.Tokens uses ADC
+
+    class MD:
+        def get(self, audience=None):
+            at = json.load(urllib.request.urlopen(urllib.request.Request(md + "/token", headers={"Metadata-Flavor": "Google"}), timeout=5))["access_token"]
+            return at, None
+
+    return MD()
+
+
+MD = token_provider()
+
+
+def id_token(aud):
+    if MD is None:
+        return ss.TOK.get()[1]
+    u = f"http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience={aud}"
+    return urllib.request.urlopen(urllib.request.Request(u, headers={"Metadata-Flavor": "Google"}), timeout=5).read().decode()
+
+
+def access_token():
+    return (MD.get()[0] if MD else ss.TOK.get()[0])
+
+
+def bearer(url, audience=None):
+    if "prediction.vertexai.goog" in url:
+        return access_token()
+    base = url.split("/", 3)
+    return id_token(audience or (base[0] + "//" + base[2]))
+
+
+# A gateway protected by IAP (Cloud Run IAP integration) only accepts service-account ID tokens whose audience is
+# one of its programmatic OAuth clients (gcloud iap settings get ... accessSettings.oauthSettings.programmaticClients),
+# not the service URL. Set PROBE_GATEWAY_AUDIENCE to one of them.
+GATEWAY_AUDIENCE = os.environ.get("PROBE_GATEWAY_AUDIENCE") or None
+
+
+class ProbeTarget(cd.Target):
+    def token(self):
+        return bearer(self.base)
+
+
+def probe_serving(name, base, n):
+    t = ProbeTarget(name, base)
+    failures = []
+    st, raw = t.request("/health", None, method="GET")
+    version = ""
+    try:
+        h = json.loads(raw)
+        version = h.get("version", "")
+        if st != 200 or not h.get("vllm_ready", True):
+            failures.append(f"health: {st} vllm_ready={h.get('vllm_ready')}")
+    except Exception:
+        failures.append(f"health: {st} {raw[:80]!r}")
+    for case in cd.CASES:
+        try:
+            r = cd.run_case(t, case)
+        except Exception as e:  # network
+            failures.append(f"{case}: {e!r}"[:160])
+            continue
+        want = EXPECTED.get(case, 200)
+        if r["status"] != want:
+            failures.append(f"{case}: HTTP {r['status']} (want {want}) {(r.get('error') or '')[:80]}")
+        elif want == 200 and not r.get("labels") and case != "systemone":
+            failures.append(f"{case}: no answers")
+    walls, denoise = [], []
+    schema = {"instructions": "Triage the ticket.", "samples": 1, "think": 0, "questions": ss.Q}
+    url = base.rstrip("/") + "/v1/chat/completions"
+    for i in range(n):
+        payload = {"model": "dgemma", "messages": [{"role": "system", "content": json.dumps(schema)},
+                                                   {"role": "user", "content": json.dumps({"ticket": ss.TICKETS[i % len(ss.TICKETS)]})}]}
+        req = urllib.request.Request(url, json.dumps(payload).encode(), {"Content-Type": "application/json", "Authorization": f"Bearer {bearer(base)}"})
+        t0 = time.perf_counter()
+        try:
+            with urllib.request.urlopen(req, timeout=120) as r:
+                body = json.loads(r.read())
+            walls.append((time.perf_counter() - t0) * 1000)
+            inner = json.loads(body["choices"][0]["message"]["content"])
+            denoise.append(float(inner.get("diagnostics", {}).get("timing", {}).get("total_ms") or 0))
+        except Exception as e:
+            failures.append(f"latency request {i}: {e!r}"[:160])
+    return result(name, failures, walls, denoise, version)
+
+
+def probe_gateway(name, base):
+    failures, walls = [], []
+    tok = bearer(base, GATEWAY_AUDIENCE)
+    try:
+        h = json.load(urllib.request.urlopen(urllib.request.Request(base.rstrip("/") + "/health", headers={"Authorization": f"Bearer {tok}"}), timeout=30))
+        version = h.get("version", "")
+    except Exception as e:
+        failures.append(f"health: {e!r}"[:160])
+        version = ""
+    for i in range(3):
+        req = urllib.request.Request(base.rstrip("/") + "/api/decide/support_triage",
+                                     json.dumps({"variables": {"ticket": ss.TICKETS[i]}}).encode(),
+                                     {"Content-Type": "application/json", "Authorization": f"Bearer {tok}", "X-DGem-Surface": "probe"})
+        t0 = time.perf_counter()
+        try:
+            with urllib.request.urlopen(req, timeout=300) as r:
+                d = json.loads(r.read())
+                used = r.headers.get("X-DGem-Backend-Used")
+            walls.append((time.perf_counter() - t0) * 1000)
+            if not d.get("answers"):
+                failures.append("decide: no answers")
+            if used != "vertex":
+                failures.append(f"decide: answered by {used} (vertex_first failed over)")
+        except urllib.error.HTTPError as e:
+            failures.append(f"decide: HTTP {e.code} {e.read()[:80]!r}")
+        except Exception as e:
+            failures.append(f"decide: {e!r}"[:160])
+    return result(name, failures, walls, [], version)
+
+
+def pct(v, q):
+    if not v:
+        return None
+    v = sorted(v)
+    return round(v[min(len(v) - 1, int(round(q * (len(v) - 1))))], 1)
+
+
+def result(name, failures, walls, denoise, version):
+    rec = {"severity": "ERROR" if failures else "INFO", "message": f"dgem probe {name}: {'FAIL' if failures else 'ok'}",
+           "probe_event": "dgem.probe", "probe_target": name, "probe_ok": not failures, "probe_failures": failures[:10],
+           "probe_failure_count": len(failures), "probe_version": version,
+           "probe_latency_p50_ms": pct(walls, 0.5), "probe_latency_p95_ms": pct(walls, 0.95),
+           "probe_denoise_p50_ms": round(statistics.median(denoise), 1) if denoise else None,
+           "probe_requests": len(walls)}
+    print(json.dumps(rec), flush=True)
+    return rec
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--target", action="append", default=[])
+    ap.add_argument("-n", type=int, default=int(os.environ.get("PROBE_LATENCY_REQUESTS", "20")))
+    a = ap.parse_args()
+    targets = a.target or [t for t in os.environ.get("PROBE_TARGETS", "").split(",") if t.strip()]
+    if not targets:
+        sys.exit("no targets (--target name=url or PROBE_TARGETS)")
+    ok = True
+    for spec in targets:
+        name, url = spec.split("=", 1)
+        rec = probe_gateway(name, url) if name.startswith("gateway") else probe_serving(name, url, a.n)
+        ok &= rec["probe_ok"]
+    sys.exit(0 if ok else 1)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/scripts/setup_alerts.py b/scripts/setup_alerts.py
new file mode 100755
index 0000000..c06832e
--- /dev/null
+++ b/scripts/setup_alerts.py
@@ -0,0 +1,240 @@
+#!/usr/bin/env python3
+"""Create or update dgem log-based metrics, an email notification channel, and alert policies.
+
+Idempotent: metrics and policies are matched by name / display name and updated in place, so re-run it after
+changing a threshold. Every threshold has a default and an environment-variable / flag override (below).
+Guide: docs/operate/monitoring.md.
+
+  ALERT_EMAIL=you@example.com GCP_PROJECT=<project> VERTEX_ENDPOINT_ID=<id> python3 scripts/setup_alerts.py
+  python3 scripts/setup_alerts.py --dry-run          # print what would be applied
+
+Policies (display names are prefixed "dgem: "):
+  vertex-no-replicas       Vertex endpoint has no available replica for VERTEX_NO_REPLICA_MIN minutes (default 5)
+  vertex-error-rate        Vertex 5xx+429 share > VERTEX_ERROR_RATE (default 0.01) over 10 min
+  gateway-failover         Share of vertex_first decisions answered by Cloud Run > FAILOVER_SHARE (0.2) over 30 min
+                           (with at least FAILOVER_MIN_DECISIONS=5 vertex_first decisions)
+  gateway-decision-p95     p95 of Vertex-answered decision wall time > DECISION_P95_MS (2000) over 15 min
+  probe-failed             A scheduled health check reported a failure (any target)
+  probe-missing            No health check result for PROBE_MISSING_HOURS (2) hours (target=vertex)
+  probe-latency            Health-check end-to-end median to Vertex > PROBE_P50_MS (400) ms
+
+Auth: gcloud access token (CLOUDSDK_AUTH_ACCESS_TOKEN_FILE or `gcloud auth print-access-token`).
+"""
+import argparse
+import json
+import os
+import subprocess
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+
+PREFIX = "dgem: "
+
+
+def env(name, default):
+    return os.environ.get(name, default)
+
+
+def token():
+    f = os.environ.get("CLOUDSDK_AUTH_ACCESS_TOKEN_FILE")
+    if f and os.path.exists(f):
+        return open(f).read().strip()
+    return subprocess.check_output(["gcloud", "auth", "print-access-token"], text=True).strip()
+
+
+def api(method, url, body=None):
+    req = urllib.request.Request(url, json.dumps(body).encode() if body is not None else None, method=method,
+                                 headers={"Authorization": f"Bearer {token()}", "Content-Type": "application/json"})
+    try:
+        with urllib.request.urlopen(req, timeout=60) as r:
+            raw = r.read()
+            return json.loads(raw) if raw else {}
+    except urllib.error.HTTPError as e:
+        raise SystemExit(f"{method} {url}: HTTP {e.code} {e.read()[:600].decode(errors='replace')}")
+
+
+# ---------------------------------------------------------------- log-based metrics
+def log_metrics():
+    decide = 'resource.type="cloud_run_revision" AND jsonPayload.span_name="dgem.gateway.decide"'
+    probe = 'jsonPayload.probe_event="dgem.probe"'
+    return {
+        # Replaces the older definition from setup_cloud_monitoring.sh, adding backend labels (labels can only be added).
+        "dgem_decisions_total": {
+            "description": "dgem gateway decisions (Studio, REST), by surface, template, backend used and requested",
+            "filter": decide,
+            "metricDescriptor": {"metricKind": "DELTA", "valueType": "INT64", "unit": "1", "labels": [
+                {"key": k, "valueType": "STRING"} for k in
+                ["surface", "template", "user", "multimodal", "reads", "backend", "backend_requested"]]},
+            "labelExtractors": {
+                "surface": "EXTRACT(jsonPayload.dgem_surface)", "template": "EXTRACT(jsonPayload.dgem_template)",
+                "user": "EXTRACT(jsonPayload.dgem_user)", "multimodal": "EXTRACT(jsonPayload.dgem_multimodal)",
+                "reads": "EXTRACT(jsonPayload.dgem_gpu_reads)", "backend": "EXTRACT(jsonPayload.dgem_backend)",
+                "backend_requested": "EXTRACT(jsonPayload.dgem_backend_requested)"},
+        },
+        "dgem_decision_wall_ms": {
+            "description": "dgem gateway decision wall time (ms), by backend used",
+            "filter": decide + " AND jsonPayload.dgem_total_wall_ms > 0",
+            "valueExtractor": "EXTRACT(jsonPayload.dgem_total_wall_ms)",
+            "bucketOptions": {"exponentialBuckets": {"numFiniteBuckets": 40, "growthFactor": 1.25, "scale": 20}},
+            "metricDescriptor": {"metricKind": "DELTA", "valueType": "DISTRIBUTION", "unit": "ms",
+                                 "labels": [{"key": "backend", "valueType": "STRING"}]},
+            "labelExtractors": {"backend": "EXTRACT(jsonPayload.dgem_backend)"},
+        },
+        "dgem_probe_runs": {
+            "description": "dgem scheduled health-check results, by target and outcome",
+            "filter": probe,
+            "metricDescriptor": {"metricKind": "DELTA", "valueType": "INT64", "unit": "1", "labels": [
+                {"key": "target", "valueType": "STRING"}, {"key": "ok", "valueType": "STRING"}]},
+            "labelExtractors": {"target": "EXTRACT(jsonPayload.probe_target)", "ok": "EXTRACT(jsonPayload.probe_ok)"},
+        },
+        "dgem_probe_latency_ms": {
+            "description": "dgem health-check end-to-end median latency per run (ms), by target",
+            "filter": probe + " AND jsonPayload.probe_latency_p50_ms > 0",
+            "valueExtractor": "EXTRACT(jsonPayload.probe_latency_p50_ms)",
+            "bucketOptions": {"exponentialBuckets": {"numFiniteBuckets": 40, "growthFactor": 1.2, "scale": 20}},
+            "metricDescriptor": {"metricKind": "DELTA", "valueType": "DISTRIBUTION", "unit": "ms",
+                                 "labels": [{"key": "target", "valueType": "STRING"}]},
+            "labelExtractors": {"target": "EXTRACT(jsonPayload.probe_target)"},
+        },
+    }
+
+
+def upsert_metric(project, name, spec, dry):
+    base = f"https://logging.googleapis.com/v2/projects/{project}/metrics"
+    body = dict(spec, name=name)
+    if dry:
+        print(f"  metric {name}")
+        return
+    try:
+        api("GET", f"{base}/{name}")
+        api("PUT", f"{base}/{name}", body)
+        print(f"  updated metric {name}")
+    except SystemExit as e:
+        if "404" not in str(e):
+            raise
+        api("POST", base, body)
+        print(f"  created metric {name}")
+
+
+# ---------------------------------------------------------------- notification channel
+def email_channel(project, email, dry):
+    base = f"https://monitoring.googleapis.com/v3/projects/{project}/notificationChannels"
+    name = f"{PREFIX}{email}"
+    if dry:
+        print(f"  channel {name}")
+        return "projects/-/notificationChannels/DRYRUN"
+    for ch in api("GET", base).get("notificationChannels", []):
+        if ch.get("type") == "email" and ch.get("labels", {}).get("email_address") == email:
+            print(f"  using channel {ch['name']} ({email})")
+            return ch["name"]
+    ch = api("POST", base, {"type": "email", "displayName": name, "labels": {"email_address": email}})
+    print(f"  created channel {ch['name']} ({email})")
+    return ch["name"]
+
+
+# ---------------------------------------------------------------- policies
+def promql(query, duration_s, eval_s=60):
+    return {"conditionPrometheusQueryLanguage": {"query": query, "duration": f"{duration_s}s",
+                                                 "evaluationInterval": f"{eval_s}s"}}
+
+
+def policies(project, ep, cfg):
+    lm = "logging_googleapis_com:user_"
+    vx = f'monitored_resource="aiplatform.googleapis.com/Endpoint",endpoint_id="{ep}"'
+    # Log-based metrics can come from several resource types (gateway = Cloud Run service, probe = Cloud Run job);
+    # alerting PromQL requires naming one.
+    rev = 'monitored_resource="cloud_run_revision"'
+    job = 'monitored_resource="cloud_run_job"'
+    doc = ("See docs/operate/monitoring.md (alert '{key}') for what this means and what to do. "
+           "Thresholds: scripts/setup_alerts.py.")
+
+    def pol(key, title, cond, duration_note, severity="ERROR", auto_close="1800s"):
+        return {"displayName": f"{PREFIX}{title}", "combiner": "OR", "severity": severity,
+                "documentation": {"content": doc.format(key=key) + f" {duration_note}", "mimeType": "text/markdown"},
+                "conditions": [dict(cond, displayName=title)],
+                "alertStrategy": {"autoClose": auto_close}, "userLabels": {"app": "dgem", "alert": key}}
+
+    return [
+        pol("vertex-no-replicas", "Vertex endpoint has no available replica",
+            promql(f'sum(avg_over_time(aiplatform_googleapis_com:prediction_online_replicas{{{vx}}}[5m])) < 1 '
+                   f'or absent(aiplatform_googleapis_com:prediction_online_replicas{{{vx}}})',
+                   int(cfg["VERTEX_NO_REPLICA_MIN"]) * 60),
+            "Traffic fails over to Cloud Run (slower; cold start if idle).", "CRITICAL"),
+        pol("vertex-error-rate", "Vertex 5xx/429 error rate",
+            promql(f'(sum(rate(aiplatform_googleapis_com:prediction_online_response_count{{{vx},response_code=~"5..|429"}}[10m])) '
+                   f'/ sum(rate(aiplatform_googleapis_com:prediction_online_response_count{{{vx}}}[10m]))) > {cfg["VERTEX_ERROR_RATE"]}',
+                   300),
+            f"Threshold {float(cfg['VERTEX_ERROR_RATE']):.0%} over 10 min."),
+        pol("gateway-failover", "Gateway failing over from Vertex to Cloud Run",
+            promql(f'(sum(increase({lm}dgem_decisions_total{{{rev},backend_requested="vertex_first",backend="cloudrun"}}[30m])) '
+                   f'/ sum(increase({lm}dgem_decisions_total{{{rev},backend_requested="vertex_first"}}[30m])) > {cfg["FAILOVER_SHARE"]}) '
+                   f'and sum(increase({lm}dgem_decisions_total{{{rev},backend_requested="vertex_first"}}[30m])) >= {cfg["FAILOVER_MIN_DECISIONS"]}',
+                   300),
+            f"More than {float(cfg['FAILOVER_SHARE']):.0%} of vertex_first decisions answered by Cloud Run in 30 min."),
+        pol("gateway-decision-p95", "Gateway decision latency p95 (Vertex)",
+            promql(f'histogram_quantile(0.95, sum by (le) (rate({lm}dgem_decision_wall_ms_bucket{{{rev},backend="vertex"}}[15m]))) > {cfg["DECISION_P95_MS"]}',
+                   300, 120),
+            f"Threshold {cfg['DECISION_P95_MS']} ms over 15 min.", "WARNING"),
+        pol("probe-failed", "Scheduled health check failed",
+            promql(f'sum by (target) (increase({lm}dgem_probe_runs{{{job},ok="false"}}[70m])) > 0', 0, 300),
+            "The log line (jsonPayload.probe_failures) lists what failed.", "ERROR", "7200s"),
+        pol("probe-missing", "Scheduled health check not running",
+            promql(f'absent_over_time({lm}dgem_probe_runs{{{job},target="vertex"}}[{int(cfg["PROBE_MISSING_HOURS"])}h])', 0, 600),
+            f"No health-check result for {cfg['PROBE_MISSING_HOURS']} h: check the Cloud Scheduler trigger and job.", "WARNING", "7200s"),
+        pol("probe-latency", "Health-check latency to Vertex",
+            promql(f'histogram_quantile(0.5, sum by (le) (rate({lm}dgem_probe_latency_ms_bucket{{{job},target="vertex"}}[3h]))) > {cfg["PROBE_P50_MS"]}',
+                   0, 600),
+            f"Median end-to-end latency above {cfg['PROBE_P50_MS']} ms (normal ~150-200 ms).", "WARNING", "7200s"),
+    ]
+
+
+def upsert_policy(project, p, channel, dry, only):
+    key = p["userLabels"]["alert"]
+    if only and key not in only:
+        return
+    p["notificationChannels"] = [channel]
+    base = f"https://monitoring.googleapis.com/v3/projects/{project}/alertPolicies"
+    if dry:
+        print(f"  policy {p['displayName']}\n    {json.dumps(p['conditions'][0])[:400]}")
+        return
+    flt = urllib.parse.quote(f'display_name="{p["displayName"]}"')
+    existing = api("GET", f"{base}?filter={flt}").get("alertPolicies", [])
+    if existing:
+        name = existing[0]["name"]
+        api("PATCH", f"https://monitoring.googleapis.com/v3/{name}", dict(p, name=name))
+        print(f"  updated policy {p['displayName']} ({name.split('/')[-1]})")
+    else:
+        r = api("POST", base, p)
+        print(f"  created policy {p['displayName']} ({r['name'].split('/')[-1]})")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--dry-run", action="store_true")
+    ap.add_argument("--only", nargs="*", help="only these policy keys (e.g. probe-latency)")
+    ap.add_argument("--skip-metrics", action="store_true")
+    a = ap.parse_args()
+    project = env("GCP_PROJECT", "")
+    email = env("ALERT_EMAIL", "")
+    ep = env("VERTEX_ENDPOINT_ID", "")
+    if not (project and email and ep):
+        sys.exit("set GCP_PROJECT, ALERT_EMAIL and VERTEX_ENDPOINT_ID")
+    cfg = {k: env(k, d) for k, d in {
+        "VERTEX_NO_REPLICA_MIN": "5", "VERTEX_ERROR_RATE": "0.01", "FAILOVER_SHARE": "0.2",
+        "FAILOVER_MIN_DECISIONS": "5", "DECISION_P95_MS": "2000", "PROBE_MISSING_HOURS": "2", "PROBE_P50_MS": "400"}.items()}
+    print(f"Project {project}, endpoint {ep}, alerts to {email}")
+    print("Thresholds: " + ", ".join(f"{k}={v}" for k, v in cfg.items()))
+    if not a.skip_metrics:
+        print("Log-based metrics:")
+        for name, spec in log_metrics().items():
+            upsert_metric(project, name, spec, a.dry_run)
+    print("Notification channel:")
+    ch = email_channel(project, email, a.dry_run)
+    print("Alert policies:")
+    for p in policies(project, ep, cfg):
+        upsert_policy(project, p, ch, a.dry_run, set(a.only or []))
+
+
+if __name__ == "__main__":
+    main()
diff --git a/scripts/template_sweep.py b/scripts/template_sweep.py
new file mode 100644
index 0000000..b709230
--- /dev/null
+++ b/scripts/template_sweep.py
@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+"""Run every policy template through a dgem gateway with its sample variables and check the answer shape.
+
+Uses GET /api/templates (id, sample_vars, multimodal) and POST /api/decide/<id>. For multimodal templates a
+fixture image from fixtures/bbox/ is attached. A template passes when the call returns 200, every question in
+the rendered schema has an answer with probabilities that sum to ~1, and (optionally) the request ran on the
+expected backend.
+
+  python3 scripts/template_sweep.py --gateway https://<your-dgem-gateway> [--backend vertex] [-o out.json]
+
+Auth: Google ID token from Application Default Credentials (as scripts/serving_speed.py).
+"""
+import argparse
+import base64
+import concurrent.futures as cf
+import glob
+import json
+import os
+import sys
+import time
+import urllib.error
+import urllib.request
+
+sys.path.insert(0, os.path.dirname(__file__))
+import serving_speed as ss  # noqa: E402
+
+REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+
+
+def image_uri():
+    p = sorted(glob.glob(os.path.join(REPO, "fixtures/bbox/*.png")))[0]
+    return "data:image/png;base64," + base64.b64encode(open(p, "rb").read()).decode()
+
+
+def call(gw, path, body=None, backend=None):
+    _, it = ss.TOK.get()
+    h = {"Authorization": f"Bearer {it}", "Content-Type": "application/json"}
+    if backend:
+        h["X-DGem-Backend"] = backend
+    req = urllib.request.Request(gw + path, json.dumps(body).encode() if body is not None else None, h)
+    t0 = time.perf_counter()
+    try:
+        with urllib.request.urlopen(req, timeout=300) as r:
+            return r.status, dict(r.headers), json.loads(r.read()), (time.perf_counter() - t0) * 1000
+    except urllib.error.HTTPError as e:
+        return e.code, dict(e.headers), {"error": e.read()[:300].decode(errors="replace")}, (time.perf_counter() - t0) * 1000
+
+
+def check(gw, tpl, backend, img):
+    body = {"variables": tpl.get("sample_vars") or {}}
+    if tpl.get("multimodal"):
+        body["image"] = img
+    st, hdr, d, ms = call(gw, f"/api/decide/{tpl['id']}", body, backend)
+    out = {"template": tpl["id"], "status": st, "wall_ms": round(ms), "backend_used": hdr.get("X-DGem-Backend-Used") or hdr.get("X-Dgem-Backend-Used")}
+    if st != 200:
+        out["error"] = str(d.get("error"))[:200]
+        return out
+    answers = d.get("answers") or {}
+    bad = []
+    for qid, a in answers.items():
+        probs = (a or {}).get("probabilities") or {}
+        if a and a.get("label") in ("", None) and not probs:
+            continue  # skipped conditional question
+        if probs and abs(sum(probs.values()) - 1) > 0.02:
+            bad.append(f"{qid}: probabilities sum {sum(probs.values()):.3f}")
+    out.update(questions=len(answers), problems=bad)
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--gateway", required=True)
+    ap.add_argument("--backend", default=None, help="X-DGem-Backend for every call (default: gateway default)")
+    ap.add_argument("-o", "--output")
+    a = ap.parse_args()
+    gw = a.gateway.rstrip("/")
+    st, _, templates, _ = call(gw, "/api/templates")
+    if st != 200:
+        sys.exit(f"GET /api/templates: {st} {templates}")
+    if isinstance(templates, dict):
+        templates = templates.get("templates", [])
+    img = image_uri()
+    with cf.ThreadPoolExecutor(4) as ex:
+        res = list(ex.map(lambda t: check(gw, t, a.backend, img), templates))
+    ok = [r for r in res if r["status"] == 200 and not r.get("problems") and r.get("questions")]
+    for r in sorted(res, key=lambda r: r["template"]):
+        flag = "ok " if r in ok else "FAIL"
+        print(f"{flag} {r['template']:34s} {r['status']} q={r.get('questions', '-')} {r['wall_ms']:>6} ms {r.get('backend_used') or ''} {r.get('error') or ' '.join(r.get('problems') or [])}")
+    print(f"\n{len(ok)}/{len(res)} templates passed")
+    if a.output:
+        json.dump({"gateway_backend": a.backend, "results": res}, open(a.output, "w"), indent=1)
+    sys.exit(0 if len(ok) == len(res) else 1)
+
+
+if __name__ == "__main__":
+    main()