decide: ✨ Ask typed questions of TypeSafe Jev and OpenAI Decisions

.claude/fragments/great-docs.yml/great-docs-local.fragment.yml

@@ -75,6 +75,24 @@ reference:
       - ClaudeConfig
       - CodexConfig
       - GeminiConfig
+  - title: Deciding
+    desc: Typed yes/no, label, and score decisions over TypeSafe Jev and OpenAI Decisions, one question set for either provider.
+    contents:
+      - decide.decide
+      - decide.decide_sync
+      - Binary
+      - Label
+      - Score
+      - Decision
+      - BinaryAnswer
+      - LabelAnswer
+      - ScoreAnswer
+      - Refused
+      - Provider
+      - DecideError
+      - DecideKeyMissing
+      - decide.api_key
+      - decide.store_key
   - title: Backends
     desc: CLI backends (Claude, Codex, and the Gemini family), the in-process Claude Agent SDK backend, the Apple Foundation Models backend over its bundled Swift sidecar, the MLX and OpenAI-endpoint backends, the specialty registry, backend selection, and install/auth status checks.
     contents:

CHANGELOG.md

@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **`decide` and `decide_sync` ask typed questions of TypeSafe Jev or OpenAI
+  Decisions.** A `Binary`, `Label`, or `Score` question set runs unchanged
+  on either provider (`JEV`, pinned to `jev-1.13.0`, or `OPENAI`, on
+  `gpt-6-luna`), and every answer comes back in declared order: a
+  `BinaryAnswer` with `p_yes`, a `LabelAnswer` or `ScoreAnswer` with
+  per-option probabilities and a confidence, or `Refused` when OpenAI
+  declines one question and answers the rest. The Rust core owns both wire
+  mappings and the retry policy as the `decide_plan` and `decide_resolve`
+  ops, pinned by 21 new golden vectors that include one recorded response
+  from each provider. The policy retries 408, 429, 5xx, 529, and a lost
+  connection, backing off from 0.5s to a 5s cap and honoring `retry-after`
+  and `retry-after-ms`, but only while the retry fits inside the caller's
+  `timeout`. A 401 or 422 raises `DecideError` on the first try. Each
+  process keeps one keep-alive client, and a warm three-question Jev call
+  measured 120 to 130 ms end to end from a Mac. Keys come from
+  `TYPESAFE_API_KEY` or `OPENAI_API_KEY`. On macOS, an unset variable falls
+  through to the Keychain item that `spawnllm key set jev` (or `openai`)
+  writes from stdin, read at call time.
 - **The OpenAI-compatible endpoint backend sets `reasoning_effort`.** A typed
   `reasoning_effort` (`none`, `minimal`, `low`, `medium`, `high` or `xhigh`)
   on the endpoint config goes into every `/chat/completions` request body,

README.md

@@ -105,6 +105,26 @@ print(call_sync("Reply with just the word: pong", backend=AppleBackend()))
 
 Auto-selection tries this backend last and only for `model="small"`; an explicit `backend=AppleBackend()` always reaches it. Session and decoding knobs (`use_case`, `guardrails`, `instructions`, `temperature`, sampling) ride in via `RunSpec(provider_configs={"apple": AppleConfig(...)})`. Structured `extract_sync` works too, nested models included, and schema constraints now bind during decoding: `minimum`/`maximum`, `minItems`/`maxItems`, and string-valued `enum`s are enforced exactly (a non-string `enum` such as `Literal[1, 2]` fails before generation, `extract_sync` raising `BackendCallError`), and a `Field(pattern=...)` constrains the value's shape — length, separators, and character families. Apple's decoder rejects bracket character classes, so the sidecar widens each to the narrowest escape it accepts (`^[A-Z]{3}-\d{4}$` decodes as `\w{3}-\d{4}`), and pydantic stays the exact validator: a value that fits the widened shape but violates your regex raises a plain `ValidationError`. Self-referential models extract cleanly; a mutually recursive pair (A referencing B referencing A) fails cleanly instead, `extract_sync` raising `BackendCallError` and `run` returning an error `Response`. Requires macOS 26+ on Apple Silicon with Apple Intelligence enabled — every other platform gets the pure-Python wheel and reports the backend as not installed.
 
+### Ask a yes/no, label, or score question in about 120 ms
+
+A classifier that runs on every hook event or every PR comment can't wait seconds for a chat model to write JSON. `decide_sync` asks TypeSafe's Jev or OpenAI Decisions instead. Both return probabilities, not prose, and one question set runs unchanged on either:
+
+```python
+from spawnllm import Binary, Label, decide_sync
+
+decision = decide_sync(
+    "Our nightly release has failed for 3 days. Please roll it back now.",
+    {
+        "rollback": Binary("Does the message ask to roll back a release?"),
+        "action": Label("Which release action does it ask for?", {"none": None, "release": None, "rollback": None}),
+    },
+)
+print(decision.answers["rollback"].p_yes)  # 0.99
+print(decision.answers["action"].choice)  # rollback
+```
+
+On a warm keep-alive connection from a Mac, a three-question Jev call took 120 to 130 ms end to end. Pass `provider=OPENAI` to ask `gpt-6-luna` instead. Retries cover 408, 429, 5xx, and 529 and stop at `timeout`. Keys come from `TYPESAFE_API_KEY` or `OPENAI_API_KEY`, or on macOS from a Keychain item that `spawnllm key set jev` (or `openai`) writes from stdin.
+
 ### Call the same backends from Go or Rust
 
 All three languages run the identical engine: argv planning, output parsing, schema strictification, and retry policy live once in a Rust core — linked natively by the Rust crate, embedded as WASM by the Go module and the Python package — pinned by a shared golden-vector suite and released in lockstep.
@@ -119,9 +139,10 @@ Both expose `Call`/`call` and typed `Extract`/`extract` against your existing CL
 ## More in the docs
 
 - **Spec-driven runs** — a literal model id, per-provider flag passthrough, and envelope-aware retry via `RunSpec` — [Running reference](https://yasyf.github.io/spawnllm/reference/#running)
+- **Decisions** — every question and answer type, the provider pins, and the retry and key-lookup rules — [Deciding reference](https://yasyf.github.io/spawnllm/reference/#deciding)
 - **Backend selection** — the priority chain, plus `specialty=` routing (`debugging` and `review` go to Codex, `general` to the Claude Agent SDK backend) — [Backends reference](https://yasyf.github.io/spawnllm/reference/#backends)
 - **Transport helpers** — `run_cli`, `collect_process`, and `map_concurrent`, the subprocess plumbing shared by every CLI backend — [Transport reference](https://yasyf.github.io/spawnllm/reference/#transport)
-- **The CLI** — `spawnllm call`, `status`, and `backends` from any shell — [CLI reference](https://yasyf.github.io/spawnllm/reference/cli/)
+- **The CLI** — `spawnllm call`, `status`, `backends`, and `key set` from any shell — [CLI reference](https://yasyf.github.io/spawnllm/reference/cli/)
 - **MLX internals** — the adapter codec, fuser, and runtime patches behind the local engine — [MLX reference](https://yasyf.github.io/spawnllm/reference/#mlx)
 
 Read the [docs](https://yasyf.github.io/spawnllm/) for the full guide and API reference. Licensed under [MIT](https://github.com/yasyf/spawnllm/blob/main/LICENSE).

conformance/vectors/decide_plan/jev-binary-criteria.json

@@ -0,0 +1,43 @@
+{
+  "name": "jev-binary-criteria",
+  "op": "decide_plan",
+  "input": {
+    "provider": {
+      "name": "jev",
+      "model": "jev-1.13.0"
+    },
+    "api_key": "test-key",
+    "state": "Help! Our nightly release has been failing for 3 days and customers are blocked. Please roll it back now.",
+    "questions": [
+      {
+        "type": "binary",
+        "id": "is_urgent",
+        "instructions": "Does this convey urgency?",
+        "yes": "Explicitly time-sensitive",
+        "no": "No urgency expressed"
+      }
+    ]
+  },
+  "expected": {
+    "kind": "http",
+    "method": "POST",
+    "url": "https://api.typesafe.ai/v1/systemone",
+    "headers": {
+      "Authorization": "Bearer test-key"
+    },
+    "body": {
+      "state": "Help! Our nightly release has been failing for 3 days and customers are blocked. Please roll it back now.",
+      "model": "jev-1.13.0",
+      "questions": {
+        "is_urgent": {
+          "type": "noul",
+          "instructions": "Does this convey urgency?",
+          "criteria": {
+            "true": "Explicitly time-sensitive",
+            "false": "No urgency expressed"
+          }
+        }
+      }
+    }
+  }
+}

conformance/vectors/decide_plan/jev-structured-state-verbatim.json

@@ -0,0 +1,117 @@
+{
+  "name": "jev-structured-state-verbatim",
+  "op": "decide_plan",
+  "input": {
+    "provider": {
+      "name": "jev",
+      "model": "jev-1.13.0"
+    },
+    "api_key": "test-key",
+    "state": {
+      "thread": [
+        {
+          "from": "user",
+          "text": "ship it"
+        },
+        {
+          "from": "bot",
+          "text": "queued",
+          "attempt": 2
+        }
+      ]
+    },
+    "questions": [
+      {
+        "type": "binary",
+        "id": "is_rollback_request",
+        "instructions": "Does the message ask to roll back a release?"
+      },
+      {
+        "type": "label",
+        "id": "action",
+        "instructions": "Which release action does the message ask for?",
+        "options": [
+          {
+            "value": "none",
+            "description": "No release action"
+          },
+          {
+            "value": "release",
+            "description": "Ship a new release"
+          },
+          {
+            "value": "rollback",
+            "description": "Roll back a release"
+          }
+        ]
+      },
+      {
+        "type": "score",
+        "id": "urgency",
+        "instructions": "How urgent is the message?",
+        "levels": [
+          {
+            "label": "Not urgent",
+            "description": null
+          },
+          {
+            "label": "Somewhat urgent",
+            "description": null
+          },
+          {
+            "label": "Blocking, needs action now",
+            "description": "customers are blocked"
+          }
+        ]
+      }
+    ]
+  },
+  "expected": {
+    "kind": "http",
+    "method": "POST",
+    "url": "https://api.typesafe.ai/v1/systemone",
+    "headers": {
+      "Authorization": "Bearer test-key"
+    },
+    "body": {
+      "state": {
+        "thread": [
+          {
+            "from": "user",
+            "text": "ship it"
+          },
+          {
+            "from": "bot",
+            "text": "queued",
+            "attempt": 2
+          }
+        ]
+      },
+      "model": "jev-1.13.0",
+      "questions": {
+        "is_rollback_request": {
+          "type": "noul",
+          "instructions": "Does the message ask to roll back a release?"
+        },
+        "action": {
+          "type": "choice",
+          "instructions": "Which release action does the message ask for?",
+          "criteria": {
+            "none": "No release action",
+            "release": "Ship a new release",
+            "rollback": "Roll back a release"
+          }
+        },
+        "urgency": {
+          "type": "score",
+          "instructions": "How urgent is the message?",
+          "criteria": [
+            "Not urgent",
+            "Somewhat urgent",
+            "Blocking, needs action now: customers are blocked"
+          ]
+        }
+      }
+    }
+  }
+}

conformance/vectors/decide_plan/jev-three-questions.json

@@ -0,0 +1,93 @@
+{
+  "name": "jev-three-questions",
+  "op": "decide_plan",
+  "input": {
+    "provider": {
+      "name": "jev",
+      "model": "jev-1.13.0"
+    },
+    "api_key": "test-key",
+    "state": "Help! Our nightly release has been failing for 3 days and customers are blocked. Please roll it back now.",
+    "questions": [
+      {
+        "type": "binary",
+        "id": "is_rollback_request",
+        "instructions": "Does the message ask to roll back a release?"
+      },
+      {
+        "type": "label",
+        "id": "action",
+        "instructions": "Which release action does the message ask for?",
+        "options": [
+          {
+            "value": "none",
+            "description": "No release action"
+          },
+          {
+            "value": "release",
+            "description": "Ship a new release"
+          },
+          {
+            "value": "rollback",
+            "description": "Roll back a release"
+          }
+        ]
+      },
+      {
+        "type": "score",
+        "id": "urgency",
+        "instructions": "How urgent is the message?",
+        "levels": [
+          {
+            "label": "Not urgent",
+            "description": null
+          },
+          {
+            "label": "Somewhat urgent",
+            "description": null
+          },
+          {
+            "label": "Blocking, needs action now",
+            "description": "customers are blocked"
+          }
+        ]
+      }
+    ]
+  },
+  "expected": {
+    "kind": "http",
+    "method": "POST",
+    "url": "https://api.typesafe.ai/v1/systemone",
+    "headers": {
+      "Authorization": "Bearer test-key"
+    },
+    "body": {
+      "state": "Help! Our nightly release has been failing for 3 days and customers are blocked. Please roll it back now.",
+      "model": "jev-1.13.0",
+      "questions": {
+        "is_rollback_request": {
+          "type": "noul",
+          "instructions": "Does the message ask to roll back a release?"
+        },
+        "action": {
+          "type": "choice",
+          "instructions": "Which release action does the message ask for?",
+          "criteria": {
+            "none": "No release action",
+            "release": "Ship a new release",
+            "rollback": "Roll back a release"
+          }
+        },
+        "urgency": {
+          "type": "score",
+          "instructions": "How urgent is the message?",
+          "criteria": [
+            "Not urgent",
+            "Somewhat urgent",
+            "Blocking, needs action now: customers are blocked"
+          ]
+        }
+      }
+    }
+  }
+}

conformance/vectors/decide_plan/openai-binary-criteria.json

@@ -0,0 +1,40 @@
+{
+  "name": "openai-binary-criteria",
+  "op": "decide_plan",
+  "input": {
+    "provider": {
+      "name": "openai",
+      "model": "gpt-6-luna"
+    },
+    "api_key": "test-key",
+    "state": "Help! Our nightly release has been failing for 3 days and customers are blocked. Please roll it back now.",
+    "questions": [
+      {
+        "type": "binary",
+        "id": "is_urgent",
+        "instructions": "Does this convey urgency?",
+        "yes": "Explicitly time-sensitive",
+        "no": "No urgency expressed"
+      }
+    ]
+  },
+  "expected": {
+    "kind": "http",
+    "method": "POST",
+    "url": "https://api.openai.com/v1/decisions",
+    "headers": {
+      "Authorization": "Bearer test-key"
+    },
+    "body": {
+      "model": "gpt-6-luna",
+      "input": "Help! Our nightly release has been failing for 3 days and customers are blocked. Please roll it back now.",
+      "questions": [
+        {
+          "type": "predicate",
+          "name": "is_urgent",
+          "instructions": "Does this convey urgency?\nYes means: Explicitly time-sensitive\nNo means: No urgency expressed"
+        }
+      ]
+    }
+  }
+}