Skip to content

feat(runtime): add a decision model seam with opt-in and fail-open - #1740

Open
AlexStocks wants to merge 3 commits into
oceanbase:masterfrom
AlexStocks:feat/decision-model-foundation
Open

AlexStocks wants to merge 3 commits into
oceanbase:masterfrom
AlexStocks:feat/decision-model-foundation

Conversation

@AlexStocks

@AlexStocks AlexStocks commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Related to #1643, #1644, #1645, #1647 and #1648 — this change is the shared prerequisite for
those issues and closes none of them.

Rationale for this change

PowerContext's Memory and Work paths are full of narrow, high-frequency judgements: is this
observation worth writing, is a recalled item actually relevant, is the evidence sufficient, is a
Handoff safe to continue. Today those judgements are either not made at all, or routed through a
general generation model — slow and costly for what amounts to a boolean, and returning free-form
text the caller still has to interpret.

This PR adds a decision role to the Runtime, beside generation, embedding and rerank. A
DecisionModel answers one bounded question with a problem-neutral yes / no / abstain
verdict, so a caller branches on a value instead of parsing prose.

Three properties are load-bearing:

  1. It is a Runtime port, not an LLM tool. Only deterministic Runtime code calls evaluate.
    The role never enters the MCP tool catalog (server/mcp.py is untouched), so a judgement can
    never authorize an action by itself.
  2. Fail-open is defined once. A single FailOpenDecisionModel envelope wraps every backend —
    configured or injected — and converts any backend failure into a no-op abstention. Call sites
    need no try/except, and asyncio.CancelledError propagates unchanged, because cancellation
    is control flow rather than failure.
  3. It is independent of the processing schema. Decision support adds no tables, no migrations
    and no processing capabilities; canonical_processing_manifest(config) and
    processing_capabilities(config) are provably unchanged. An unavailable decision backend is
    never reported as ProcessingSchemaNotReadyError.

The role is disabled by default (runtime.decision_assistance_enabled=false) and ships with
no consumers: this PR lands the contract and the wiring only.

One envelope, one policy — a known limit

The envelope applies a single failure policy to every consumer. That is correct while the role
has no consumers, and it is correct for the consumers that want fail-open. It is not sufficient
for a consumer that must not silently degrade: RFC 0080 makes Memory reranking fail-closed, so a
rerank consumer cannot use this envelope as it stands and needs the policy resolved per role.

That separation is deliberately not part of this PR. With no second consumer it would be
unexercised code with nothing to prove it, which is the same shape as the dead configuration this
project has already had to remove once. It will land together with the first consumer that needs
the opposite policy, where the branch is visible end to end.

Failure semantics (worth reviewing)

  • Runtime failures — rejected credentials, timeout, malformed or schema-invalid output, empty
    answer — degrade to outcome=abstain with used_fallback=true, and consumers must treat that
    as "do nothing".
  • Assembly-time configuration errors fail fast. If the role is explicitly enabled together
    with an explicitly configured backend that cannot be constructed, open_builtin_runtime raises
    instead of silently degrading. Degrading here would hide a configuration mistake behind a role
    that always abstains, which is the failure mode this design most wants to avoid. This matches
    the existing generation / rerank behaviour.
  • abstain deliberately covers both a backend's considered refusal (used_fallback=false) and the
    fail-open degradation (used_fallback=true); consumers must treat both identically.

What changes are included in this PR?

New:

  • src/powercontext/builtin/runtime/decision_model.py — DecisionOutcome, DecisionRequest,
    DecisionResult, DecisionInput / DecisionOutput, the DecisionModel Protocol
    (policy_id + async evaluate(request, /)), LLMDecisionModel, and the FailOpenDecisionModel
    envelope.
  • Six test modules covering the contract, configuration, composition, fail-open, default-off
    regressions, and schema decoupling.

Changed:

  • runtime/config.py — decision_assistance_enabled: bool = False, plus decision_model,
    decision_base_url, decision_headers, decision_model_settings, decision_timeout_seconds
    and decision_max_requests, wired into the existing identifier, header and settings validators
    and into the cross-field override checks.
  • runtime/composition.py — assembly (a dedicated decision model, or the generation model),
    fail-open wrapping, an optional tracing wrapper, a decision-model configuration error key, and
    a non-blocking inference.decision readiness probe. Three small helpers were extracted to keep
    open_builtin_runtime and _generation_pipelines within the configured cyclomatic-complexity
    limit.
  • runtime/relational.py, runtime/application.py — read-only seams
    (RelationalContexts.decision_model, BuiltinRuntime.decision_model) for later consumers. No
    existing behaviour is touched.
  • runtime/__init__.py — re-exports the new public names.
  • .env.example — documents the new decision settings, along with the previously undocumented
    POWERCONTEXT_SERVER_RUNTIME_RECALL_GATE_* settings.
  • tests/builtin/runtime/test_readiness.py — covers the non-blocking decision probe.

Are there any user-facing changes?

Additive and opt-in only. With decision_assistance_enabled unset the role is simply absent: no
backend is built, no readiness check is registered, and no model call is made.

No breaking changes to public APIs or persisted formats. No new dependency —
pyproject.toml and uv.lock are untouched.

How was this change tested?

Every gate was run inside an isolated worktree, using its own virtual environment:

  • ruff check . — All checks passed
  • ruff format --check . — 886 files already formatted
  • ty check --python-version 3.11 <changed src files> — All checks passed
  • pytest — 3208 passed, 228 skipped, 25 failed

The 25 failures are pre-existing and unrelated to this change. They all live in
tests/client/test_receiver_service.py, test_ci_release_verification.py,
test_config_wizard*.py, test_scan_e2e_evidence.py, test_setup_transport.py and
test_system_cli.py; none of them imports any symbol this PR adds, and running the same eight
modules on an unmodified master worktree with a fresh virtual environment reproduces the
identical 25 failures. Their root causes are Windows/Linux platform assumptions — for example a
0o600 file-mode assertion, and a path assertion that expects forward slashes. In contrast
tests/builtin/runtime/ passes with 454 passed / 0 failed.

CI on this commit is green for quality, tests (3.11) through tests (3.14),
windows-unit-portability and both Acceptance jobs. Those jobs are the authority for the
platform-specific failures above, which are reproducible on unmodified master.

Beyond the suite, the behaviour was verified by driving real failures rather than by reading code:

  • Pointed a real backend at a local endpoint that returned 401, hung past its timeout, and
    returned an invalid enum, asserting abstain + used_fallback=true for each case.
  • Cancelled an in-flight evaluate and asserted asyncio.CancelledError propagates instead of
    being swallowed into an abstention.
  • Enabled the role with an injected backend and asserted the exposed object is always the
    fail-open wrapper (tracing outermost), and that a dedicated backend with no generation model
    boots and answers.
  • Asserted the disabled default registers no readiness check and issues zero HTTP calls, by
    intercepting the transport layer rather than inspecting logs.
  • Asserted canonical_processing_manifest(config) and processing_capabilities(config) are
    identical across decision_* variations.

AI usage statement

AI assistance was used to develop this change: the implementation and its tests were produced with
a WorkBuddy agent (Claude-family model) working from a human-reviewed design. The diff, and every
command listed above, were reviewed and run by the author before opening this PR.

Add the cross-family decision role contract (DecisionModel, DecisionRequest, DecisionResult, DecisionOutcome, DecisionInput/Output, LLMDecisionModel, FailOpenDecisionModel) with its configuration surface, composition wiring, and a non-blocking inference.decision readiness probe.

The role is disabled by default and has no consumers: no decision backend is built and no model call is made until runtime.decision_assistance_enabled is set or a backend is injected. The shared FailOpenDecisionModel envelope turns any backend failure into a no-op abstention while asyncio.CancelledError propagates, so callers need no try/except at the call site.

Decision support depends on no persistence schema: it adds no tables, migrations, or processing capabilities, and leaves canonical_processing_manifest unchanged. It is never registered as an MCP tool.
The decision-seam tests must pass the repository-wide 'ty check' (make check runs it globally, including tests/), not only the five production files.

- Type the _config(**runtime) and InferenceConfig(**overrides) helpers with Any so per-field splats are accepted.
- Build decision_base_url through AnyHttpUrl, matching the existing inference-endpoint tests.
- Suppress the intentional frozen-dataclass assignment with ty's own '# ty: ignore[invalid-assignment]' rule code, which ty recognizes (the previous mypy '# type: ignore[misc]' did not apply).
- Narrow the Memory | None returned by remember() before passing it to search().

@AsperforMias AsperforMias left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The managed HTTP backend handles the tested failure and timeout paths, but the injected-backend path has the timeout gap described inline. There are no production consumers in this PR, so I am recording this as a boundary issue rather than a demonstrated current outage.

Design note (non-blocking): this is shared infrastructure, not completion of #1643, #1644, #1645, #1647 or #1648; their domain experiments and acceptance criteria remain follow-up work, as the PR states. Please also reconcile the overlapping, different contract in #1739 before landing both.

"""Delegate one decision, converting any backend failure into an abstention."""

try:
return await self._delegate.evaluate(request)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Bound waiting on injected decision backends

An injected backend is awaited here without a deadline; decision_timeout_seconds is applied only when constructing the managed generator. Through open_builtin_runtime(..., decision_model=HangingBackend()), a backend awaiting an unset event remained pending after 0.12s with decision_timeout_seconds=0.02, and returned only after external cancellation. No abstention was produced.

This leaves injected backends without the timeout protection that managed backends receive: once a consumer awaits evaluation in a request path, an unresponsive backend can hold that request indefinitely. Apply a deadline at the common boundary while preserving cancellation propagation, or explicitly document that injected implementations must enforce their own deadlines and that this setting does not cover them. The current PR has no production consumers, so the reproduction establishes an unbounded wait, not an existing production outage.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b02f2614. The shared FailOpenDecisionModel envelope now accepts an optional timeout and open_builtin_runtime() passes decision_timeout_seconds (falling back to generation_timeout_seconds) into that common boundary. Injected decision backends therefore get the same bounded wait as managed backends, while CancelledError still propagates.

Added a regression test with an injected hanging backend and a 0.01s configured timeout.

Validation: uv run --no-sync pytest tests/builtin/runtime/test_decision_composition.py -q (8 passed), ruff check, ruff format --check, and ty check --python-version 3.11 on the changed files.

@AlexStocks
AlexStocks requested review from PsiACE and Teingi September 26, 2026 04:14
@AlexStocks

Copy link
Copy Markdown
Contributor Author

Foundation PR for the decision-model series. #1742 stacks on top of this branch, so merging this one first unblocks the rest.

CI is green (23/23) at 35bf44e2. Requesting review from the default owners — one approval is all that is left.

Apply the configured decision timeout at the shared fail-open decision envelope so injected decision models receive the same deadline protection as managed backends while cancellation still propagates.

Tested: uv run --no-sync pytest tests/builtin/runtime/test_decision_composition.py -q; ruff check/format --check decision files; ty check decision files

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants