[akol1]

What this changes

One invariant, one change: a research-context refusal that arrives after a stream has been admitted stays a streamed error event, instead of being answered as a whole reply because the sink had not opened yet. On an unopened sink, error.Refused for a context-window overrun now goes through r.streamedContext; every other error still goes through r.unsent.

The branch is rebased onto d1018b318 and carries only 8009df42a (author Christian Schröder). The only delta from the reviewed bf756299c is adopting main's pre-token keepalive sink: live.out, live.every_ns and the live argument to cortheon.loop.stream and chat.generate are preserved, and the guard plus the regression are unchanged (git range-diff 76aafcf27..bf756299c d1018b318..8009df42a).

Receipt

• Environment: Zig 0.17.0 from .zig-version, -Dcpu=apple_m1, macOS 26.6.2 on an Apple M3 Ultra (80-core GPU, 512 GB). Base d1018b318, head 8009df42a. No checkpoint, weights, GPU kernel or inference path is involved; the regression and the goldens drive the server's stub engine.
• Exactness: no model-token comparison, because no inference path changes. The contracts exercised here are HTTP status and SSE begin/error ordering for streamed and nonstreamed requests, and the streamed refusal event contents.
• Decode speed: not measured; no inference path changed.
• Prompt speed: not measured; no inference path changed.
• Tests run: zig build test test-golden - 356/356 steps, 4924/4945 tests passed (21 skipped). The server module alone is 507/507. zig fmt --check zig and git diff --check clean. python3 tools/zig/lean_check.py - 3339 files, 444 findings at both d1018b318 and 8009df42a, the identical set (0 added).
• Goldens: translated-protocol zig/tests/server/golden/cases.json - 306/319 equal to server goldens, 0 unexpected differences, with 13 declared KNOWN-DIFFER cases, the same set as at base. Native HTTP security goldens - 26/26 equal to server goldens, 0 unexpected differences.
• Refusals: streaming via errors/stream-context.json (200 + SSE begin + context_length_exceeded event); nonstreaming via errors/context-prompt.json and errors/context-reply.json (400 JSON, unchanged path). Cancellation via cancel/client-leaves.json and cancel/metrics-after.json, plus the research cancellation tests; the preparing count returns to zero.
• New regression: test "research expansion keeps a context-window refusal streamed" drives 2000 and 2400 reply-token windows. With only the guard hunk reverted in the same tree, zig build test fails exactly at cortheon_test.zig:369 on try std.testing.expect(got.opened); with the guard the module is 507/507.
• Not run: CUDA, Linux host tests and dist (no CUDA host here); any GPU kernel or whole-model/checkpoint qualification (no kernel or model-path change).

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. Not applicable: no model, lane, cache or draft logic changed, and the suite runs unchanged.
• No precision traded for speed. No sum or bit changes.
• Prompt processing is no slower than the last release. No inference path changed.
• The description names the Metal platform I ran (M3 Ultra, -Dcpu=apple_m1) and what I could not run (CUDA, Linux, dist).
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thank you, @akol1, for preserving admission's HTTP 503 across Chat Completions, Messages and Responses.

At a801b9a, the common error mapping is right. One status case remains: core/lane_host_status.zig:23 reports healthy when the host is closing, although submit refuses it. Also, the new readiness callback is installed only in the test so far; the intended backend still needs to publish its readiness.

Please include closing in the health snapshot and connect the intended backend's nonblocking readiness flag. We can fold these small changes in at landing, keeping the released health identity, memory and authenticated status-only behavior.

We'd add route-level health tests with keys on/off, streaming and nonstreaming 503 cases for all three protocols, and rerun the frozen server corpus with its documented differences. This can join the next server-fix batch once those checks pass.

Posted by Ash's AI Assistant Synthena

[cwschroeder]

Requesting changes on 91f9f4bf: the streamed admission and legacy closing cases still disagree with the new health/503 contract.

• With stream:true, openai.zig opens output before chat.generate calls engine submit. Expanding your admission matrix gives six failures: Closed/Busy across Chat Completions, Messages and Responses each have opened=true, no whole-reply status, and three events. All six nonstreamed cases return 503. These probes run the request translators and shared handler with recorded output callbacks. Please keep capacity refusal before opening SSE; preserve the separate context-window streaming policy.
• The legacy Flash Next host's real engine.drain() leaves status healthy even though the next submit returns Closed and its queue is empty. Its statusFn inherits healthy=true. The diagnostic fails without GPU/model access. Adding .healthy = !h.closing under its existing status mutex makes the native-only control pass: 43/43 steps, 26 tests passed, one skipped.
• The only host.health assignment remains in lane_host_health_test.zig. The intended production backend's nonblocking readiness flag requested above still needs wiring.

I also added the missing unavailable-health socket matrix. All four keys-enabled/disabled and valid/missing-authorization cases return the exact minimal 503 body.

The diagnostic tests, one-line closing fix and observed output pin this head. Zig 0.17.0, M5 Max, macOS 26.6.2. The full diagnostic build reports three failed test executions for the two distinct regressions, since the server suite runs through two roots. Current CI is green with its narrower coverage. No device qualification is claimed.

[akol1]

Thank you for the concrete regressions. I reproduced both failures before the fixes and preserved your diagnostic tests and native closing fix as separate commits with your authorship.

Updated head: 8f87428802834954826d3457f9d8edac814aefd7; main base: b090e3b19014bfa1eb2b7f38ea3cc4d39abec9be.

• Streaming now opens only after successful engine submission, with the cancellation/mailbox-drain guard already installed. Closed/Busy admission retains HTTP 503 for streamed and nonstreamed Chat Completions, Messages and Responses. Context-window refusals retain their existing streamed error policy.
• The legacy Flash Next host reports healthy = !closing under its existing mutex.
• The native CUDA host now forwards an optional family healthy callback into production lane health/admission. The callback reads thread-safe host state without CUDA binding or device waits; families without it retain existing behavior. A CPU test exercises the actual host handoff and atomic availability transitions.
• Regressions also cover all four unavailable-health key/auth combinations, cancellation and drain when opening fails after admission, research progress before final admission, and compaction followed by final admission before opening.

Validation on macOS 26.6.2 arm64 with Zig 0.17.0:

• zig build test test-native test-golden test-kernel-contract -Dcpu=apple_m1 -j2 --summary all: 216/216 steps succeeded, 2534/2542 tests passed, 8 skipped.
• zig build native -Dcpu=apple_m1 -Dversion=1.0.0 -j2 --summary all: 45/45 steps succeeded; output not executed.
• zig build fuzz-cortheon -Dcpu=apple_m1 -j2 --summary all: 3/3 steps and 7/7 tests passed (built-in deterministic seeds and 100,000 structure-aware edits; no extra coverage-guided inputs).
• Linux x86_64 compile-only: 61/61 steps succeeded using explicit placeholder CUDA assets; outputs were never executed and must never be deployed. This proves source compilation only, not CUDA kernel/runtime qualification.
• Frozen server corpus: 309/319 equal, 10 documented known differences, zero unexpected differences; fixtures unchanged.
• Formatting and diff checks pass. Lean: 517 inherited findings versus the same 517 on the main base, zero added.

The new regressions are CPU-only. The standard macOS host workflow also exercises its existing Metal tests; no full-model or CUDA device qualification is claimed.

[cwschroeder]

At 8f874288, the three earlier source findings are addressed, but one context-stream regression remains after Cortheon research. I have an applicable small patch and actual regression at 393e5c7d, composed on current main 6ee828c2 with your branch merged unchanged.

The scripted engine admits the planner and completes a web search at context windows 2000/2400 with max_tokens:1. When research enlarges the final prompt beyond that window, the new unopened catch returns HTTP 400 with no SSE events. Initial context refusals still use the streamed error policy. The regression fails before the fix and passes afterward in Debug and Fast. Only Refused + context_length needs streamedContext before unsent; capacity errors retain HTTP 503.

Independent current-head checks pass: CPU server 377/377, CUDA-host 647/649 (two skips), native host 26/27 (one skip). Restoring the old streaming implementation reproduces the six admission failures; removing only the closing-health assignment reproduces the drain failure. With the offered fix, the CPU server passes 378/378. Whole-tree format, whitespace and lean (517/517, zero new) pass; Astra reviewed the bounded fix with no further finding. Your four current-head CI checks are separately green.

Complete candidate CI is green at 393e5c7d: the Linux No GPU workflow ran zig build test, test-kernel-contract, test-golden, fuzz-cortheon and lean (517/517, zero new). Whole-tree format and both app/mobile workflows also pass. The draft validation PR isolates these two files on f622d935; it is an adoption aid for this PR.

The first PR desktop run failed in the unchanged guest.test.ts output-limit case with Operation not permitted; its failed-job rerun passed without source changes. The same commit’s push desktop run and the unchanged composition base’s app workflow had passed already. Thirty focused base probes did not reproduce that error, so its cause remains undiagnosed.

Scope correction to the older linked diagnostic: its full macOS zig build test command includes existing Metal device tests. Only the individual probes are CPU-only. The README now says so. This current recheck used the selected host targets above, with no model, CUDA/Metal device workload or deployment.

[cwschroeder]

[P2] Preserve the streamed context refusal after research

The unopened catch sends every failure to unsent. Research can enlarge the final prompt after the initial prepare: with the existing scripted Cortheon Rig at a 2000/2400 window and max_tokens 1, the planner and search complete, but final prepare produces context_length and the request becomes HTTP 400 instead of the documented context_length_exceeded stream. The attached actual regression fails here. Before unsent, route only Refused + context_length through streamedContext; keep Closed/Busy on the HTTP 503 path.

[akol1]

@ashhart — closing the two items from your 2026-10-16 comment, plus the review question about HTTP 503 versus a streamed refusal.

Health reports the intended backend, not just the lane. A drained Flash Next host now reports unhealthy (healthy = !closing) in its host status, instead of looking ready until the next submit fails; the status call reaches that bit before a request is attempted. Native families can also export healthy(ctx) bool, which the CUDA host forwards into lane health and admission without binding a device context or waiting on device work — the callback contract (thread-safe state reads only, no GPU binding, no blocking work, no inference lock) is documented in docs/recipes/adding-a-zig-family.md. Families without the callback keep the lane's existing closing and stale checks. /health answers a minimal 503 for the unavailable state, covered with keys enabled or disabled and with or without a valid key.

The rule for 503 versus a streamed refusal is admission. Closed and busy admission are refused before any stream opens, so Chat Completions, Messages and Responses return HTTP 503 for both states, streaming or not; the preflight test runs both reasons across all three routes and both body kinds, and requires 503, no opened sink and a zero preparing count in every case. After admission the answer is in-band by contract: a context-window refusal is an SSE context_length_exceeded event, never a whole-reply status. 3bb0b759 covers the remaining path that broke this rule: a research turn that compacts, passes compaction and then refuses at the final context-window check before the sink opens used to return a JSON status; the regression drives both research windows and asserts the search ran, the stream opened, no whole-reply status or body followed, the stream carried context_length_exceeded, and preparing returned to zero.

Authorship. f859ed4e and e8f50daa are @cwschroeder's diagnostic tests and native closing bit; 3bb0b759 is @cwschroeder's late-context patch, carrying the authorship from the review comment; 8f874288 is the admission and readiness wiring. Head is 3bb0b759410f66163edce4f90b14c010c175a2e9.

Receipts at the head. zig build test -Dcpu=apple_m1 — 206/206 steps, 396/397 tests passed (1 skipped); test-native test-golden test-kernel-contract — 50/50 steps, 309/319 frozen answers equal with 10 documented differences and 0 unexpected; native — 45/45; fuzz-cortheon — 7/7 tests; the plain build — 179/179; test-server-cpu — 378/378; zig fmt --check and git diff --check clean; the lean checker reports 517 problems, the same count as base b090e3b1 (0 added). All four checks on the head are green. Reverting only the openai.zig hunk of 3bb0b759 fails the new regression at cortheon_test.zig:369 (372/373) while the head passes at 378/378.

Not covered. No CUDA, Linux or dist run, and no whole-model or checkpoint qualification; this change touches no kernel or model path. The route-level health tests are in health_test.zig and stream_preflight_test.zig; if you had different cases in mind, name them and I will add them.

[ashhart]

Thank you, @akol1, for exercising admission and health through the actual server boundaries.

We merged your work for 1.1.0 in 56cce23. Requests are admitted before a streaming response opens, cancellation cleanup is installed first, and closing hosts report unavailable. The native CUDA host forwards a family's optional readiness callback.

The current health/auth/memory schema stays in place; each family still owns connecting its fatal state to readiness.

Posted by Ash's AI Assistant Synthena

[akol1]

Rebased onto 76aafcf27 so this PR shows its real delta: one commit (bf756299c, author Christian Schröder) and two files, +27/-1.

That is all of #912 the 1.1.0 merge did not take. 56cce2330 merged the branch at 8f874288; 3bb0b759 was left behind, so git cherry origin/main lists only that commit. The 19-file diff before the rebase came from the old base b090e3b, not from this PR.

At the new head: test-server-cpu 434/434 (429 + 5), zig fmt --check and git diff --check clean, lean checker 2625 files / 503 findings, identical to base 76aafcf27 (0 added). With only the four-line openai.zig hunk reverted the new test fails at cortheon_test.zig:369 on expect(got.opened) (428/429); with it, 429/429. git range-diff shows the rebased commit is patch-equal to the reviewed one, and fmt/mobile are already green on the new head. Body updated with the scope and receipt.

[akol1]

zig-windows fails on this head for the base's reason, not this diff. 76aafcf27 breaks 15 Windows build steps in main's own files: zig/src/server/speech.zig:279, zig/src/vision/qwen_cuda_test.zig:273, and the flashnext CUDA test roots (cuda_lanes_restore_test.zig, cuda_ple.zig, cuda_rank_media_test.zig, cuda_rank_wire.zig). #1034 ("Windows: main's host tests build again on Windows hosts") fixes exactly those files, lists them in its diff, and has zig-windows green on the same base.

fmt, mobile, desktop and zig are green here. This diff touches openai.zig and cortheon_test.zig; nothing in it can change the Windows build. The job goes green for every PR based on 76aafcf27 once #1034 lands, and I can rebase this after that if you prefer the head to be green before the merge decision.

[ashhart]

Thanks @akol1 for isolating the unlanded change, and @cwschroeder for the fix and regression. The late research context-window refusal now follows the existing streamed error path; other unopened failures keep their HTTP status. This is a source-ready 1.1.2 candidate, and the PR stays open until it lands. Its merge must preserve the current pre-token keepalives. Please rerun the server/preflight and translated-protocol golden checks on that merged source, including streaming and nonstreaming refusals, cancellation and preparing-count cleanup. The reported Mac host result is useful; no new GPU or model qualification is needed for this routing-only change.

Posted by Ash's AI Assistant Synthena

[akol1]

Post-rebase gates on d1018b318

Branch refreshed: base d1018b318, head 8009df42a, one commit by Christian Schröder. git range-diff 76aafcf27..bf756299c d1018b318..8009df42a shows the only delta from the reviewed bf756299c is adopting main's pre-token keepalive sink (live.out / live.every_ns, and live passed to cortheon.loop.stream and chat.generate); the guard and the regression are unchanged. Those keepalives are preserved in the merged source: against d1018b318 the diff is the four-line guard in openai.zig plus the new test.

Host: M3 Ultra (80-core GPU, 512 GB unified memory), macOS 26.6.2, Xcode 27.0, Metal toolchain 27.1.266.1, Zig 0.17.0 from .zig-version, -Dcpu=apple_m1. No checkpoint, weights or GPU path is involved; the suites drive the stub engine.

• zig build test test-golden - 356/356 steps, 4924/4945 tests passed (21 skipped), exit 0.
• Server module (includes the regression) - 507/507 when run directly.
• Translated-protocol goldens zig/tests/server/golden/cases.json - 306/319 equal to server goldens, 0 unexpected differences; the 13 KNOWN-DIFFER cases are the same declared set as at base and unrelated to this change.
• Native HTTP security goldens zig/tests/server/golden/security-cases.json - 26/26 equal to server goldens, 0 unexpected differences.
• Streaming refusal: errors/stream-context.json - 200 + SSE begin, then the context_length_exceeded error event. Nonstreaming refusals: errors/context-prompt.json and errors/context-reply.json - 400 JSON, path unchanged.
• Cancellation: cancel/client-leaves.json and cancel/metrics-after.json, plus the research cancellation tests; the preparing count returns to zero.
• New regression: drives 2000 and 2400 reply-token windows and asserts the refusal event and preparing back to 0. With only the guard hunk reverted in the same tree, zig build test fails exactly at cortheon_test.zig:369 on try std.testing.expect(got.opened); with the guard the module is 507/507.
• zig fmt --check zig and git diff --check clean; python3 tools/zig/lean_check.py - 3339 files, 444 findings at both d1018b318 and 8009df42a, the identical set (0 added).
• Not run: CUDA, Linux and dist (no CUDA host here); no GPU or whole-model qualification, as this is routing-only.

No checkpoint revision applies here.

[ashhart]

The remaining streaming research fix is included in 1.1.2 (https://github.com/ashhart/TensorFold/releases/tag/v1.1.2): late context-length refusals stay on the response stream, including while waiting for the first token. The earlier health reporting and HTTP 503 admission behavior remain in place. Closing this as carried, with both contributors credited as coauthors. Thank you both.

It landed as 27dc69b.

- Synthema, Ash's AI Assistant.