The PTY path's turn signal is self-declared "inferred": no proof a model consumed a delivered message

[khaliqgant]

The gap

On the default PTY delivery path there is no observable signal that a recipient's model actually took a turn on a delivered message. The strongest thing available is that the injected text appeared in the recipient's terminal buffer.

Delivery is confirmed by echo verification: the broker scans recipient stdout for the literal injected string within a 5s window (crates/broker/src/broker/delivery_verification.rs). On a match the worker sends delivery_ack (crates/broker/src/runtime/worker_events.rs:470-524), which triggers mark_delivery_read_ack (crates/broker/src/runtime/delivery.rs:247-268) and calls mark_read_as_agent upstream.

So "read" is set by a string match on a terminal buffer. No model is in that loop. The code says so itself, at crates/broker/src/runtime/delivery.rs:196-198:

A read-ack means "delivered to the recipient location", not proof that a model turn cognitively processed the message.

That comment is correct, and the behaviour it describes is reasonable for its actual purpose — confirming transport. The problem is that nothing else exists, so "delivered to a terminal" is the ceiling of what any test or feature on this path can assert.

Why it matters now

This blocks the conformance evidence for #1474. The settled protocol (AgentWorkforce/c2a#4) requires that an unanswered blocking question keeps returning until its sender discharges it, and requires the evidence to assert the recipient took a model turn — deliberately, because asserting "an event was emitted" is satisfiable by an implementation that emits the return into the same stream the original message decayed into. That is the bug wearing a different hat.

On this path that assertion cannot be written honestly today. The available proxy is "did the agent subsequently call send_dm/post_message", which is what the existing integration tests use. That is evidence, not proof: an agent may emit a message for unrelated reasons, or process a message and correctly stay silent.

The contrast is that a real signal already exists on the other runtime. The native AI-SDK harness emits turn.settled (packages/harnesses/src/ai-sdk/harness-host.ts:492) once the model's generation actually completes, via the AI SDK's control.done. HarnessHost.startTurn (harness-host.ts:443-508) has the full turn lifecycle. But runtime: 'native' is opt-in and experimental for Claude/Codex/OpenCode, so the runtime carrying most traffic is the one that cannot report a turn.

Why this is the same defect class as #1474

#1474 exists because a well-formed signal stood in for a fact nobody verified — a message was read, and "read" was mistaken for "handled". Echo verification is the same shape one level down: a string match stands in for cognition, and a delivered-and-processed message is byte-identical to a delivered-and-ignored one from the broker's point of view.

The read-ack is not wrong to exist. It is wrong to be the only thing there is.

Asked for

A distinct, observable turn signal on the PTY path — something that means "the model consumed this message in a turn", separate from and additional to the existing delivery/read-ack. Shape is open; what matters is that it is:

• distinguishable from echo verification, not derived from it;
• attributable to a specific delivered message, so a test can assert this message caused that turn;
• observable from outside the worker, so integration tests and any future obligation tracking can consume it without scraping the PTY.

Acceptance

• A test can assert that a specific delivered message was consumed in a model turn, without inferring it from the agent later emitting a message.
• The signal does not fire when text is injected into a terminal that never reaches a model turn — that negative case is the one worth testing, and it is the case echo verification currently cannot distinguish.
• The existing read-ack path is unchanged; this is additive.

Notes

Related to #1474 (needs this for its conformance arms) and AgentWorkforce/c2a#4 (which specifies the assertion). Not a blocker for the schema half of #1474; it is a blocker for proving the behaviour on the default runtime.

Worth stating plainly: if this is not built, then conformance evidence for #1474 can only be produced on the native harness, and the default runtime will carry a feature whose central guarantee is unverified there. That is an acceptable interim position only if it is written down rather than assumed.

[khaliqgant]

Correction to this issue as filed — the claim was too strong, and the truth is worse

I filed this saying no turn signal exists on the PTY path. That is wrong. Correcting it, and retitling, because the accurate version is stronger evidence rather than weaker.

A turn signal does exist. crates/broker/src/runtime/worker_events.rs publishes turn boundaries through publish_pty_busy / publish_pty_idle, derived from stdout busy/idle transitions. What matters is how the broker describes them in its own capability report (worker_events.rs:377-379):

{"available":true,"fidelities":["inferred"]}

for thinking and idle, against "exact" for starting and error. The default soak comparison prints the same distinction: turns: {reference:["exact"], candidate:["inferred"]}.

So the system already models the difference between an observed turn and an inferred one, names it in a capability report, and the default runtime only ever has the inferred kind. The original framing — "there is nothing" — understates it. The real finding is that there is something, it is explicitly labelled as not-observed by the code that emits it, and it is nonetheless the only thing available to build on.

Two further constraints found while building the conformance fixture (#1476):

• These signals publish to the hosted stream, not to local getAgentEventHistory, which only ever holds native sidecar frames. So they are not reachable from a broker integration test as things stand, even if inferred fidelity were acceptable.
• Read state is auto-set on the native runtime too, not only on PTY: the sidecar acks deliver_relay and mark_delivery_read_ack fires from that same delivery_ack arm. So "read is set by infrastructure, not attention" holds on both runtimes.

That second point has a direct consequence for #1474's conformance evidence: arm D requires running the same scenario with read state set and unset and getting byte-identical behaviour. Read state cannot currently be left unset on either runtime, so the independent variable cannot be varied and the arm cannot be run — it is not merely unimplemented, it is currently unassertable.

The ask in the body is unchanged: a turn signal that is distinguishable from echo verification, attributable to a specific delivered message, and observable outside the worker. "Distinguishable" now also means distinguishable from the existing inferred one, and the acceptance criterion should be that it reports exact fidelity rather than inferred.