What happens today. When a /v1 request is routed to a peer, the routing node picks the peer. No request header or body field lets a client say "send this to X" or "not to Y" — the only routing knob is the sticky hash on prompt_cache_key / user. A client that wants to send the same deterministic request to two named peers and compare the answers (a cheap consistency check across providers) cannot do it; it also cannot learn which peer actually answered without parsing provenance.
Proposed change (host). Two optional request headers on the remote-mesh routing branch, both no-ops when absent:
x-mesh-target: <EndpointId> — dispatch to exactly that peer if it currently advertises the requested model; otherwise fail closed with a 409 naming the mismatch, never silently rerouted to another peer or served locally. Checked after exclusion, so excluding your own target also fails closed.x-mesh-exclude: <EndpointId>[,...] — remove peers from the candidate set before selection.
The routing node echoes the resolved peer as x-mesh-served-by: <EndpointId> on the response, only when x-mesh-target was used. Malformed or ambiguous values are a 400, never ignored. Absent both headers, request and response are byte-for-byte identical to today.
Acceptance. Target a peer that serves the model → served by that peer, response carries x-mesh-served-by. Target a peer that doesn't → 409, no fallback. Exclude the only candidate → 409. No headers → unchanged bytes. Existing routing tests unchanged.
Out of scope here. A single-request "send to two peers, return both" flag. With these two headers a client already gets that as two calls; the one-request form can sit on top later if wanted.
PR: stacked on #1668 (same file); only the top commit is new. Live results (multi-node run, test output) are on our side — happy to attach here on request.
What happens today. When a
/v1request is routed to a peer, the routing node picks the peer. No request header or body field lets a client say "send this to X" or "not to Y" — the only routing knob is the sticky hash onprompt_cache_key/user. A client that wants to send the same deterministic request to two named peers and compare the answers (a cheap consistency check across providers) cannot do it; it also cannot learn which peer actually answered without parsing provenance.Proposed change (host). Two optional request headers on the remote-mesh routing branch, both no-ops when absent:
x-mesh-target: <EndpointId>— dispatch to exactly that peer if it currently advertises the requested model; otherwise fail closed with a 409 naming the mismatch, never silently rerouted to another peer or served locally. Checked after exclusion, so excluding your own target also fails closed.x-mesh-exclude: <EndpointId>[,...]— remove peers from the candidate set before selection.The routing node echoes the resolved peer as
x-mesh-served-by: <EndpointId>on the response, only whenx-mesh-targetwas used. Malformed or ambiguous values are a 400, never ignored. Absent both headers, request and response are byte-for-byte identical to today.Acceptance. Target a peer that serves the model → served by that peer, response carries
x-mesh-served-by. Target a peer that doesn't → 409, no fallback. Exclude the only candidate → 409. No headers → unchanged bytes. Existing routing tests unchanged.Out of scope here. A single-request "send to two peers, return both" flag. With these two headers a client already gets that as two calls; the one-request form can sit on top later if wanted.
PR: stacked on #1668 (same file); only the top commit is new. Live results (multi-node run, test output) are on our side — happy to attach here on request.