Add a gRPC wire beside the websocket for inference sessions - #728
Add a gRPC wire beside the websocket for inference sessions#728v-positronic wants to merge 46 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27633ace1c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27e0f042e2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60e3960b05
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 740b67efbb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa7c87bc16
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f1b11063a7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fa3b12f87
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66c25367cb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 617be708d2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
`positronic/offboard` serves a session over two wires now. `wire.py` holds the seam and the websocket wire; `grpc_wire.py` holds the gRPC one. Every frame stays the same msgpack message, and a wire carries it as opaque bytes, so `protocol.py` does not change. A gRPC session is one bidirectional stream. A generic handler with no serialiser carries the frames, so there is no protobuf schema and no generated code. The session path, the query and the `Authorization` header cross as gRPC metadata, so both wires accept the same model ids and the same session params. `grpc://host:port/api/v1/session/<model_id>` selects the wire, and `PolicyServer(grpc_port=...)` serves it beside the websocket one. Both wires run on one event loop, so they share the model slot, the inference lock and the idle watchdog. Python's websocket stack costs about 30 ms per 846 KiB observation, which gRPC does in about 1 ms. The websocket wire stays the default, because a managed HTTPS front drops the HTTP/2 frame detail gRPC needs. Ticket: none - a transport measurement with no ticket; the numbers are in the pull request
The writing rules refuse a sentence that opens on its reason. Ticket: none - a one-line prose fix the style gate named
Cut the cold-backend comment to the fact a reader acts on, and drop the HTTP/2 rationale the offboard README already carries. Ticket: none - a comment pass on the change in flight
gRPC's target syntax needs an IPv6 literal in brackets, so a server hosted on `::` bound `:::<port>` — which gRPC refuses by returning port 0, and the server then started, accepted nothing and said nothing. The bind reports a refusal now instead of starting deaf. Ticket: none - a review finding on an unmerged branch
…shake refuses Three findings from the review round. The 16 MiB ceiling was gRPC's alone and matched uvicorn's `ws_max_size` default by coincidence, so a move in that default would have parted the two wires silently. It lives in `wire` now and both the gRPC options and `uvicorn.Config` are passed it. A session the server refuses in a protocol frame — an unknown model, a session param it rejects — raises past every transport handler, and a gRPC connection holds a reader thread until it is closed. `_open_session` owns one attempt and closes the connection whenever the handshake does not finish. The query test reads `LOCAL_STACK` and `SEQ` rather than spelling them. Ticket: none - review findings on an unmerged branch
An authenticated endpoint is served by a TLS-terminating front, so `grpcs://` dials the gRPC port through one: the client verifies the edge against its own roots and the server keeps its plaintext port, which is what every front hands an HTTP/2 stream to. A `grpcs://` URL defaults to port 443 like the other TLS schemes. A front also drops a connection it has read nothing from, and one inference sends nothing until it answers, so the client pings through the silence. Three of gRPC's own defaults stop that working and each is set here: the client sends two pings and stops, it spaces them five minutes apart, and the server answers a 20s ping with `GOAWAY too_many_pings`. A certificate the client's roots do not cover now raises the exception the websocket wire raises, so the connect loop reads it as permanent instead of retrying its whole deadline out. gRPC's readiness future says only that the channel is down, so one call on it fetches the reason. Ticket: Positronic-Robotics/internal#1191 #refs
It answers one constant chunk, so a round trip measures the wire and nothing else. `?delay_sec=120` holds an inference open for two minutes, which is what a cold model does to a connection with nothing crossing it meanwhile. Ticket: Positronic-Robotics/internal#1191 #refs
The endpoint now exposes port 9000 beside 8000 and prints the `grpcs://` URL for it. The gRPC port is declared as an ordinary HTTP port: that front negotiates HTTP/2 over ALPN and carries a gRPC session end to end, where a port declared `/tcp` gets a `tls://` URL whose front negotiates no ALPN at all and gRPC refuses it. The README records what a round trip costs through such a front, and where the front stops being the faster wire's problem. Ticket: Positronic-Robotics/internal#1191 #refs
…sked for Three review findings. A front that selects no HTTP/2 over ALPN refuses every gRPC client, exactly as an unverifiable certificate does, and both answer `UNAVAILABLE` like a cold backend — so the connect loop reads the status details rather than the code alone, and stops. The connection raises what gRPC blamed instead of a bare timeout, which is what carries those details. A test edge that selects no protocol pins the new case; it binds one address, because gRPC reports the last address it failed on and a second family refusing the connection hides what the first blamed. `serve.sh` exposed port 9000 whatever the caller asked the server to listen on. It now reads the port back out of `--grpc_port` when one is given. The test proxy caught every `OSError`, which hid a broken edge behind the reset an ending session is expected to raise. It now catches the two that end a session. Ticket: Positronic-Robotics/internal#1191 #refs
Two review findings. A timed-out inference closes the connection, and the server can still answer inside the five seconds `close` waits for the stream to end. That reply sat in the inbox, where the next `infer` would have read it — one observation's actions against the next observation's state. `send` and `recv` now refuse a closed session. `channel_credentials` wrapped one call and had one caller, so it was a production name held open for the tests. It is inlined, and the tests substitute roots on gRPC's own function. Ticket: Positronic-Robotics/internal#1191 #refs
Three review findings. gRPC stops reading the request iterator once a stream ends, so a send after the caller has been told why it ended sat in the outbox while the next `recv` waited out a whole inference timeout on an inbox nothing refills. The connection records the end when it hands out the reason, and refuses a send past it. The websocket client kept the library's 1 MiB receive default while the server, the gRPC client and the gRPC server all carry the shared 16 MiB ceiling, so a result in between closed the session rather than answering it. `serve.sh` printed `grpcs://:443` when the endpoint served no URL for the gRPC port. It fails there instead. Ticket: Positronic-Robotics/internal#1191 #refs
`channel_ready_future` and the probe that asks why it failed each took the whole `open_timeout`, so a target that black-holes connection attempts blocked for nearly twice the documented limit before the connect loop saw a status. Both now run against one deadline, with a second reserved for the probe. Ticket: Positronic-Robotics/internal#1191 #refs
The scheme was dispatched as a closed set of strings, and `grpcs` was spelled a second time to recover its TLS half. A scheme added to one tuple and not the other would have left the wire and the security disagreeing. One enum member now carries both, and the parse fails on anything it does not name. Ticket: Positronic-Robotics/internal#1191 #refs
The create declared `--container-port 8000` and the poll selected `https://port8000-`, so a port change could leave the reader looking for an endpoint creation never made. Ticket: Positronic-Robotics/internal#1191 #refs
The `InferenceClient` section named `https` and `wss`, so adding `grpcs` left it wrong: the default is 443 whenever the scheme is TLS. The rule cannot go stale as schemes arrive; the wire table above it names which they are. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 569d4dc63b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
The header claimed gRPC on 9000, while `--grpc_port` decides it and 9000 is only the default. "Container port" was the same claim one line down; the websocket port is the one this script fixes. Ticket: Positronic-Robotics/internal#1191 #refs
The `--policy.url` section named `http`, `ws` and `wss` and stopped there, so the wire this branch adds was absent from the one doc a user reads to reach a server. The offboard README and `InferenceClient` already list `grpc(s)`; this was the third copy and the only stale one. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0b9ab74e0b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
The guide opened by calling remote inference WebSocket-only and labelled the offboard README "WebSocket protocol", so its overview contradicted the URL schemes further down. One protocol, two wires, said once in each place. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c84de5cd91
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
`PolicyServer` held each wire's bind, its accept-time auth and its model-id extraction: a FastAPI app and two routes for the websocket, `grpc_session` and `_start_grpc` for gRPC. `wire.ServerConnection` already covered an open session, so the inference loop was transport-free while opening a session was not, and a third wire cost five edits to the server. `wire.Wire` covers the other half: bind, accept, refuse and stop. Each wire reads its own route for the model a session names and checks its own session headers. `serve` takes the list of them, and the CLI entry point is where the two wires are named. The handshake meta names the port that carried the session. A gRPC session reported the websocket's port before. The test fixture serves through `PolicyServer.serve` rather than rebuilding it, which removed a second copy of the wire lifecycle and gave the idle watchdog its first test. Ticket: Positronic-Robotics/internal#1191 #refs
`serve` gathered the wire tasks with `return_exceptions=True` and read none of the results, so a wire that failed as it stopped was discarded without a line in the log. One failure now raises out of `serve` and every other one is logged at ERROR. `STOP_GRACE_SEC` moves into `WebsocketWire`, which is its only user, and the idle test's own constant moves above the test that reads it. Ticket: Positronic-Robotics/internal#1191 #refs
The class docstring restated what a wire reads and checks, which `wire.Wire` and the offboard README both already say. It names what this server does with a wire instead. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14a2c3861e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 71609909a1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
`PolicyServer.default_session` and `model_session` are gone, and the two lerobot vendor suites still called them. Both skip without their vendor package, so a local run reported them as skipped and CI failed on them. Each now builds the connection the websocket wire builds and calls `_serve_session`, which is the same handler the wire reaches. They are the only two callers the removal left. Two comments in `_listening_socket` said `is what` where the verb does the work. Ticket: Positronic-Robotics/internal#1191 #refs
`serve` started every wire before it entered the cleanup `try`, so a `GrpcWire` that bound stayed bound when the `WebsocketWire` after it found its port taken: a live listener in front of a server nothing serves. Startup runs inside the `try` now, and the cleanup stops the wires that actually started. The status-update section said the frames prevent a WebSocket keepalive timeout. They hold the client's 30s-per-message handshake open, which is the same on either wire. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 43536c7aa0
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
A wire binds its listening socket in `start`, and uvicorn takes it over only when `serve` runs. A startup that rolls back stops a wire that bound but never served, so `stop` set uvicorn's `should_exit` on a server that never ran and the socket kept its port. Close the socket in `stop` where the wire never served. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55bb7f4a5b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
`wire.py` keeps the protocol every wire implements. `websocket_wire.py` holds the websocket wire and the two ends of a websocket session, beside `grpc_wire.py`. Ticket: Positronic-Robotics/internal#1191 #refs
Each comment and docstring the gRPC wire adds states one fact, in one or two lines. A sentence that gave a reason, weighed an alternative, or told the reader how to read a fact is gone. A docstring that repeated its test name is gone. Ticket: Positronic-Robotics/internal#1191 #refs
The inference guide states the GPU split as a fact. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c01b1d3cfa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…at holds The name says what the tuple holds, and the comment above it goes. Five passages predate the pull request and sit in files it touches. Each one carried design rationale or ran far past the docstring budget. The rationale is in the offboard README and in `workflows/nebius/README.md`, next to the subject it explains. The passages keep each constraint and each footgun: - `PolicyServer`: what it serves, the two halves of the marker, the fixed source, session params, and the pinned default checkpoint. - `serve`: the token comes from the environment, a flag exposes a secret. - `InferenceClient`: the URL forms, the gRPC port, the headers, the timeouts. - `serve.sh`: the managed URL changes on re-create, and the `--auth token` ingress strips the WebSocket upgrade headers. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9e66e2862
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
`wire.Refusal` names what a refused connect says about the server, and `wire.ConnectRefused` carries it with the library error as the cause. `websocket_wire.dial` and `grpc_wire.dial` open a client's connection and classify what their library refuses. The gRPC wire also classifies a status that ends the stream before its first message, and the websocket client end raises `PeerDisconnected` for a closed connection. `InferenceClient.new_session` catches `ConnectRefused`, `TimeoutError` and `PeerDisconnected`, spends the refusal against the retry budget, and keeps the connect deadline. It raises `ConnectRefused` where it raised the library exception. `client.py` imports neither `grpc` nor `websockets`. Ticket: Positronic-Robotics/internal#1191 #refs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9738f0c6a9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
A status that ends the stream before its first message is the server refusing the call. A status after one is a lost peer, and the wire raised the `grpc.RpcError` itself. `InferenceClient.new_session` retries `ConnectRefused`, `TimeoutError` and `PeerDisconnected`, so that error escaped a cold start the websocket wire retries: the websocket client end raises `PeerDisconnected` for a closed connection whether or not a frame crossed. The gRPC wire now raises it too, with the status as the cause. Ticket: Positronic-Robotics/internal#1191 #refs
A failed session tells the client before it cleans up. That send reaches a peer which may already be gone, and one blanket catch recorded every outcome at DEBUG. A peer that has gone raises `PeerDisconnected` and stays at DEBUG. Any other failure names the peer at ERROR, so a delivery path that breaks for another reason is not silent. Ticket: Positronic-Robotics/internal#1191 #refs
`PolicyServer` takes no `host` or `port`, and `serve` takes the wires it serves on, so the custom-model example in `docs/connect-your-model.md` raised `TypeError`. It now builds the server, then serves it a `WebsocketWire` over `server.api`. The offboard README says every wire carries the same frames, while its session endpoints were named and shown for the websocket alone. The headings name the path, and the examples show both schemes. Ticket: Positronic-Robotics/internal#1191 #refs
1 participant
A session runs over a websocket today. This adds a gRPC wire beside it — an open port on a
LAN, and an authenticated endpoint behind a TLS-terminating front — and puts both wires
behind one interface.
What it ships
The server takes the wires it serves on.
wire.Wirecovers what a transport doesoutside an open session: bind, accept, refuse an unauthorized peer, stop.
WebsocketWireand
GrpcWireimplement it, each in its own module, andPolicyServer.servetakes thelist. A third wire costs a class and a flag. Each wire reads its own route for the model a
session names — a path parameter on one, session metadata on the other — and checks its
own headers.
server.apiholds the model catalogue, which a wire that speaks HTTP serveson the port it carries sessions on.
ServerConnectionalready covered an open session, so the inference loop wastransport-free while opening one was not. The handshake meta now names the port that
carried the session, where a gRPC session took the websocket's before.
serve(wires, on_ready=...)reports when every wire has bound; a wire asked for port 0can be dialled.
The client names no wire either. Each wire dials its own session (
websocket_wire.dial,grpc_wire.dial) and raises onewire.ConnectRefusedthat says what the refusal means:cold, forbidden, or final. The connect loop retries those and a lost peer, and
client.pyimports neither library.
new_sessionraisesConnectRefusedwhere it raised a library's ownexception; a bad certificate is final on both wires.
grpcs://dials the gRPC port through a TLS edge. The client verifies the edge againstits own roots and the server keeps its plaintext port — every authenticated shape puts a
front in the way, and a front hands the server an HTTP/2 stream. One enum member carries a
scheme's wire and its TLS half together; a scheme added later cannot leave the two
disagreeing.
Pings through a silent wait. A front drops a connection it has read nothing from — the
Nebius one after about 90 s — and one inference sends nothing until it answers. The gRPC
defaults space two client pings five minutes apart and answer a 20 s ping with
GOAWAY too_many_pings; the options here set both.An edge no client can use is permanent. An uncovered certificate and a front selecting
no HTTP/2 over ALPN both answer
UNAVAILABLE, as a cold backend does. The connect loopreads the status details and stops.
A wire-only stub server (
positronic.offboard.stub) answers one constant chunk; around trip measures the transport alone.
workflows/nebius/serve.shserves both wires andprints the
grpcs://URL.The front has to negotiate HTTP/2 over ALPN
Declare the gRPC port as an ordinary HTTP port and dial its
https://host asgrpcs://<host>:443. A port declared/tcpis fronted by atls://URL that negotiatesno ALPN, and gRPC refuses it with
Cannot check peer: missing selected ALPN property.What the smoke measured
A CPU endpoint running the stub on a live Nebius Serverless Endpoint, over its managed
https://front. Agrpcs://session infers an 846 KiB observation in 8 ms with thebearer in its metadata, and answers
PERMISSION_DENIEDwithout it. A session idle 150 sstill infers, and
?delay_sec=120answers after two minutes — both past the front's idleclose.
Round trip against the websocket, same front: 1.8 ms against 6.2 at 63 KiB, 3.0 against
19.2 at 255 KiB, 6.0 against 61.3 at 846 KiB. A back-to-back loop of 846 KiB settles at
about 83 ms after some 11 MB and recovers after a minute of quiet: the front shapes a
session that sends faster than it forwards. The websocket holds 60 ms throughout, below
that rate.
CI carries the production shape: a TLS terminator in the test process, the bearer refused
without it, an edge with no ALPN and one with an unknown root, and the keepalives. Both
wires answer one observation alike apart from the endpoint, each naming its own port.