Repository navigation
Encrypt NEAR completions requests and responses - #227
Conversation
Teep had no route for /v1/completions, so text completions on NEAR did not work through teep. NEAR's model hosts encrypt this endpoint: on 2026-11-14 a completion with an encrypted prompt and NEAR's E2EE headers returned choices[].text encrypted to the client key, as JSON and in each SSE chunk. NEAR's gateway answers an encrypted completion with "End-to-end encryption is not supported on /v1/completions", so only neardirect gets the route. Changes: - EncryptCompletionsNearCloud encrypts prompt, a string or an array of strings. Besides it, a request may hold only sampling and limit fields; logprobs, prompt_logprobs and suffix only as null, and echo and return_token_ids only as null or false. Any other field, a prompt of token ids or an empty prompt array fails with ErrCompletionsShape before the request is sent: its content would travel or return unencrypted. An integer 0 still turns logprobs on. - The relay decrypts choices[].text of a completions response, as JSON and in SSE chunks. Each choice must carry an encrypted text; besides it only structural metadata, and logprobs, prompt_logprobs, token_ids and prompt_token_ids only as null. Anything else fails closed. A NEAR completions stream must end with [DONE], as a chat stream must. - /v1/completions is routed for neardirect and confirms e2ee_usable as chat does. nearcloud has no completions route. - The API support document states the field map. To verify: go test ./internal/e2ee -run Completions and go test ./internal/proxy -run TestNearDirectCompletionsE2EE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review from GLM 5.3: Full suite passes — 27 packages ok, zero failures, clean build. Here's my review. VerdictNo major security issues. This one is solid. The design follows the codebase's established fail-closed pattern on both directions of the wire, and the test coverage actually matches the PR description's claims (I verified each listed case exists in the tests). What's done wellResponse side is genuinely fail-closed. Request side allowlist is strict. Consistency details check out:
Nits worth a comment (not blockers)1. 2. Duplicate JSON keys in the request body aren't rejected. 3. Nested-object members aren't validated. 4. The
5. Minor: Test verificationRan on the PR head (
Recommendation: approve. If the author wants to act on anything, the |
The completions encryptor decoded the request into a map, which keeps only the last of duplicate members. The allowlist check then saw that last value, while the upstream can read another one: with "logprobs":5,"logprobs":null the check saw null, and the request could ask for token log probabilities, which return unencrypted. The encryptor now runs jsonstrict.ValidateUniqueFields on the body first and refuses a duplicate member at any depth, also one spelled with another JSON escape, with ErrCompletionsShape. Nothing is sent. To verify: go test ./internal/e2ee -run TestEncryptCompletionsNearCloudRefuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The completions row of the encrypted-field map listed `stop` among the sampling fields and did not name `user`. Both pass in plaintext: `stop` holds stop sequences, which can hold text, and `user` is an end-user identifier. The row now names them, says that every accepted field other than `prompt` passes in plaintext, and lists the refusal of a duplicate member. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thank you for the review. Two of the notes are now in the branch:
Not changed: On the model-lister note: this pull request does not touch the lister. The code you saw is #220, which is in this branch's base and was reverted on main by #229 after NEAR stopped serving the model. Merging this branch keeps the revert. |
Problem
Teep had no route for
/v1/completions, so text completions on NEAR did not work through teep.NEAR's model hosts encrypt this endpoint. On 2026-11-14 a completion sent to a NearDirect model host, with an encrypted
promptand NEAR's E2EE headers, returnedchoices[].textencrypted to the client key, as JSON and in each SSE chunk. NEAR's gateway answers an encrypted completion withEnd-to-end encryption is not supported on /v1/completions.Change
EncryptCompletionsNearCloudencryptsprompt, a string or an array of strings.prompt, a request may hold only sampling and limit fields (max_tokens,temperature,top_p,stop,n,stream,seedand similar).logprobs,prompt_logprobsandsuffixare accepted only asnull, andechoandreturn_token_idsonly asnullorfalse; an integer0still turns logprobs on.ErrCompletionsShapebefore the request is sent, because its content would travel or return unencrypted. A new upstream field therefore fails closed.choices[].textof a completions response, as JSON and in SSE chunks. Each choice must carry an encryptedtext; besides it only structural metadata, andlogprobs,prompt_logprobs,token_idsandprompt_token_idsonly asnull. Anything else fails closed.[DONE], as a chat stream must./v1/completionsis routed forneardirectand confirmse2ee_usableas chat does.nearcloudhas no completions route, because NEAR's gateway does not encrypt it.docs/api_support.mdstates the field map.Effect
To verify
Each prints
ok. The tests cover:ErrCompletionsShape, includinglogprobs: 0,return_token_ids: trueand an unknown field, and the accepted "off" values;choices[].textin a JSON response and an SSE chunk, and the fail-closed result for a plaintexttext, a missingtext, token ids, logprobs and an unknown choice key;[DONE]requirement for a completions stream;e2ee_usableconfirmed.Checklist
make checkpasses.make integrationandmake reportspass.