Skip to content

Encrypt NEAR completions requests and responses - #227

Merged
13rac1 merged 3 commits into
13rac1:mainfrom
ai-13rac1:near-completions-e2ee
Nov 14, 2026
Merged

13rac1 merged 3 commits into
13rac1:mainfrom
ai-13rac1:near-completions-e2ee

Conversation

@ai-13rac1

Copy link
Copy Markdown
Contributor

Problem

Teep had no route for /v1/completions, so text completions on NEAR did not work through teep.

NEAR's model hosts encrypt this endpoint. On 2026-11-14 a completion sent to a NearDirect model host, with an encrypted prompt and NEAR's E2EE headers, returned choices[].text encrypted to the client key, as JSON and in each SSE chunk. NEAR's gateway answers an encrypted completion with End-to-end encryption is not supported on /v1/completions.

Change

  • EncryptCompletionsNearCloud encrypts prompt, a string or an array of strings.
  • Besides prompt, a request may hold only sampling and limit fields (max_tokens, temperature, top_p, stop, n, stream, seed and similar). logprobs, prompt_logprobs and suffix are accepted only as null, and echo and return_token_ids only as null or false; an integer 0 still turns logprobs on.
  • Any other field, a prompt of token ids, a missing prompt or an empty prompt array fails with ErrCompletionsShape before the request is sent, because its content would travel or return unencrypted. A new upstream field therefore fails closed.
  • The relay decrypts choices[].text of a completions response, as JSON and in SSE chunks. Each choice must carry an encrypted text; besides it only structural metadata, and logprobs, prompt_logprobs, token_ids and prompt_token_ids only as null. Anything else fails closed.
  • A NEAR completions stream must end with [DONE], as a chat stream must.
  • /v1/completions is routed for neardirect and confirms e2ee_usable as chat does. nearcloud has no completions route, because NEAR's gateway does not encrypt it.
  • docs/api_support.md states the field map.

Effect

  • Text completions work through teep on NearDirect, end-to-end encrypted.
  • A completion request that NEAR's E2EE cannot carry fails before it is sent.
  • Other endpoints and providers do not change.

To verify

go test ./internal/e2ee -run Completions
go test ./internal/proxy -run TestNearDirectCompletionsE2EE

Each prints ok. The tests cover:

  • the encryption of a string and an array prompt;
  • each refused request shape with ErrCompletionsShape, including logprobs: 0, return_token_ids: true and an unknown field, and the accepted "off" values;
  • decryption of choices[].text in a JSON response and an SSE chunk, and the fail-closed result for a plaintext text, a missing text, token ids, logprobs and an unknown choice key;
  • the [DONE] requirement for a completions stream;
  • a NearDirect completion end to end, with e2ee_usable confirmed.

Checklist

  • I added unit tests for new code and regression tests for fixed bugs.
  • make check passes.
  • For a major feature: make integration and make reports pass.
  • I updated the documentation that this change affects.

Teep had no route for /v1/completions, so text completions on NEAR did
not work through teep. NEAR's model hosts encrypt this endpoint: on
2026-11-14 a completion with an encrypted prompt and NEAR's E2EE headers
returned choices[].text encrypted to the client key, as JSON and in each
SSE chunk. NEAR's gateway answers an encrypted completion with
"End-to-end encryption is not supported on /v1/completions", so only
neardirect gets the route.

Changes:

- EncryptCompletionsNearCloud encrypts prompt, a string or an array of
  strings. Besides it, a request may hold only sampling and limit
  fields; logprobs, prompt_logprobs and suffix only as null, and echo
  and return_token_ids only as null or false. Any other field, a prompt
  of token ids or an empty prompt array fails with ErrCompletionsShape
  before the request is sent: its content would travel or return
  unencrypted. An integer 0 still turns logprobs on.
- The relay decrypts choices[].text of a completions response, as JSON
  and in SSE chunks. Each choice must carry an encrypted text; besides
  it only structural metadata, and logprobs, prompt_logprobs, token_ids
  and prompt_token_ids only as null. Anything else fails closed. A NEAR
  completions stream must end with [DONE], as a chat stream must.
- /v1/completions is routed for neardirect and confirms e2ee_usable as
  chat does. nearcloud has no completions route.
- The API support document states the field map.

To verify: go test ./internal/e2ee -run Completions and go test
./internal/proxy -run TestNearDirectCompletionsE2EE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@13rac1

13rac1 commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Review from GLM 5.3:

Full suite passes — 27 packages ok, zero failures, clean build. Here's my review.

Verdict

No major security issues. This one is solid. The design follows the codebase's established fail-closed pattern on both directions of the wire, and the test coverage actually matches the PR description's claims (I verified each listed case exists in the tests).

What's done well

Response side is genuinely fail-closed. decryptCompletionChoice requires an encrypted choices[].text in every choice, and rejects any unknown choice key, any non-null value for logprobs/prompt_logprobs/token_ids/prompt_token_ids, and a missing text — all with ErrDecryptionFailed. The "new upstream field fails closed" property holds because an unrecognized key can't slip through (default branch of the key loop). Tested for both JSON and SSE shapes.

Request side allowlist is strict. completionsAllowedFields + completionsOffFields with exact-value matching means logprobs: 0 (which NEAR treats as "on"), echo: true, token-id prompts, and any new upstream field all fail with ErrCompletionsShape before the request is sent. The raw-string comparison of "off" values (null, false via bytes.TrimSpace) is crude but correct for the JSON primitive forms.

Consistency details check out:

  • EncFieldCompletionText = "text" doesn't collide with the chat policy paths (content[].text is a different path; IsResponseFieldEncrypted for text at choice level is completions-only).
  • Session zeroing on every error path via the deferred closure + named returns is correct (no leaked X25519 key material on refusal).
  • [DONE] end-marker requirement extended to EndpointCompletions for NEAR sessions only — doesn't change other protocols.
  • nearcloud correctly gets no completions route (NEAR's gateway doesn't encrypt it) — it stays unsupported rather than silently sending plaintext.
  • Reusing parseChatRequest is safe: it reads only top-level model and stream, which have the same names in the completions schema.

Nits worth a comment (not blockers)

1. stop and user travel in plaintext — arguably inconsistent with the PR's own bar. The PR refuses fields whose "content would travel unencrypted," but stop (an array of user-authored strings that frequently quote or anticipate prompt content) and user (free-text) are in the allowlist. This matches what chat does and what the ACI v2 docs explicitly acknowledge, so it's a pre-existing, documented policy — but completions is a new surface, and the PR description's field map should probably state these two plaintext fields as explicitly as the nanogpt doc does, rather than leaving them under "sampling and limit fields."

2. Duplicate JSON keys in the request body aren't rejected. EncryptCompletionsNearCloud parses with plain json.Unmarshal into a map; the ACI/1 work in PR 211 used jsonstrict.ValidateUniqueFields to fail on duplicates. A duplicate "prompt" silently collapses to the last value. Not exploitable (the dropped value never travels, and teep re-marshals what it parsed), but it's a hardening inconsistency with the newer code in the same repo.

3. Nested-object members aren't validated. stream_options and logit_bias are allowlisted as opaque values — a content-carrying member added inside stream_options by NEAR in the future would pass. The top-level allowlist would catch a new top-level field, so the residual risk is narrow. Same pattern as the other NEAR endpoints, so consistent.

4. The z-ai/glm-5.3-flash-long model-lister change is scope creep. The PR is titled and described as completions E2EE, but it also introduces neardirect.NewModelLister — a new mechanism that appends a hand-pinned extra model to the catalog after a live per-model endpoint-list lookup (servesModel). Two notes:

  • servesModel does an uncached network round trip per extra model on every ListModels call. Harmless at one model, but it won't stay one model forever — consider caching the lookup per verification cycle.
  • The error handling treats an endpoint-list outage as "not served" (debug log, model silently disappears from the list). Conservative and safe, but a transient outage changes the visible model list; worth a sentence in the docs.

5. Minor: prompt: "" (empty string) encrypts an empty plaintext and produces a valid ciphertext, while prompt: [] is refused. Slightly asymmetric but harmless.

Test verification

Ran on the PR head (eec3081):

  • go test ./internal/e2ee/ -run Completions — all 4 new tests pass
  • go test ./internal/proxy/ -run TestNearDirectCompletionsE2EE — passes, including the assertion that the plaintext prompt never reaches the mock upstream and that e2ee_usable is confirmed on the shared authorization
  • Full go test ./... — 27 packages, zero failures

Recommendation: approve. If the author wants to act on anything, the stop/user plaintext disclosure in docs/api_support.md (nit 1) is the only one I'd ask for before merge; the rest can be follow-ups.

ai-13rac1 and others added 2 commits November 14, 2026 04:36
The completions encryptor decoded the request into a map, which keeps
only the last of duplicate members. The allowlist check then saw that
last value, while the upstream can read another one: with
"logprobs":5,"logprobs":null the check saw null, and the request could
ask for token log probabilities, which return unencrypted.

The encryptor now runs jsonstrict.ValidateUniqueFields on the body
first and refuses a duplicate member at any depth, also one spelled
with another JSON escape, with ErrCompletionsShape. Nothing is sent.

To verify: go test ./internal/e2ee -run
TestEncryptCompletionsNearCloudRefuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The completions row of the encrypted-field map listed `stop` among the
sampling fields and did not name `user`. Both pass in plaintext: `stop`
holds stop sequences, which can hold text, and `user` is an end-user
identifier. The row now names them, says that every accepted field
other than `prompt` passes in plaintext, and lists the refusal of a
duplicate member.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ai-13rac1

Copy link
Copy Markdown
Contributor Author

Thank you for the review. Two of the notes are now in the branch:

  • docs/api_support.md states that every accepted completions field except prompt travels in plaintext, and names stop and user.
  • The completions encryptor refuses a duplicate JSON key at any depth before anything is sent, with the same check the ACI/1 parsers use. "logprobs":5,"logprobs":null is one of the test cases.

Not changed: stream_options and logit_bias stay opaque, as on the other NEAR endpoints, and an empty prompt string encrypts an empty plaintext while an empty array is refused.

On the model-lister note: this pull request does not touch the lister. The code you saw is #220, which is in this branch's base and was reverted on main by #229 after NEAR stopped serving the model. Merging this branch keeps the revert.

@13rac1
13rac1 merged commit 0d48f2e into 13rac1:main Nov 14, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants