Repository navigation
Description
Problem
With E2EE on, which is the default for Phala Cloud, teep refuses every chat request that carries tools, or a message with tool_calls. An agent that calls tools sends both on almost every request, so an agent cannot use Phala Cloud through teep with E2EE on. The client gets HTTP 502 and the text inference authorization or upstream request failed; see server logs. Only the server log names the field.
Observed on 2026-11-19
teep serve built from 6548996 plus #239 (so that the session of deepseek/deepseek-v4.1-flash-1017 verifies), with phalacloud as the only provider. The tools request has one function (get_weather) and tool_choice: "auto".
| E2EE | Request | Result |
|---|---|---|
| on | chat, no tools | HTTP 200 |
| on | chat with tools | HTTP 502, inference authorization or upstream request failed; see server logs. Log: authorized inference failed ... err="E2EE v2 request refused: tools holds free text, and E2EE v2 of the gateway defines no field path for it" ... failure_phase=prepare |
| off | chat with tools | HTTP 200, finish_reason: tool_calls, one call get_weather({"city": "Paris"}) |
Cause
The cause is the gateway's E2EE v2 protocol, not a teep defect. refuseUnencryptedFreeText (internal/provider/aci/e2ee.go, from #210) refuses the request because the protocol has no place to put these fields:
spec/e2ee-v2.md§5 inDstack-TEE/private-ai-gateway(last changed in 9b745bda, 2026-11-10) defines request field paths for message content, completion prompts and embedding inputs only. Response field paths covercontent,reasoning,reasoning_contentandaudio.data. There is no path fortools,messages[].tool_calls, or a response'stool_calls.- §1 says v2 is "frozen except for security, correctness, and interoperability fixes. New features will not expand its wire contract." E2EE v3 is to replace it, and v3 has no specification yet.
- The gateway code at cc49df8 (main, read on 2026-11-19) does what the spec says.
decrypt_request_payloadinsrc/aggregator/service/e2ee_crypto.rsdecrypts onlymessages[].content. The response encryptors in the same file encrypt only content, reasoning, reasoning_content and audio.
If teep sent the request anyway, the tool schemas, earlier calls and their arguments would go in plaintext next to the encrypted content, and the model's tool_calls would come back unencrypted. Refusing is correct. No teep change can encrypt these fields under v2.
What teep could change
- The status and message. Refuse an endpoint outside the provider's E2EE coverage with 403 and say why #230 refuses an endpoint outside a provider's E2EE coverage with HTTP 403
e2ee_not_coveredand a message that names the setting. A request field outside the coverage gets 502, which the client cannot tell apart from an upstream failure. The same 403 form would fit: name the field, and say that withe2ee = falsethe request goes over the attested TLS connection. That would mean checking the body before authorization, likerefuseUncoveredEndpoint, and not in the prepare phase. I have not written this because it changes the refusal path for every E2EE provider. - Documentation.
docs/api_support.mdlists the fields E2EE v2 leaves out, but it does not say that tool calling needse2ee = falsefor Phala Cloud and NanoGPT. With E2EE off, a Phala Cloud request still goes over TLS pinned to the keyset TLS key that the gateway quote binds (phalacloud_support.md, TLS and E2EE), and the pinned Phala model CVM session states atls_spki_sha256channel binding for its endpoint. E2EE v2 terminates at the gateway (§2), so the gateway process reads the request in plaintext with E2EE on or off.teep serverequires a verified receipt for every successful Phala Cloud response, with E2EE on or off (phalacloud_support.md, Receipts), so the HTTP 200 above passed that check.
A third option is to wait for E2EE v3. Whether v3 will cover tool fields is a question for the gateway project.
Posted with Claude.
Activity