Skip to content

Phala Cloud with E2EE on refuses every chat request with tools, and the client gets a 502 #240

Open

Description

@amiller

Problem

With E2EE on, which is the default for Phala Cloud, teep refuses every chat request that carries tools, or a message with tool_calls. An agent that calls tools sends both on almost every request, so an agent cannot use Phala Cloud through teep with E2EE on. The client gets HTTP 502 and the text inference authorization or upstream request failed; see server logs. Only the server log names the field.

Observed on 2026-11-19

teep serve built from 6548996 plus #239 (so that the session of deepseek/deepseek-v4.1-flash-1017 verifies), with phalacloud as the only provider. The tools request has one function (get_weather) and tool_choice: "auto".

E2EE Request Result
on chat, no tools HTTP 200
on chat with tools HTTP 502, inference authorization or upstream request failed; see server logs. Log: authorized inference failed ... err="E2EE v2 request refused: tools holds free text, and E2EE v2 of the gateway defines no field path for it" ... failure_phase=prepare
off chat with tools HTTP 200, finish_reason: tool_calls, one call get_weather({"city": "Paris"})

Cause

The cause is the gateway's E2EE v2 protocol, not a teep defect. refuseUnencryptedFreeText (internal/provider/aci/e2ee.go, from #210) refuses the request because the protocol has no place to put these fields:

  • spec/e2ee-v2.md §5 in Dstack-TEE/private-ai-gateway (last changed in 9b745bda, 2026-11-10) defines request field paths for message content, completion prompts and embedding inputs only. Response field paths cover content, reasoning, reasoning_content and audio.data. There is no path for tools, messages[].tool_calls, or a response's tool_calls.
  • §1 says v2 is "frozen except for security, correctness, and interoperability fixes. New features will not expand its wire contract." E2EE v3 is to replace it, and v3 has no specification yet.
  • The gateway code at cc49df8 (main, read on 2026-11-19) does what the spec says. decrypt_request_payload in src/aggregator/service/e2ee_crypto.rs decrypts only messages[].content. The response encryptors in the same file encrypt only content, reasoning, reasoning_content and audio.

If teep sent the request anyway, the tool schemas, earlier calls and their arguments would go in plaintext next to the encrypted content, and the model's tool_calls would come back unencrypted. Refusing is correct. No teep change can encrypt these fields under v2.

What teep could change

  1. The status and message. Refuse an endpoint outside the provider's E2EE coverage with 403 and say why #230 refuses an endpoint outside a provider's E2EE coverage with HTTP 403 e2ee_not_covered and a message that names the setting. A request field outside the coverage gets 502, which the client cannot tell apart from an upstream failure. The same 403 form would fit: name the field, and say that with e2ee = false the request goes over the attested TLS connection. That would mean checking the body before authorization, like refuseUncoveredEndpoint, and not in the prepare phase. I have not written this because it changes the refusal path for every E2EE provider.
  2. Documentation. docs/api_support.md lists the fields E2EE v2 leaves out, but it does not say that tool calling needs e2ee = false for Phala Cloud and NanoGPT. With E2EE off, a Phala Cloud request still goes over TLS pinned to the keyset TLS key that the gateway quote binds (phalacloud_support.md, TLS and E2EE), and the pinned Phala model CVM session states a tls_spki_sha256 channel binding for its endpoint. E2EE v2 terminates at the gateway (§2), so the gateway process reads the request in plaintext with E2EE on or off. teep serve requires a verified receipt for every successful Phala Cloud response, with E2EE on or off (phalacloud_support.md, Receipts), so the HTTP 200 above passed that check.

A third option is to wait for E2EE v3. Whether v3 will cover tool fields is a question for the gateway project.

Posted with Claude.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions