Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 45 additions & 8 deletions docs/api_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ These are the OpenAI-style endpoints that clients use for inference:
| `/v1/images/generations` | POST | Image generation |
| `/v1/rerank` | POST | Document reranking |
| `/v1/score` | POST | Text similarity scoring |
| `/v1/images/edits` | POST | Image edits (multipart) |
| `/v1/privacy/classify` | POST | PII span classification |
| `/v1/privacy/redact` | POST | PII redaction |
| `/v1/models` | GET | List available models |

`/v1/models` is a proxy-aggregated endpoint that returns the combined model list from all configured providers. Each model's `id` field is rewritten to `provider:upstreamID` (e.g. `venice:e2ee-qwen3-5-122b-a10b`, `neardirect:Qwen/Qwen3-VL-30B-A3B-Instruct`) so clients can route requests to the correct provider. NanoGPT lists only its TEE models, whose IDs start with `TEE/`; its other models serve no attestation. It is not included in the per-provider matrices below because it is handled entirely by the proxy, does not forward requests to individual providers, and is not E2EE-encrypted (GET request, no sensitive data).
Expand Down Expand Up @@ -92,38 +95,66 @@ This matrix applies to OpenAI-compatible inference endpoints.
| Endpoint | NearDirect | NearCloud | Chutes | Venice | Phala Cloud | NanoGPT | Tinfoil Cloud | Tinfoil Direct |
|---|---|---|---|---|---|---|---|---|
| Chat completions | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Responses | — | — | — | — | — | — | Yes | Yes |
| Responses | Yes | Yes | — | — | — | — | Yes | Yes |
| Embeddings | Yes | Yes | Yes | — | Yes | — | Yes | Yes |
| Audio transcriptions | Yes | — | — | — | — | — | Yes | Yes |
| Audio transcriptions | Yes | Yes | — | — | — | — | Yes | Yes |
| Text-to-speech | — | — | — | — | — | — | Yes | Yes |
| Image generation | Yes | Yes | — | — | — | — | — | — |
| Reranking | Yes | Yes | — | — | — | — | — | — |
| Score | Yes | Yes | — | — | — | — | — | — |
| Completions | Yes | — | — | — | — | — | — | — |
| Image edits | Yes | Yes | — | — | — | — | — | — |
| Privacy classify and redact | Yes | Yes | — | — | — | — | — | — |

**—** = Not implemented. Provider returns HTTP 400.

With E2EE enabled, an endpoint outside the provider's E2EE coverage is refused. See the E2EE support matrix and [why teep refused my request](#why-teep-refused-my-request).

## E2EE Support Matrix

| Endpoint | NearDirect | NearCloud | Chutes | Venice | Phala Cloud | NanoGPT | Tinfoil Cloud | Tinfoil Direct |
|---|---|---|---|---|---|---|---|---|
| Chat completions | Encrypted | Encrypted | Encrypted | Encrypted | Encrypted (ACI/1 models) | Encrypted (ACI/1 models) | Encrypted | Encrypted |
| Responses | — | — | — | — | — | — | Encrypted | Encrypted |
| Responses | Refused when E2EE is enabled | Refused when E2EE is enabled | — | — | — | — | Encrypted | Encrypted |
| Embeddings | Encrypted | Encrypted | Encrypted | — | Fail closed when E2EE is enabled; plaintext when disabled | — | Encrypted | Encrypted |
| Audio transcriptions | Plaintext (pinned) | — | — | — | — | — | Fail closed when E2EE is enabled; plaintext when disabled | Fail closed when E2EE is enabled; plaintext when disabled |
| Audio transcriptions | Refused when E2EE is enabled | Refused when E2EE is enabled | — | — | — | — | Refused when E2EE is enabled | Refused when E2EE is enabled |
| Image edits | Refused when E2EE is enabled | Refused when E2EE is enabled | — | — | — | — | — | — |
| Privacy classify and redact | Refused when E2EE is enabled | Refused when E2EE is enabled | — | — | — | — | — | — |
| Text-to-speech | — | — | — | — | — | — | Encrypted | Encrypted |
| Image generation | Encrypted | Encrypted | — | — | — | — | — | — |
| Reranking | Encrypted | Encrypted | — | — | — | — | — | — |
| Score | Request encrypted; response plaintext (pinned) | Request encrypted; response plaintext (pinned) | — | — | — | — | — | — |
| Completions | Encrypted | — | — | — | — | — | — | — |

**Encrypted** = E2EE is applied to request and response fields.
**Refused when E2EE is enabled** = the provider's E2EE does not cover the endpoint, so with E2EE enabled teep refuses it with HTTP 403 and sends nothing. With `e2ee = false` for the provider, teep sends it over the attested, pinned TLS connection.
**Fail closed** = Proxy rejects the request with an error because E2EE is not supported for this endpoint in the end-to-end path, either because the proxy does not implement E2EE field dispatch for that request type or because the upstream TEE cannot decrypt it.
**Plaintext (pinned)** = Request and response transit in plaintext, but the connection is TLS-pinned to the attested TEE (no E2EE field encryption).
**Request encrypted; response plaintext (pinned)** = Request fields are encrypted end-to-end, but upstream currently returns plaintext response fields for this endpoint. This is a known upstream NearAI limitation (inference-proxy/cloud-api score response path), not a teep policy preference. Responses still transit on pinned TLS to attested TEEs.
**Encrypted (ACI/1 models)** = E2EE v2 of the private-ai-gateway encrypts the request and response content fields when the model answers with the gateway's ACI/1 attestation. A model that answers with another attestation format fails closed when E2EE is enabled.
**—** = Endpoint not available for this provider.

## Why teep refused my request

With E2EE enabled, an endpoint outside the provider's E2EE coverage is refused.
Teep sends nothing and answers HTTP 403 with an error of type
`e2ee_not_covered` that names the provider and the endpoint:

```json
{"error": {"type": "e2ee_not_covered", "provider": "<name>", "endpoint": "/v1/audio/transcriptions",
"message": "teep did not send this request. E2EE is enabled for provider <name>, and its end-to-end encryption does not cover /v1/audio/transcriptions. ..."}}
```

Nothing is wrong with teep or with the provider: you asked for end-to-end
encryption, and this endpoint cannot be encrypted end to end. To send it
without E2EE, set `e2ee = false` for the provider in your teep config. If the
provider's policy enforces `e2ee_usable`, also add `e2ee_usable` to its
`allow_fail` list. The request then uses TLS, which attestation binds only for
a provider that uses TLS binding (NearDirect, NearCloud, Phala Cloud and
Tinfoil); for Venice, NanoGPT and Chutes it is WebPKI TLS only. Each refusal logs one warning with the provider and the
endpoint, and the dashboard counts the refusals for each provider and lists the
endpoints that its E2EE covers and does not.

## Provider Details

**Teep E2EE Header:** Teep automatically sets `X-Encrypt-All-Fields: true` on all E2EE-enabled requests to NearDirect and NearCloud. This enables full-field encryption of all sensitive request and response fields. The encryption coverage documented below reflects teep's behavior with this header active.
Expand Down Expand Up @@ -257,7 +288,10 @@ another endpoint; see [models the probe does not cover](providers/near/near_atte
|---|---|---|---|
| Chat completions | `/v1/chat/completions` | Yes | Sensitive fields encrypted; streaming forced when E2EE active |
| Embeddings | `/v1/embeddings` | Yes | Sensitive fields encrypted; supports string and array input formats |
| Audio transcriptions | `/v1/audio/transcriptions` | Requires E2EE disabled | Multipart requests use attested TLS to the model TEE. The proxy rejects this endpoint when field-level E2EE is enabled |
| Audio transcriptions | `/v1/audio/transcriptions` | Refused when E2EE is enabled | Multipart requests use attested TLS to the model TEE when E2EE is disabled |
| Image edits | `/v1/images/edits` | Refused when E2EE is enabled | Multipart; attested TLS to the model TEE when E2EE is disabled |
| Privacy classify and redact | `/v1/privacy/classify`, `/v1/privacy/redact` | Refused when E2EE is enabled | NEAR does not decrypt an encrypted `input` on these endpoints (2026-11-14). Attested TLS to the model TEE when E2EE is disabled |
| Responses | `/v1/responses` | Refused when E2EE is enabled | NEAR model hosts answer 404 (2026-11-14); with E2EE disabled teep relays NEAR's answer |
| Image generation | `/v1/images/generations` | Yes | Sensitive fields encrypted; `prompt`, `b64_json`, and `revised_prompt` encrypted |
| Reranking | `/v1/rerank` | Yes | Sensitive fields encrypted; `query` and `documents[]` encrypted |
| Score | `/v1/score` | Request only | Request fields (`text_1`, `text_2`) encrypted; response `data[].score` currently plaintext (known upstream NearAI limitation) |
Expand All @@ -277,7 +311,10 @@ another endpoint; see [models the probe does not cover](providers/near/near_atte
|---|---|---|---|
| Chat completions | `/v1/chat/completions` | Yes | Gateway forwards E2EE headers to model TEE |
| Embeddings | `/v1/embeddings` | Yes | Gateway forwards E2EE headers to model TEE |
| Audio transcriptions | Not configured | Not supported | The proxy has no NearCloud audio route |
| Audio transcriptions | `/v1/audio/transcriptions` | Refused when E2EE is enabled | With E2EE disabled, attested TLS to the gateway TEE, which reads the request |
| Image edits | `/v1/images/edits` | Refused when E2EE is enabled | With E2EE disabled, attested TLS to the gateway TEE |
| Privacy classify and redact | `/v1/privacy/classify`, `/v1/privacy/redact` | Refused when E2EE is enabled | With E2EE disabled, attested TLS to the gateway TEE |
| Responses | `/v1/responses` | Refused when E2EE is enabled | The gateway implements this stateful API itself and stores responses; with E2EE it fails for NEAR's own models (2026-11-14). With E2EE disabled, attested TLS to the gateway TEE |
| Image generation | `/v1/images/generations` | Yes | Gateway forwards E2EE headers to model TEE |
| Reranking | `/v1/rerank` | Yes | Gateway forwards E2EE headers to model TEE |
| Score | `/v1/score` | Request only | Request fields (`text_1`, `text_2`) encrypted; response `data[].score` currently plaintext (known upstream NearAI limitation) |
Expand Down Expand Up @@ -399,7 +436,7 @@ Venice only exposes chat completions. No other endpoints are available.
| Chat completions | `/v1/chat/completions` | Yes | Full-body EHBP encryption; streaming and non-streaming supported |
| Responses | `/v1/responses` | Yes | Full-body EHBP encryption; streaming and non-streaming supported |
| Embeddings | `/v1/embeddings` | Yes | Full-body EHBP encryption |
| Audio transcriptions | `/v1/audio/transcriptions` | No when E2EE is enabled | Multipart route is implemented, but the proxy rejects Tinfoil multipart requests when E2EE is enabled. With E2EE disabled, it forwards over attested, SPKI-pinned TLS |
| Audio transcriptions | `/v1/audio/transcriptions` | Refused when E2EE is enabled | Tinfoil's E2EE (EHBP) in teep does not cover this multipart endpoint, so teep refuses it with HTTP 403 when E2EE is enabled. With E2EE disabled, it forwards over attested, SPKI-pinned TLS |
| Text-to-speech | `/v1/audio/speech` | Yes | Full-body EHBP encryption |

**E2EE field coverage:** EHBP encrypts the entire HTTP request and response body. There are **no field-level encryption gaps** — all request fields (messages, tools, parameters) and all response fields (content, tool_calls, usage) are encrypted by construction. Adding new OpenAI API fields requires zero changes to the encryption layer.
Expand All @@ -425,7 +462,7 @@ Venice only exposes chat completions. No other endpoints are available.
| Chat completions | `/v1/chat/completions` | Yes | Full-body EHBP encryption; streaming and non-streaming supported |
| Responses | `/v1/responses` | Yes | Full-body EHBP encryption; streaming and non-streaming supported |
| Embeddings | `/v1/embeddings` | Yes | Full-body EHBP encryption |
| Audio transcriptions | `/v1/audio/transcriptions` | No when E2EE is enabled | Multipart route is implemented, but the proxy rejects Tinfoil multipart requests when E2EE is enabled. With E2EE disabled, it forwards over attested, SPKI-pinned TLS |
| Audio transcriptions | `/v1/audio/transcriptions` | Refused when E2EE is enabled | Tinfoil's E2EE (EHBP) in teep does not cover this multipart endpoint, so teep refuses it with HTTP 403 when E2EE is enabled. With E2EE disabled, it forwards over attested, SPKI-pinned TLS |
| Text-to-speech | `/v1/audio/speech` | Yes | Full-body EHBP encryption |

**E2EE field coverage:** Identical to Tinfoil Cloud — full-body EHBP, no field-level gaps.
Expand Down
14 changes: 13 additions & 1 deletion internal/e2ee/session.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@
// Dependency flow: attestation → e2ee → provider → proxy → cmd
package e2ee

import "io"
import (
"errors"
"io"
)

// EndpointType identifies the canonical OpenAI-compatible endpoint for field
// encryption policy routing. These types are independent of provider-specific
Expand All @@ -34,8 +37,17 @@ const (
EndpointSpeech EndpointType = "speech"
// EndpointCompletions is /v1/completions.
EndpointCompletions EndpointType = "completions"
// EndpointPrivacy is /v1/privacy/classify and /v1/privacy/redact.
EndpointPrivacy EndpointType = "privacy"
// EndpointImagesEdits is /v1/images/edits (multipart).
EndpointImagesEdits EndpointType = "images_edits"
)

// ErrEndpointNotCovered is returned by an encryptor for an endpoint that its
// E2EE protocol does not support. The encryptor returns it before it reads
// the key or the body.
var ErrEndpointNotCovered = errors.New("E2EE does not support this endpoint")

// EncField* constants identify the dot-notation sub-field paths within inference
// API JSON structures that are subject to field-level encryption. Each constant
// value is the exact path string passed to Decryptor.IsResponseFieldEncrypted
Expand Down
14 changes: 12 additions & 2 deletions internal/provider/aci/e2ee.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ func NewE2EE() *E2EE { return &E2EE{now: time.Now} }
// v2 encrypts to. The body's top-level model is bound into every field's
// associated data, so a party that changes it makes the request fail.
func (a *E2EE) EncryptRequest(body []byte, raw *attestation.RawAttestation, endpoint e2ee.EndpointType) (e2ee.EncryptResult, error) {
if endpoint != e2ee.EndpointChat {
return e2ee.EncryptResult{}, fmt.Errorf("E2EE v2 unsupported endpoint %q", endpoint)
if !a.CoversE2EE(endpoint) {
return e2ee.EncryptResult{}, fmt.Errorf("E2EE v2: endpoint %q: %w", endpoint, e2ee.ErrEndpointNotCovered)
}
if raw.BackendFormat != attestation.FormatACI1 {
return e2ee.EncryptResult{}, fmt.Errorf("E2EE v2 requires an ACI/1 attestation, got format %q", raw.BackendFormat)
Expand Down Expand Up @@ -226,3 +226,13 @@ func parsesAsJSONArray(s string) bool {
t := strings.TrimLeft(s, " \t\r\n")
return strings.HasPrefix(t, "[") && json.Valid([]byte(t))
}

// CoversE2EE reports whether this E2EE covers endpoint. E2EE v2 is defined for chat completions only.
func (a *E2EE) CoversE2EE(endpoint e2ee.EndpointType) bool {
switch endpoint {
case e2ee.EndpointChat:
return true
default:
return false
}
}
19 changes: 17 additions & 2 deletions internal/provider/chutes/e2ee.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package chutes

import (
"errors"
"fmt"

"github.com/13rac1/teep/internal/attestation"
"github.com/13rac1/teep/internal/e2ee"
Expand All @@ -20,8 +21,11 @@ func NewE2EE() *E2EE {
// (ML-KEM-768 + ChaCha20-Poly1305).
// Returns ChutesE2EE for the Preparer to inject headers.
// Requires raw.InstanceID, raw.E2ENonce, and raw.ChuteID from attestation.
// The endpoint parameter is not used; Chutes uses full-body encryption for all endpoints.
func (c *E2EE) EncryptRequest(body []byte, raw *attestation.RawAttestation, _ e2ee.EndpointType) (e2ee.EncryptResult, error) {
// An endpoint outside CoversE2EE fails with e2ee.ErrEndpointNotCovered.
func (c *E2EE) EncryptRequest(body []byte, raw *attestation.RawAttestation, endpoint e2ee.EndpointType) (e2ee.EncryptResult, error) {
if !c.CoversE2EE(endpoint) {
return e2ee.EncryptResult{}, fmt.Errorf("chutes E2EE: endpoint %q: %w", endpoint, e2ee.ErrEndpointNotCovered)
}
encPayload, session, err := e2ee.EncryptChatRequestChutes(body, raw.SigningKey)
if err != nil {
return e2ee.EncryptResult{}, err
Expand All @@ -45,3 +49,14 @@ func (c *E2EE) EncryptRequest(body []byte, raw *attestation.RawAttestation, _ e2
}
return e2ee.EncryptResult{Body: encPayload, Chutes: meta}, nil
}

// CoversE2EE reports whether this E2EE covers endpoint. Chutes E2EE
// encrypts the whole body of chat and embeddings requests.
func (c *E2EE) CoversE2EE(endpoint e2ee.EndpointType) bool {
switch endpoint {
case e2ee.EndpointChat, e2ee.EndpointEmbeddings:
return true
default:
return false
}
}
17 changes: 16 additions & 1 deletion internal/provider/neardirect/e2ee.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ func NewE2EE() *E2EE { return &E2EE{} }
//
// Unsupported endpoints fail closed.
func (n *E2EE) EncryptRequest(body []byte, raw *attestation.RawAttestation, endpoint e2ee.EndpointType) (e2ee.EncryptResult, error) {
if !n.CoversE2EE(endpoint) {
return e2ee.EncryptResult{}, fmt.Errorf("NearAI E2EE: endpoint %q: %w", endpoint, e2ee.ErrEndpointNotCovered)
}
modelKey, err := e2ee.ParseNearModelKey(raw.SigningKey)
if err != nil {
return e2ee.EncryptResult{}, err
Expand All @@ -52,11 +55,23 @@ func (n *E2EE) EncryptRequestWithModelKey(body []byte, modelKey e2ee.NearModelKe
case e2ee.EndpointCompletions:
encrypt = e2ee.EncryptCompletionsNearCloud
default:
return e2ee.EncryptResult{}, fmt.Errorf("NearAI E2EE not supported for endpoint %q", endpoint)
return e2ee.EncryptResult{}, fmt.Errorf("NearAI E2EE: endpoint %q: %w", endpoint, e2ee.ErrEndpointNotCovered)
}
encBody, session, err := encrypt(body, modelKey)
if err != nil {
return e2ee.EncryptResult{}, err
}
return e2ee.EncryptResult{Body: encBody, Session: session}, nil
}

// CoversE2EE reports whether this E2EE covers endpoint. NEAR E2EE covers
// chat, completions, embeddings, image generation, rerank, and score (request
// fields only).
func (n *E2EE) CoversE2EE(endpoint e2ee.EndpointType) bool {
switch endpoint {
case e2ee.EndpointChat, e2ee.EndpointCompletions, e2ee.EndpointEmbeddings, e2ee.EndpointImages, e2ee.EndpointRerank, e2ee.EndpointScore:
return true
default:
return false
}
}
12 changes: 12 additions & 0 deletions internal/provider/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,9 @@ type RequestPreparer interface {
// - Tinfoil/EHBP: EHBP (full-body state, decrypted before relay)
type RequestEncryptor interface {
EncryptRequest(body []byte, raw *attestation.RawAttestation, endpoint e2ee.EndpointType) (e2ee.EncryptResult, error)
// CoversE2EE reports whether the provider's E2EE covers endpoint. With
// E2EE enabled, teep refuses an endpoint that it does not cover.
CoversE2EE(endpoint e2ee.EndpointType) bool
}

// ModelLister fetches the list of available models from a provider.
Expand Down Expand Up @@ -142,6 +145,15 @@ type Provider struct {
// CompletionsPath is the upstream API path for text completions
// (e.g. "/v1/completions"). Empty means unsupported.
CompletionsPath string
// PrivacyClassifyPath and PrivacyRedactPath are the upstream API paths
// of privacy classification and redaction (e.g. "/v1/privacy/classify").
// Empty means unsupported.
PrivacyClassifyPath string
PrivacyRedactPath string

// ImagesEditsPath is the upstream API path for image edits
// (e.g. "/v1/images/edits", multipart). Empty means unsupported.
ImagesEditsPath string

// ResponsesPath is the upstream API path for responses
// (e.g. "/v1/responses"). Empty means unsupported.
Expand Down
Loading
Loading