Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/attestation_gaps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ New reports should follow [TEMPLATE.md](TEMPLATE.md).
- [Model Weight Authentication](model_weights.md)
- [Proof of Cloud Registry: JWT Validation Failure](poc_jwt_validation.md)
- [Chutes/sek8s Integrity Chain](sek8s_integrity.md)
- [Tinfoil Model Endpoints: Verify Probe Covers Chat Only](tinfoil_model_endpoints.md)
- [Phala Cloud Model Endpoints: No Endpoint Field in the Model List](phalacloud_model_endpoints.md)
- [ACI/1 Gateway: Attested Gateway, Unattested Inference Host](aci_gateway.md)
- [NanoGPT Chutes Evidence: No Nonce Binding, No Serving Instance](nanogpt_chutes_evidence.md)

Expand Down
92 changes: 92 additions & 0 deletions docs/attestation_gaps/phalacloud_model_endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Phala Cloud Model Endpoints: No Endpoint Field in the Model List

`teep verify` tests end-to-end encryption with one chat completion, called
the probe. Four Phala Cloud models have no text output modality, so teep's
verifier does not test their encryption today. Phala Cloud's model list has
no endpoint field. Only modalities, pricing keys and, for one model, the
free-text description show which endpoint a model serves. `teep serve` has
no Phala Cloud path for image generation, audio transcription or
`/v1/systemone`, so it serves none of these models.

## The Problem

Anyone who checks a Phala Cloud model before use, such as a user's client or
an auditor, needs to send it a real request to see that encryption works. The
request must go to an endpoint that the model serves. The model list has no
field that states that endpoint, and a model without chat completions answers
a chat request with an error.

## Impact

**The encryption of models without text output is not tested before use.**

- **Who:** Phala Cloud, or a party that controls a Phala Cloud model
deployment or the gateway.
- **What they do:** serve a model without chat completions whose encryption
does not work.
- **What they get:** a model that passes verification without an encryption
test. They read no request through teep: teep sends these models no request
that they serve.
- **What stops it today:** `teep serve` has no Phala Cloud path for these
endpoints. It sends these models only chat completions, encrypted when E2EE
is on, and the gateway refuses them. Nothing confirms their encryption. In
`teep verify`, nothing: the factor stays skipped.

## Technical Background

Phala Cloud's model list gives, for each model, `input_modalities`,
`output_modalities`, `pricing` and a `description`. It has no endpoint field.

The probe of `teep verify` is one streaming chat completion with E2EE v2.
The model must answer with encrypted fields that teep decrypts, and the
response must carry a receipt that teep checks.

## Detailed Gap Analysis

Phala Cloud's model list of 2026-11-14 has four models whose output modality
is not text:

| Model | Modalities in the model list | Endpoint kind (derived by teep) |
| --- | --- | --- |
| `black-forest-labs/flux.2-klein-4b` | text → image | image generation |
| `openai/whisper-large-v3` | audio → transcription | audio transcription |
| `mistralai/voxtral-small-24b-2507` | audio → transcription | audio transcription |
| `jaredpalmer/kev-4b` | text → decisions | `/v1/systemone` (named in the description) |

On 2026-11-14, `jaredpalmer/kev-4b` got the chat probe, and the gateway
answered "upstream returned HTTP 404: no route available for model
jaredpalmer/kev-4b". `e2ee_usable`, which Phala Cloud's policy allows to
fail, failed, and the report did not block. The reports of
`black-forest-labs/flux.2-klein-4b` and `openai/whisper-large-v3` were
blocked by `aci_session_verified`, because teep does not verify the evidence
of their gateway session, so they got no probe. The run of 2026-11-14 made
no report for `mistralai/voxtral-small-24b-2507`.

## Remediation

- **Phala Cloud:** add the served endpoint (or endpoints) to each entry of the
model list, as a field.
- **teep:** probe each model on an endpoint that it serves, once E2EE v2
covers that endpoint.

## References

- [Phala Cloud provider support: known limitations](../providers/phalacloud/phalacloud_support.md#known-limitations)
- [Tinfoil Model Endpoints: Verify Probe Covers Chat Only](tinfoil_model_endpoints.md)
- [NEAR Model Endpoints: No Endpoint in the Model List](near_model_endpoints.md)
- [The table of unprobed Phala Cloud models in teep](../../internal/provider/phalacloud/kinds.go)

## Teep Status

`teep verify` sends no chat probe to the four models above and logs a
warning. `e2ee_usable`, which the policy allows to fail, stays skipped with
the detail "not probed: the model serves <kind>; the verify probe sends a
chat completion". `aci_receipt_verified`, which is enforced, stays skipped
with the detail "no probe: the model serves <kind>; …". Neither blocks.
Because `e2ee_usable` is allowed to fail, a failed probe did not block
either: the skip changes the detail in the report, not whether the report
blocks. A model that is not in the table gets the chat probe, so a new model
without chat completions fails the factor and is reported.

`teep serve` has no Phala Cloud path for these endpoints, so no request
confirms the encryption of these four models.
98 changes: 98 additions & 0 deletions docs/attestation_gaps/tinfoil_model_endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Tinfoil Model Endpoints: Verify Probe Covers Chat Only

`teep verify` tests end-to-end encryption with one chat completion, the
probe. Ten Tinfoil models serve no chat completions endpoint, so teep's
verifier does not test their encryption today. Their other factors give the
same results as Tinfoil's chat models. Tinfoil's model list states the
endpoint of each model, so the gap is in teep, not in Tinfoil's
documentation.

## The Problem

Anyone who checks a Tinfoil model before use, such as a user's client or an
auditor, needs to send it a real request to see that encryption works. The
request must go to an endpoint that the model serves. A model without chat
completions answers a chat request with HTTP 404, so a check that sends only
chat cannot test it. Each request body stays encrypted to the attested key,
but no check confirms a full encrypted exchange before use.

## Impact

**The encryption of models without chat completions is not tested before
use.**

- **Who:** Tinfoil, or a party that controls a Tinfoil model enclave.
- **What they do:** serve a model without chat completions whose encryption
does not work.
- **What they get:** a model that passes verification without an encryption
test. They read no request: teep encrypts each request body to the attested
key.
- **What stops it today:** EHBP (HPKE X25519 with AES-256-GCM, full-body
encryption) encrypts the whole body of every request that teep sends with
E2EE on, and the response must decrypt under the same session. For
`nomic-embed-text`, `qwen3-tts` and `voxtral-tts`, the first real request in
`teep serve` checks the encryption, and a decryption failure blocks the
request. `teep serve` refuses audio transcription while encryption is on,
and has no route for `/redact`, `/v1/convert/file`, `/v1/realtime` or
`/v1/decisions`. No request to these models is sent unencrypted, but nothing
confirms their encryption. In `teep verify`, nothing: the factor stays
skipped.

## Technical Background

Tinfoil's model list (`inference.tinfoil.sh/v1/models`) gives, for each model,
a `type` and an `endpoints` list of the paths that the model serves. Teep does
not read this list during verification.

The probe of `teep verify` is one streaming chat completion through EHBP. The
model must answer with a body that teep decrypts.

## Detailed Gap Analysis

Tinfoil's model list of 2026-11-14 has ten models whose `endpoints` have no
`/v1/chat/completions`:

| Model | `type` | `endpoints` |
| --- | --- | --- |
| `nomic-embed-text` | embedding | `/v1/embeddings` |
| `pii-filter` | safety | `/redact` |
| `doc-upload` | document | `/v1/convert/file` |
| `whisper-large-v3-turbo` | audio | `/v1/audio/transcriptions` |
| `qwen3-tts` | tts | `/v1/audio/speech` |
| `voxtral-tts` | tts | `/v1/audio/speech` |
| `voxtral-mini-4b-realtime` | audio | `/v1/realtime` |
| `decision-large` | decision | `/v1/decisions` |
| `decision-medium` | decision | `/v1/decisions` |
| `decision-small` | decision | `/v1/decisions` |

On 2026-11-14, six of them got the chat probe: `nomic-embed-text`,
`pii-filter`, `qwen3-tts`, `voxtral-mini-4b-realtime`, `voxtral-tts` and
`whisper-large-v3-turbo`. Each answered "upstream returned HTTP 404: Not
found.", and `e2ee_usable` failed and blocked the model.

`voxtral-small-24b` serves audio transcription and also chat completions, so
it gets the chat probe.

## Remediation

- **teep:** read the `endpoints` field of Tinfoil's model list during
verification in place of a fixed table, and probe each model on an endpoint
that it serves, such as embeddings or speech, which EHBP already encrypts.
- **Tinfoil:** none. The model list states the endpoints.

## References

- [Tinfoil provider support: models the probe does not cover](../providers/tinfoil/tinfoil_support.md#models-the-probe-does-not-cover)
- [NEAR Model Endpoints: No Endpoint in the Model List](near_model_endpoints.md)
- [The table of unprobed Tinfoil models in teep](../../internal/provider/tinfoil/kinds.go)

## Teep Status

`teep verify` sends no chat probe to the ten models above and logs a warning.
`e2ee_usable` stays skipped, enforced and not probed, with the detail "not
probed: the model serves <endpoint>; the verify probe sends a chat
completion". It counts as skipped and does not block. A model that is not in
the table gets the chat probe, so a new model without chat completions fails
the factor and is reported.

`teep serve` does not change.
14 changes: 13 additions & 1 deletion docs/providers/phalacloud/phalacloud_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,11 +414,23 @@ report carries no GPU evidence, and `nonce_match` and
is no probe receipt to check: in `teep serve`, which checks the receipt of
every response instead, in `teep verify` when the report blocks inference
requests (teep sends no probe then), without an API key, with `--offline`
(an offline run sends no probe), and after a probe whose
(an offline run sends no probe), for a model that the probe does not cover
(see [Known limitations](#known-limitations)), and after a probe whose
stream failed.

## Known limitations

- `teep verify` tests E2EE with one chat completion, called the probe. It
sends no probe to the four Phala Cloud models whose output modality is not
text, and it logs a warning. Those models are
`black-forest-labs/flux.2-klein-4b`, `openai/whisper-large-v3`,
`mistralai/voxtral-small-24b-2507` and `jaredpalmer/kev-4b`, listed in
`unprobedKinds` in
[`internal/provider/phalacloud/kinds.go`](../../../internal/provider/phalacloud/kinds.go).
`e2ee_usable` (allowed to fail) and `aci_receipt_verified` (enforced) stay
skipped and do not block. A model that is not in that table gets the
probe. `teep serve` has no path for the endpoints of these models. See
[Phala Cloud Model Endpoints: No Endpoint Field in the Model List](../../attestation_gaps/phalacloud_model_endpoints.md).
- For a model whose gateway session is replaced at each re-verification,
the pin is refused and teep serves no request (observed on 2026-11-03;
see [Attested sessions](#attested-sessions)).
Expand Down
18 changes: 18 additions & 0 deletions docs/providers/tinfoil/tinfoil_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,24 @@ as a single AEAD stream. There are **no field-level gaps** for those encrypted
exchanges. Bodyless endpoints (for example `GET /v1/models`) are plaintext at
the HTTP-body layer by design.

### Models the probe does not cover

`teep verify` tests E2EE with one streaming chat completion, the probe. Ten
models in Tinfoil's model list of 2026-11-14 serve no `/v1/chat/completions`:
`nomic-embed-text`, `pii-filter`, `doc-upload`, `whisper-large-v3-turbo`,
`qwen3-tts`, `voxtral-tts`, `voxtral-mini-4b-realtime`, and
`decision-large`, `-medium` and `-small`. On 2026-11-14, the six of them that
got the probe answered it with HTTP 404. `teep verify` sends these ten models
no probe and logs a warning. `e2ee_usable` stays enforced with the status
Skip and the detail "not probed: the model serves <endpoint>; the verify
probe sends a chat completion", and does not block. A model that is not in
the table gets the probe, so a new model without chat completions fails
`e2ee_usable` and is reported. The table is `unprobedKinds` in
[`internal/provider/tinfoil/kinds.go`](../../../internal/provider/tinfoil/kinds.go),
copied from the `endpoints` field of Tinfoil's model list of 2026-11-14. The
gap is recorded in
[Tinfoil Model Endpoints: Verify Probe Covers Chat Only](../../attestation_gaps/tinfoil_model_endpoints.md).

### Endpoint Routing and Request-Shape Mechanics

To produce a compatible implementation without depending on upstream source
Expand Down
11 changes: 4 additions & 7 deletions internal/provider/nearroute/kinds.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,9 @@ var unprobedKinds = map[string]string{
"openai/privacy-filter": "an endpoint other than chat completions (unknown)",
}

// UnprobedDetail returns the e2ee_usable detail for a NEAR model whose
// endpoint the verify probe does not cover, and false for any other model.
func UnprobedDetail(model string) (string, bool) {
// UnprobedKind returns the endpoint kind of a NEAR model that serves an
// endpoint other than chat completions, and false for any other model.
func UnprobedKind(model string) (string, bool) {
kind, ok := unprobedKinds[model]
if !ok {
return "", false
}
return "not probed: the model serves " + kind + "; the verify probe sends a chat completion", true
return kind, ok
}
18 changes: 8 additions & 10 deletions internal/provider/nearroute/kinds_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,23 @@ package nearroute

import "testing"

// The five non-chat models of 2026-11-14 get a full-sentence detail; any
// other model gets the chat probe.
func TestUnprobedDetail(t *testing.T) {
for model, kind := range map[string]string{
// The five non-chat models of 2026-11-14 get their endpoint kind; any other
// model gets the chat probe.
func TestUnprobedKind(t *testing.T) {
for model, want := range map[string]string{
"black-forest-labs/FLUX.2-klein-4B": "image generation",
"Qwen/Qwen3-Embedding-0.6B": "embeddings",
"Qwen/Qwen3-Reranker-0.6B": "reranking",
"openai/whisper-large-v3": "audio transcription",
"openai/privacy-filter": "an endpoint other than chat completions (unknown)",
} {
detail, ok := UnprobedDetail(model)
want := "not probed: the model serves " + kind + "; the verify probe sends a chat completion"
if !ok || detail != want {
t.Errorf("%s: detail %q, %v; want %q", model, detail, ok, want)
if kind, ok := UnprobedKind(model); !ok || kind != want {
t.Errorf("%s: kind %q, %v; want %q", model, kind, ok, want)
}
}
for _, model := range []string{"z-ai/glm-5.3-flash", "qwen/qwen3-embedding-0.6b", "Qwen/Qwen3-Embedding-0.6B-v2", ""} {
if detail, ok := UnprobedDetail(model); ok {
t.Errorf("%q is not listed but got %q", model, detail)
if kind, ok := UnprobedKind(model); ok {
t.Errorf("%q is not listed but got %q", model, kind)
}
}
}
26 changes: 26 additions & 0 deletions internal/provider/phalacloud/kinds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package phalacloud

// unprobedKinds lists, by exact model id, the Phala Cloud models whose output
// is not text, with the endpoint kind that each serves. The E2EE probe of
// teep verify sends a chat completion; on 2026-11-14 the gateway answered it
// for jaredpalmer/kev-4b with HTTP 404 "no route available for model". A
// model that is not listed gets the chat probe, so a new model without chat
// completions fails e2ee_usable and is reported.
//
// Read from Phala Cloud's model list of 2026-11-14: every model whose output
// modality is not text. The list has no endpoint field, so the kind follows
// from the modalities, except for jaredpalmer/kev-4b, whose description names
// its endpoint.
var unprobedKinds = map[string]string{
"black-forest-labs/flux.2-klein-4b": "image generation", // text → image
"openai/whisper-large-v3": "audio transcription", // audio → transcription
"mistralai/voxtral-small-24b-2507": "audio transcription", // audio → transcription
"jaredpalmer/kev-4b": "/v1/systemone", // text → decisions; named in the description
}

// UnprobedKind returns the endpoint kind of a Phala Cloud model whose output
// is not text, and false for any other model.
func UnprobedKind(model string) (string, bool) {
kind, ok := unprobedKinds[model]
return kind, ok
}
23 changes: 23 additions & 0 deletions internal/provider/phalacloud/kinds_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package phalacloud

import "testing"

// The four models without text output in Phala Cloud's model list of
// 2026-11-14 get their kind; any other model gets the chat probe.
func TestUnprobedKind(t *testing.T) {
for model, want := range map[string]string{
"black-forest-labs/flux.2-klein-4b": "image generation",
"openai/whisper-large-v3": "audio transcription",
"mistralai/voxtral-small-24b-2507": "audio transcription",
"jaredpalmer/kev-4b": "/v1/systemone",
} {
if kind, ok := UnprobedKind(model); !ok || kind != want {
t.Errorf("%s: kind %q, %v; want %q", model, kind, ok, want)
}
}
for _, model := range []string{"z-ai/glm-5.3", "openai/whisper-large-v3-turbo", "Jaredpalmer/kev-4b", ""} {
if kind, ok := UnprobedKind(model); ok {
t.Errorf("%q is not listed but got %q", model, kind)
}
}
}
30 changes: 30 additions & 0 deletions internal/provider/tinfoil/kinds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package tinfoil

// unprobedKinds lists, by exact model id, the Tinfoil models that serve no
// chat completions endpoint, with the endpoint that each serves. The E2EE
// probe of teep verify sends a chat completion, which these models answer
// with HTTP 404. A model that is not listed gets the chat probe, so a new
// model without chat completions fails e2ee_usable and is reported.
//
// Read from the `endpoints` field of Tinfoil's model list of 2026-11-14:
// every model whose list has no /v1/chat/completions. Reading that field at
// verification time, in place of this table, is a follow-up change.
var unprobedKinds = map[string]string{
"nomic-embed-text": "/v1/embeddings",
"pii-filter": "/redact",
"doc-upload": "/v1/convert/file",
"whisper-large-v3-turbo": "/v1/audio/transcriptions",
"qwen3-tts": "/v1/audio/speech",
"voxtral-tts": "/v1/audio/speech",
"voxtral-mini-4b-realtime": "/v1/realtime",
"decision-large": "/v1/decisions",
"decision-medium": "/v1/decisions",
"decision-small": "/v1/decisions",
}

// UnprobedKind returns the endpoint of a Tinfoil model that serves no chat
// completions endpoint, and false for any other model.
func UnprobedKind(model string) (string, bool) {
kind, ok := unprobedKinds[model]
return kind, ok
}
Loading
Loading