Skip to content

Report the Phala Cloud inference host factors from the session audit - #241

Draft
amiller wants to merge 5 commits into
13rac1:mainfrom
amiller:phala-aci-mode-2026-11
Draft

amiller wants to merge 5 commits into
13rac1:mainfrom
amiller:phala-aci-mode-2026-11

Conversation

@amiller

@amiller amiller commented Nov 20, 2026 •

Copy link
Copy Markdown

Stacked on #239: the first two commits are #239's; this PR adds the third.

Problem

teep verify phalacloud reads the canonical ACI/1 report (/v1/aci/attestation). It carries the gateway's quote and no quote of the inference host. The evidence of the inference host is in the attested sessions, which aci_session_verified audits and enforces. The core factors of the inference host still evaluate against the response: tee_quote_present, tee_cert_chain, tee_reportdata_binding, intel_pcs_collateral, event_log_integrity, compose_binding, the nvidia_* factors and others fail with "no TEE quote" or "ACI/1 publishes no compose manifest". PhalaCloudACIDefaultAllowFail waives them, so a user's table waives them too. The report then shows a check that the session audit makes (the quote's certificate chain) the same way as one it does not make (an RTMR policy). The second kind is hidden by the waivers of the first.

The compose detail is also out of date for the Phala model CVM: its session evidence carries tcb_info.app_compose, and since #239 the audit checks it against MRConfigID version 1 or 3.

Change

  • SessionAuditor.Covers(s) (session_audit.go) names the report factors that Audit checks for a session. For every session: tee_quote_present, tee_quote_structure, tee_cert_chain, tee_quote_signature, tee_debug_disabled, tee_measurement (the built-in dstack or Chutes MRTD and MRSEAM list), tee_reportdata_binding (the session's signing key and the gateway's evidence nonce), nvidia_payload_present, nvidia_signature and nvidia_claims. Online only: intel_pcs_collateral, tee_tcb_current and tee_tcb_not_revoked (the audit requires UpToDate). For the Phala model CVM only: event_log_integrity and compose_binding.
  • AuditSessions sets ACISessionResult.Covered to the factors that every pinned session covers.
  • BuildReport calls applyACISessionCoverage when the report is the canonical ACI/1 report, has no quote of its own, and at least one session is pinned. Each factor in Covered passes with the detail covered by aci_session_verified: …. Every other factor keeps its own result. With no pinned session, nothing changes.
  • PhalaCloudACIDefaultAllowFail no longer lists the factors that every session covers online, so they are enforced. event_log_integrity and compose_binding stay allowed, because a Chutes session does not cover them. nvidia_signature is enforced; nvidia_nonce_client_bound stays allowed.
  • phalacloud_support.md, Factor enforcement: what ACI mode reports and what the audit does not check.

What the audit does not check, and what therefore still fails in ACI mode: the RTMR values of the inference host (tee_hardware_config, tee_boot_config; the event log is replayed, but no RTMR policy exists), the client nonce in the GPU evidence (the session's GPU evidence answers the gateway's nonce), NRAS, cpu_gpu_chain, measured_model_weights, component_recognition and cpu_id_registry.

The E2EE authorization is unchanged: for Phala Cloud it reads gateway_tee_reportdata_binding (GatewayBindsE2EEKey), not tee_reportdata_binding.

Tests

  • TestACISessionCoverage (internal/attestation): a covered factor passes and is enforced; an uncovered factor (tee_quote_signature when only the chain is covered, tee_hardware_config, nvidia_nonce_client_bound, cpu_gpu_chain) does not pass, and an enforced one blocks; with no pinned session, or a session error, the covered factors fail; a response with its own quote is not in ACI mode.
  • TestAuditSessionsCovered (internal/provider/phalacloud): the Phala model CVM session covers compose_binding and event_log_integrity; with a Chutes session also pinned, offline, neither they nor the TCB factors are covered. Every factor that a session covers online is enforced by PhalaCloudACIDefaultAllowFail, and the uncovered ones stay allowed.
  • TestVerifyRun_PhalaCloudCanonical_Fixture: the fixture (Phala model CVM and Chutes sessions) reports tee_quote_present, tee_quote_signature, tee_reportdata_binding and nvidia_signature as enforced passes covered by the audit, and compose_binding and event_log_integrity as allowed failures.
  • TestBuildReportCanonicalBody: without an audited session, nvidia_signature is now an enforced failure.

go test -short ./cmd/... ./internal/...: ok for every package. golangci-lint run ./cmd/... ./internal/...: 0 issues. teeplint: 0 failed.

Effect

Live, 2026-11-20, teep verify phalacloud --model deepseek/deepseek-v4.1-flash-1017 with e2ee = false and this allow_fail:

allow_fail = ["e2ee_usable", "tee_hardware_config", "tee_boot_config", "nvidia_nonce_client_bound", "nvidia_nras_verified", "cpu_gpu_chain", "measured_model_weights", "component_recognition", "nonce_match", "gateway_nonce_match", "aci_backend_named", "cpu_id_registry", "gateway_cpu_id_registry", "gateway_tee_hardware_config"]

exits 0:

  ✓ tee_cert_chain           covered by aci_session_verified: the audit of each of the 1 pinned sessions checked it  [ENFORCED]
  ✓ tee_tcb_current          covered by aci_session_verified: …  [ENFORCED]
  ✓ nvidia_signature         covered by aci_session_verified: …  [ENFORCED]
  ✓ compose_binding          covered by aci_session_verified: …  [ENFORCED]
  ✓ event_log_integrity      covered by aci_session_verified: …  [ENFORCED]
  - tee_hardware_config      no parseable TEE quote/report; cannot check hardware config  [ALLOWED]
  - nvidia_nonce_client_bound  no NVIDIA payload; nonce not checked  [ALLOWED]
  ✓ aci_session_verified     1 attested sessions pinned … (2 refused; first: teep does not verify the evidence of private-ai-verifier/tinfoil/v1)  [ENFORCED]
  ✓ aci_receipt_verified     receipt verified, including the request hash  [ENFORCED]
Score: 34/47 passed, 6 skipped, 7 failed (0 enforced, 7 allowed), 7 n/a

Before this change, the same route ran with 35 entries. Of the 21 dropped, 15 are factors that the session audit checks; the other 6 (e2ee_capable, response_schema, sigstore_verification and three supply chain factors) pass or are N/A on this route.

Checklist

  • I added unit tests for new code and regression tests for fixed bugs.
  • go test -short ./cmd/... ./internal/... and golangci-lint pass. make coverage not run.
  • For a major feature: make integration and make reports pass. Not run; teep verify phalacloud was run live for this model only.
  • I updated the documentation that this change affects.

Pin the gateway compose hash (e2ac2e8, 1c7d77d)

gateway_compose_binding showed that the gateway's app_compose matches its MRConfigID, but accepted any compose. A new gateway_compose_hash_allow policy key lists the sha256(app_compose) values the gateway may run. When set, gateway_compose_binding fails unless the binding holds and the hash is listed; a response with no app_compose fails instead of skipping. When absent, nothing changes. It merges like the other gateway allowlists, and --update-config keeps it.

Live on tee.redpill.ai, deepseek/deepseek-v4.1-flash-1017:

gateway_compose_hash_allow = ["04d79a23…6085"]
  ✓ gateway_compose_binding  gateway sha256(app_compose) matches MRConfigID; compose hash 04d79a23…6085 in gateway_compose_hash_allow  [ENFORCED]   exit 0
gateway_compose_hash_allow = ["0000…0000"]
  ✗ gateway_compose_binding  gateway compose hash 04d79a23…6085 not in gateway_compose_hash_allow  [ENFORCED]   exit 1

The pinned hash is observed live, not yet reproduced from source. The live compose sets PRIVATE_AI_GATEWAY_REPO_COMMIT=8d0018f81e8dbadfe7b733bb294a0350f82b338c, and the launcher (ghcr.io/redpill-ai/private-ai-launcher@sha256:c083ff9e…, GHCR tag 0.2.1, redpill-ai/private-ai-launcher a64f4d4) builds the gateway from that commit at boot. Nobody has rebuilt the launcher to that digest, and the CI log that printed it has expired.

Posted with Claude.

Phala Cloud's model CVMs run dstack-0.6.3. On 2026-11-19 the session of
deepseek/deepseek-v4.1-flash-1017 reported MRTD 83baf004... and MRSEAM
eddfa0c1..., and the session audit refused it at the MRTD check.

- MRTD 83baf004... is the TDX MRTD that dstack publishes for dstack-0.6.3,
  0.6.4 and 0.6.5 (image-hashes.txt of mkosi-os-v0.6.3 and v0.6.4, release
  notes of mkosi-os-v0.6.5).
- MRSEAM eddfa0c1... is the sha384 of Intel's signed TDX module 2.0.20
  binary and the hash in its sigstruct (tdx-module.binaries 73b2a66e).
  Intel's hash table gives 57a1a66f... for 2.0.20; the same two methods
  give the table value for 2.0.08.

Both lists are shared by every dstack provider.
dstack 0.6 sets MR_CONFIG_ID version 3 for a launch with a key provider:
0x03, sha256("dstack-mr-config-v3:" || 0x00 || document), 15 zero bytes.
VerifyComposeBinding accepts only version 1, so the session audit refused
every Phala model CVM on dstack 0.6 at the compose binding.

VerifyComposeBindingV3 builds the document that the dstack VMM builds
(prepare_mr_config_v3): compose_hash from sha256(app_compose),
key_provider, key_provider_id and the init script hashes from app_compose,
app_id, instance_id and gpu_policy_hash from the RTMR3 events. The audit
now replays the event log first, so the values it reads are the ones the
quote binds.

TestSessionAuditPhalaModelCVMDstack063 runs the captured session of
2026-11-19 through the audit; it fails on main at the MRTD check, and with
the measurements only at the compose binding.

Fixture: With dstack 0.6 sessions verified, three Phala model CVM sessions of the
2026-11-13 capture pass the audit, and the pinned set no longer matches the
captured probe request. The new capture of qwen/qwen3.8-27b
(2026-11-19T20:08:56Z) replaces it. The receipt fixture, citedSession and
liveReceiptID come from the same capture; the receipt cites the Phala
model CVM session 2fe1110a....
The canonical ACI/1 report of Phala Cloud carries no quote of the
inference host: that evidence is in the attested sessions, which the
enforced aci_session_verified audits. The core factors failed with "no TEE
quote", so the default allow_fail waived them, and a check that the audit
does not make looked the same as one it does.

SessionAuditor.Covers names the factors that the audit checks for a
session: quote structure, certificate chain, signature, debug bit,
MRTD/MRSEAM, REPORTDATA, the NVIDIA payload, signature and claims, online
the collateral and TCB, and for the Phala model CVM the event-log replay
and the compose binding. AuditSessions keeps the factors that every pinned
session covers, and BuildReport passes those with aci_session_verified
when the response has no quote of its own. Every other factor keeps its
own result: the RTMR policy, the client nonce of the GPU evidence, NRAS,
cpu_gpu_chain, the weights and the components still fail.

PhalaCloudACIDefaultAllowFail no longer waives the factors that every
session covers online. event_log_integrity and compose_binding stay
allowed, because a Chutes session does not cover them.
gateway_compose_binding checks that sha256(app_compose) matches the
gateway MRConfigID, but any compose passes: a gateway that redeploys with
a new source pin or new sidecar images still verifies.

gateway_compose_hash_allow lists the sha256(app_compose) values that the
gateway may run, 32 bytes of hex. When it is set, gateway_compose_binding
fails unless the binding holds and the hash is in the list; with no
app_compose in the response it fails instead of skipping. When it is
absent, nothing changes. It merges like the gateway measurement
allowlists, so a provider policy that sets only this key is kept.
UpdateConfig decodes the file into updatePolicy and writes it back, so a
key the struct does not name is dropped. updatePolicy now carries
gateway_compose_hash_allow, which --update-config never adds to.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant