Repository navigation
Conversation
Phala Cloud's model CVMs run dstack-0.6.3. On 2026-11-19 the session of deepseek/deepseek-v4.1-flash-1017 reported MRTD 83baf004... and MRSEAM eddfa0c1..., and the session audit refused it at the MRTD check. - MRTD 83baf004... is the TDX MRTD that dstack publishes for dstack-0.6.3, 0.6.4 and 0.6.5 (image-hashes.txt of mkosi-os-v0.6.3 and v0.6.4, release notes of mkosi-os-v0.6.5). - MRSEAM eddfa0c1... is the sha384 of Intel's signed TDX module 2.0.20 binary and the hash in its sigstruct (tdx-module.binaries 73b2a66e). Intel's hash table gives 57a1a66f... for 2.0.20; the same two methods give the table value for 2.0.08. Both lists are shared by every dstack provider.
dstack 0.6 sets MR_CONFIG_ID version 3 for a launch with a key provider:
0x03, sha256("dstack-mr-config-v3:" || 0x00 || document), 15 zero bytes.
VerifyComposeBinding accepts only version 1, so the session audit refused
every Phala model CVM on dstack 0.6 at the compose binding.
VerifyComposeBindingV3 builds the document that the dstack VMM builds
(prepare_mr_config_v3): compose_hash from sha256(app_compose),
key_provider, key_provider_id and the init script hashes from app_compose,
app_id, instance_id and gpu_policy_hash from the RTMR3 events. The audit
now replays the event log first, so the values it reads are the ones the
quote binds.
TestSessionAuditPhalaModelCVMDstack063 runs the captured session of
2026-11-19 through the audit; it fails on main at the MRTD check, and with
the measurements only at the compose binding.
Fixture: With dstack 0.6 sessions verified, three Phala model CVM sessions of the
2026-11-13 capture pass the audit, and the pinned set no longer matches the
captured probe request. The new capture of qwen/qwen3.8-27b
(2026-11-19T20:08:56Z) replaces it. The receipt fixture, citedSession and
liveReceiptID come from the same capture; the receipt cites the Phala
model CVM session 2fe1110a....
The canonical ACI/1 report of Phala Cloud carries no quote of the inference host: that evidence is in the attested sessions, which the enforced aci_session_verified audits. The core factors failed with "no TEE quote", so the default allow_fail waived them, and a check that the audit does not make looked the same as one it does. SessionAuditor.Covers names the factors that the audit checks for a session: quote structure, certificate chain, signature, debug bit, MRTD/MRSEAM, REPORTDATA, the NVIDIA payload, signature and claims, online the collateral and TCB, and for the Phala model CVM the event-log replay and the compose binding. AuditSessions keeps the factors that every pinned session covers, and BuildReport passes those with aci_session_verified when the response has no quote of its own. Every other factor keeps its own result: the RTMR policy, the client nonce of the GPU evidence, NRAS, cpu_gpu_chain, the weights and the components still fail. PhalaCloudACIDefaultAllowFail no longer waives the factors that every session covers online. event_log_integrity and compose_binding stay allowed, because a Chutes session does not cover them.
gateway_compose_binding checks that sha256(app_compose) matches the gateway MRConfigID, but any compose passes: a gateway that redeploys with a new source pin or new sidecar images still verifies. gateway_compose_hash_allow lists the sha256(app_compose) values that the gateway may run, 32 bytes of hex. When it is set, gateway_compose_binding fails unless the binding holds and the hash is in the list; with no app_compose in the response it fails instead of skipping. When it is absent, nothing changes. It merges like the gateway measurement allowlists, so a provider policy that sets only this key is kept.
UpdateConfig decodes the file into updatePolicy and writes it back, so a key the struct does not name is dropped. updatePolicy now carries gateway_compose_hash_allow, which --update-config never adds to.
This was referenced Nov 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #239: the first two commits are #239's; this PR adds the third.
Problem
teep verify phalacloudreads the canonical ACI/1 report (/v1/aci/attestation). It carries the gateway's quote and no quote of the inference host. The evidence of the inference host is in the attested sessions, whichaci_session_verifiedaudits and enforces. The core factors of the inference host still evaluate against the response:tee_quote_present,tee_cert_chain,tee_reportdata_binding,intel_pcs_collateral,event_log_integrity,compose_binding, thenvidia_*factors and others fail with "no TEE quote" or "ACI/1 publishes no compose manifest".PhalaCloudACIDefaultAllowFailwaives them, so a user's table waives them too. The report then shows a check that the session audit makes (the quote's certificate chain) the same way as one it does not make (an RTMR policy). The second kind is hidden by the waivers of the first.The compose detail is also out of date for the Phala model CVM: its session evidence carries
tcb_info.app_compose, and since #239 the audit checks it against MRConfigID version 1 or 3.Change
SessionAuditor.Covers(s)(session_audit.go) names the report factors thatAuditchecks for a session. For every session:tee_quote_present,tee_quote_structure,tee_cert_chain,tee_quote_signature,tee_debug_disabled,tee_measurement(the built-in dstack or Chutes MRTD and MRSEAM list),tee_reportdata_binding(the session's signing key and the gateway's evidence nonce),nvidia_payload_present,nvidia_signatureandnvidia_claims. Online only:intel_pcs_collateral,tee_tcb_currentandtee_tcb_not_revoked(the audit requires UpToDate). For the Phala model CVM only:event_log_integrityandcompose_binding.AuditSessionssetsACISessionResult.Coveredto the factors that every pinned session covers.BuildReportcallsapplyACISessionCoveragewhen the report is the canonical ACI/1 report, has no quote of its own, and at least one session is pinned. Each factor inCoveredpasses with the detailcovered by aci_session_verified: …. Every other factor keeps its own result. With no pinned session, nothing changes.PhalaCloudACIDefaultAllowFailno longer lists the factors that every session covers online, so they are enforced.event_log_integrityandcompose_bindingstay allowed, because a Chutes session does not cover them.nvidia_signatureis enforced;nvidia_nonce_client_boundstays allowed.phalacloud_support.md, Factor enforcement: what ACI mode reports and what the audit does not check.What the audit does not check, and what therefore still fails in ACI mode: the RTMR values of the inference host (
tee_hardware_config,tee_boot_config; the event log is replayed, but no RTMR policy exists), the client nonce in the GPU evidence (the session's GPU evidence answers the gateway's nonce), NRAS,cpu_gpu_chain,measured_model_weights,component_recognitionandcpu_id_registry.The E2EE authorization is unchanged: for Phala Cloud it reads
gateway_tee_reportdata_binding(GatewayBindsE2EEKey), nottee_reportdata_binding.Tests
TestACISessionCoverage(internal/attestation): a covered factor passes and is enforced; an uncovered factor (tee_quote_signaturewhen only the chain is covered,tee_hardware_config,nvidia_nonce_client_bound,cpu_gpu_chain) does not pass, and an enforced one blocks; with no pinned session, or a session error, the covered factors fail; a response with its own quote is not in ACI mode.TestAuditSessionsCovered(internal/provider/phalacloud): the Phala model CVM session coverscompose_bindingandevent_log_integrity; with a Chutes session also pinned, offline, neither they nor the TCB factors are covered. Every factor that a session covers online is enforced byPhalaCloudACIDefaultAllowFail, and the uncovered ones stay allowed.TestVerifyRun_PhalaCloudCanonical_Fixture: the fixture (Phala model CVM and Chutes sessions) reportstee_quote_present,tee_quote_signature,tee_reportdata_bindingandnvidia_signatureas enforced passes covered by the audit, andcompose_bindingandevent_log_integrityas allowed failures.TestBuildReportCanonicalBody: without an audited session,nvidia_signatureis now an enforced failure.go test -short ./cmd/... ./internal/...: ok for every package.golangci-lint run ./cmd/... ./internal/...: 0 issues. teeplint: 0 failed.Effect
Live, 2026-11-20,
teep verify phalacloud --model deepseek/deepseek-v4.1-flash-1017withe2ee = falseand thisallow_fail:exits 0:
Before this change, the same route ran with 35 entries. Of the 21 dropped, 15 are factors that the session audit checks; the other 6 (
e2ee_capable,response_schema,sigstore_verificationand three supply chain factors) pass or are N/A on this route.Checklist
go test -short ./cmd/... ./internal/...andgolangci-lintpass.make coveragenot run.make integrationandmake reportspass. Not run;teep verify phalacloudwas run live for this model only.Pin the gateway compose hash (e2ac2e8, 1c7d77d)
gateway_compose_bindingshowed that the gateway'sapp_composematches its MRConfigID, but accepted any compose. A newgateway_compose_hash_allowpolicy key lists the sha256(app_compose) values the gateway may run. When set,gateway_compose_bindingfails unless the binding holds and the hash is listed; a response with noapp_composefails instead of skipping. When absent, nothing changes. It merges like the other gateway allowlists, and--update-configkeeps it.Live on
tee.redpill.ai,deepseek/deepseek-v4.1-flash-1017:The pinned hash is observed live, not yet reproduced from source. The live compose sets
PRIVATE_AI_GATEWAY_REPO_COMMIT=8d0018f81e8dbadfe7b733bb294a0350f82b338c, and the launcher (ghcr.io/redpill-ai/private-ai-launcher@sha256:c083ff9e…, GHCR tag 0.2.1, redpill-ai/private-ai-launcher a64f4d4) builds the gateway from that commit at boot. Nobody has rebuilt the launcher to that digest, and the CI log that printed it has expired.Posted with Claude.