Repository navigation
Conversation
Phala Cloud's model CVMs run dstack-0.6.3. On 2026-11-19 the session of deepseek/deepseek-v4.1-flash-1017 reported MRTD 83baf004... and MRSEAM eddfa0c1..., and the session audit refused it at the MRTD check. - MRTD 83baf004... is the TDX MRTD that dstack publishes for dstack-0.6.3, 0.6.4 and 0.6.5 (image-hashes.txt of mkosi-os-v0.6.3 and v0.6.4, release notes of mkosi-os-v0.6.5). - MRSEAM eddfa0c1... is the sha384 of Intel's signed TDX module 2.0.20 binary and the hash in its sigstruct (tdx-module.binaries 73b2a66e). Intel's hash table gives 57a1a66f... for 2.0.20; the same two methods give the table value for 2.0.08. Both lists are shared by every dstack provider.
dstack 0.6 sets MR_CONFIG_ID version 3 for a launch with a key provider:
0x03, sha256("dstack-mr-config-v3:" || 0x00 || document), 15 zero bytes.
VerifyComposeBinding accepts only version 1, so the session audit refused
every Phala model CVM on dstack 0.6 at the compose binding.
VerifyComposeBindingV3 builds the document that the dstack VMM builds
(prepare_mr_config_v3): compose_hash from sha256(app_compose),
key_provider, key_provider_id and the init script hashes from app_compose,
app_id, instance_id and gpu_policy_hash from the RTMR3 events. The audit
now replays the event log first, so the values it reads are the ones the
quote binds.
TestSessionAuditPhalaModelCVMDstack063 runs the captured session of
2026-11-19 through the audit; it fails on main at the MRTD check, and with
the measurements only at the compose binding.
Fixture: With dstack 0.6 sessions verified, three Phala model CVM sessions of the
2026-11-13 capture pass the audit, and the pinned set no longer matches the
captured probe request. The new capture of qwen/qwen3.8-27b
(2026-11-19T20:08:56Z) replaces it. The receipt fixture, citedSession and
liveReceiptID come from the same capture; the receipt cites the Phala
model CVM session 2fe1110a....
This was referenced Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
teep verify phalacloud --model deepseek/deepseek-v4.1-flash-1017fails the enforcedaci_session_verified, soteep serveblocks every request to the model. Observed on 2026-11-19 at 6548996.The gateway lists three attested sessions for the model. teep has no verifier for two of them (
private-ai-verifier/tinfoil/v1,private-ai-verifier/near-ai-gateway/v1). The third is Phala's own model CVM (private-ai-verifier/phala-direct/v1), and its audit fails at three checks in turn:83baf004a06c83f6…dstackMRTDAlloweddfa0c1750b39ba…dstackMRSEAMAllow03cbc87e…VerifyComposeBindingaccepts only version 1 (0x01‖ sha256(app_compose))This is not the first dstack 0.6 CVM at Phala. The current
phalacloud_acifixture (qwen/qwen3.8-27b, captured 2026-11-13) holds three Phala model CVM sessions (qwen38-27b-{0,1,2}.usc2.phala.com) with the same MRTD and a version 3 MRConfigID, on TDX module 1.5.24. teep refused all three.The model CVM runs
dstack-0.6.3(vm_configimage, os_image_hash38a6f97b…) on a 4-GPU B300 host. dstack 0.6 sets MR_CONFIG_ID version 3 for a launch with a key provider (mr_config_versionindstack/vmm/src/app/mr_config.rs).The session audit reads the hard-coded dstack lists (
sessionMeasurementPolicy,internal/provider/phalacloud/session_audit.go), notmrtd_allowormrseam_allowfrom the config, so a user cannot add the values. #190 plans signed allowlist updates; until then I followed #183 and added the values to the built-in lists with their sources.The report shows only the first refusal (
refusalSummary,internal/attestation/report.go:2267). For this model that is the Tinfoil session, so the report never showed why the Phala session failed. I found the three reasons with a local build that prints every refusal, adding one value at a time. I did not include that change here. Listing every refusal reason, or at least the reason of each session that teep has a verifier for, would have shown the cause directly.Change
dstackMRTDAllow: add83baf004…, the TDX MRTD that dstack publishes fordstack-0.6.3,0.6.4and0.6.5:image-hashes.txtof themkosi-os-v0.6.3andmkosi-os-v0.6.4releases, and the release notes ofmkosi-os-v0.6.5. The three images share one MRTD. The live session's os_image_hash is the0.6.3value in that file.dstackMRSEAMAllow: addeddfa0c1…for TDX module 2.0.20, from Intel's signed binary. See the point of discussion.attestation.VerifyComposeBindingV3: checks an MRConfigID of version 3. It is0x03‖ sha256("dstack-mr-config-v3:"‖0x00‖ document) ‖ 15 zero bytes (MrConfigV3indstack-types/src/mr_config.rs). The quote does not carry the document, so teep builds the one that the VMM builds (prepare_mr_config_v3):compose_hash= sha256(app_compose);key_provider,key_provider_idand, from manifest version 3, the sha256 of each init script come from app_compose;app_id,instance_idandgpu_policy_hashcome from the RTMR3 events. teep builds one document and does not try variants.checkModelCVMIntegrityreplays the event log first, so the RTMR3 values it reads are the ones the quote binds. It then checks the compose binding: version 3 withVerifyComposeBindingV3, otherwiseVerifyComposeBindingas before.internal/provider/aci/testdata/session_phala_direct_0_6_3.json, the captured session record.TestSessionAuditPhalaModelCVMDstack063runs it throughSessionAuditor.Audit. With only this test and the fixture applied to 6548996, it fails withMRTD 83baf004… is not allowed. On the first commit it fails withcompose binding: MRConfigID does not match app_compose hash. (The full test file does not build on main, becauseTestComposeBindingV3Refusescalls the new function.)TestComposeBindingV3Refuseschanges each input of the document (app_compose, the GPU flag, the app-id and instance-id events, a second app-id event) and requires a refusal.dstack_defaults_test.goare updated.phalacloud_acifixture: with this change the three 2026-11-13 Phala sessions above pass. The captured probe request had pinned only four sessions, soTestVerifyRun_PhalaCloudCanonical_Fixturefailed. I replaced the fixture with a new capture ofqwen/qwen3.8-27b(2026-11-19T20:08:56Z), asdocs/transport/testing.mddescribes, and updatedfixtureSessionIDs. The receipt fixture (internal/provider/aci/testdata/receipt.json),citedSessionandliveReceiptIDnow come from the same capture. The receipt cites the Phala model CVM session2fe1110a…(qwen3-8-27b.usw7-d9.phala.com), which runs dstack 0.6.4 on TDX module 2.0.20, with a version 3 MRConfigID. The refreshed fixture therefore needs both new values and the version 3 check, including the MRSEAM in the point of discussion.docs/measurement_allowlists.md, and check 6 of the session audit and its test list indocs/providers/phalacloud/phalacloud_support.md. teep requires app_compose to namekey_providerand to givemanifest_versionas a string. dstack also accepts a compose withoutkey_provider(it derives the provider fromkms_enabled) or with a numeric version. teep refuses those composes, and the doc says so.Effect
teep verify phalacloud --model deepseek/deepseek-v4.1-flash-1017pins the Phala model CVM session, and the probe passes with E2EE v2 and a verified receipt. The Tinfoil and NEAR sessions are still refused.Two commits: the measurements, then the version 3 check with the fixture refresh. Without the MRSEAM, the
deepseek/deepseek-v4.1-flash-1017session and the refreshed fixture's cited session stay refused, so dropping it also needs another fixture.The MRTD and MRSEAM lists are shared, so the two values also reach Venice, NanoGPT, NEAR Direct, NEAR Cloud (model and gateway), Tinfoil and Chutes. Only the Phala session audit reads a version 3 MRConfigID. The other dstack compose checks (
verify.go:267,attest.go:197,proxy.go:1279,proxy.go:1424) still accept version 1 only and will refuse a dstack 0.6 CVM with a key provider.To verify
Live, 2026-11-19, with
PHALA_API_KEYset:On main the same command shows
✗ aci_session_verified … (3 refused; first: teep does not verify the evidence of private-ai-verifier/tinfoil/v1)andno probe: the report blocks inference requests.To check the sources:
tdx_mrtdinhttps://github.com/Dstack-TEE/dstack/releases/download/mkosi-os-v0.6.3/image-hashes.txt, the same inmkosi-os-v0.6.4, and theTDX MRTDrow of themkosi-os-v0.6.5release notes.sha384sum separate_files/2.0/2.0.20/tdx_module_2.0.20.bininintel/confidential-computing.tdx.tdx-module.binariesat73b2a66e, and bytes 900-947 oftdx_module_2.0.20.sigstruct. Both methods give the table value for 2.0.08 (476a2997…).The new capture's
teep verifyexited 1 at its capture self-check:--reverifyreports the recorded receipt as a deferred skip (#211), so the report differs onaci_receipt_verified. The 2026-11-13 fixture shows the same difference with main's binary, so I did not treat it as a regression. The capture holds no API key (scanned).With this change, a chat request with
toolsto this model is still refused while E2EE is on: E2EE v2 has no field path for tool fields. That is #240.Point of discussion
MRSEAM of TDX module 2.0.20. Intel publishes two different values:
intel/confidential-computing.tdx.tdx-module, commit859989ac, 2026-11-12) and theTDX_MODULE_2.0.20release notes give57a1a66f3704887a….eddfa0c1750b39ba6c9b039c579a5039baea307f8290fc5e3636d70e6001749dc95a368b9d8a9c50c84d1c22b5642d3c. Phala's quote reports this value. The quote's TEE_TCB_SVN (0903…) matches the 2.0.20 release notes, but 2.0.18 has the same SVNs, so it does not separate the two.1.5.36, released the same day, has the same mismatch: the table gives
06b9e658…and the binary hashes to28254b2a…. I added only the 2.0.20 value, because it is the one in use, labeled with where it comes from. Do you want to accept a value that comes from Intel's signed binary and not from its table, or wait until Intel corrects one of the two? #139 (move MRSEAM to tracking) would make this question smaller.Checklist
make checkpasses (golangci-lint: 0 issues; every package ok), andmake coveragepasses (86.5% total).make integrationandmake reportspass. Not run;teep verify phalacloudwas run live for this model only.Posted with Claude.