fix(state-node): relay reads to members with member-side authorization; fix read auth + phantom history

[somasekimoto]

Summary

Fixes the read path for content a gateway-facing node does not replicate (bug #93 follow-up). After #53, write placement picks members by capacity/XOR distance, so the node the gateway talks to is often not a member of a content network. Writes relay correctly, but reads were broken in three stacked ways — found while E2E-testing the Drive UI (#47) against the 4-node AWS deployment:

1. /state/* reads always 401'd. The SDK signs write requests itself via monas-account (Authorization: user:<hex> + P-256 signature), but forwarded the caller's (empty) auth headers for reads. → The SDK now signs read requests via the account, exactly like the write path. Explicit caller Authorization is still passed through.

2. Phantom history defeated local-presence checks. crsl-lib's linear_history(genesis) returns [genesis] even when the node holds nothing, so ensure_content_local never pulled, and sync_from_peers requested ops since=genesis — which providers answer by skipping the Create operation, so a member that missed the initial push synced "0 operations from 2 providers" forever (observed on node4). → Both now gate on has_genesis (actual DAG node lookup).

3. Non-member reads had no authorized path. ensure_content_local pulled ops via the read-only FetchOperations RPC, but members (correctly) reject that pull for non-members — the two #93-era fixes contradicted each other. Rather than dropping the membership gate, this implements the hardened read-relay the old SECURITY NOTE planned:

   • New ReadContent / ReadHistory relay RPCs carry the original caller's token, request signature and timestamp.
   • The member re-authenticates the caller and enforces the access policy (StateNodeService::authorize_read, now shared with the HTTP read path) before serving data / version / history.
   • Read endpoints relay when has_genesis is false; ensure_content_local is removed. Operations are never copied to non-members, and FetchOperations keeps its membership gate.
   • Member verdicts map back to typed errors so the relaying node returns the member's 401/403/404.

Guarantees, and what this PR does not cover

This PR makes reads reachable and authorized. It does not make relayed responses verifiable — a relaying node still has to take the member's word for the bytes. Response verification is PR #56, which stacks on this branch.

Do not merge #54 on its own. Without #56 there is no client-side check that the returned payload matches the requested version, so a hostile member can serve forged ciphertext. The two are a single safe unit; merge #54 → #56 in order.

Known limits tracked separately:

• [Security] relay read: 未証明の DHT 候補を member として扱い、credential 転送と認可否定の受理を行っている #63 — relay candidates discovered via the DHT are not proven to hold the content. This PR stops trusting their verdicts (see below) but does not add authenticated member discovery, which is what would actually close it.
• [Security] access policy 欠落時に read を許可している(fail-open)。genesis と policy が原子的でない #64 — a content's genesis and its owner policy are created as separate operations, so a replica can legitimately hold the content with no policy. The fail-open half is fixed here; the atomicity half is not.
• [Security] 版メタデータの trust anchor: 観測者が偽 parents で読み単調性を bypass できる #59 — version/writer/latest authenticity. Out of scope for this stack.

Deep-review fixes

Round 1 (207f0af)

• Typed relay verdicts: relayed-read failures cross the wire as ContentResponse::ReadFailed { kind, message } (RelayReadErrorKind in the port layer) — no more substring matching on stringified errors.
• authorize_read fails closed on a policy-store error instead of falling through to the no-policy allow.
• get_version scoped to the content series: the version CID must belong to the authorized content's DAG, closing a cross-content version read (applied to local and relayed paths).
• In-repo tests for previously E2E-only paths: relay-read happy path, typed 403 mapping, authorize_read owner/no-policy/deny/fail-closed, phantom-history sync gating, cross-content get_version rejection, fetch_encrypted contract, revoke-by-remote_content_id.

Round 2 — missing access policy is now a denial (c5afba0)

authorize_read allowed the read when the content had no access policy at all. That state is not proof the content is public: it is indistinguishable from "the owner policy operation has not replicated here yet", which is reachable because genesis and owner policy are separate operations (#64). A replica in that state served ciphertext and history to anyone. Missing policy now denies, with an error that tells the caller to retry or read from a node that has the policy. Tests cover both a missing policy and a genesis-only replica.

Round 3 — unproven DHT peers are no longer trusted as members (9e05c59)

A relay hands the caller's token and request signature to whatever resolve_members returns, then interprets the answer. That list has two very different origins and the code treated them identically:

• a local ContentNetwork record, built from this content's ContentCreated event — the nodes the network actually assigned
• DHT neighbours of sha256(content_id) — nothing ties them to the content, so anyone who can place a Peer ID near the key lands in the list

resolve_members now reports which it produced, and that provenance decides two things.

Auth verdicts from unproven peers no longer end the member loop. A 401/403 used to end it unconditionally. From an unproven peer that proves nothing — one hostile node squatting near the DHT key could deny every read and write by answering 403 first, and an honest but partially-synced replica can return 403 from a policy it has not finished replicating, stopping failover to a healthy one. Such a verdict is now remembered as the answer of last resort while the loop continues. (At this point the early exit was kept for local-record members; Round 5 removes it there too — see below.)

Credentials are still forwarded to every candidate, and that is correct. The member is the one that evaluates authorization, and it cannot do that without the caller's token and signature. It is also safe: authorization is proof-of-possession — the token is either a self-contained key id (a public key) or a delegated JWT whose aud is likewise a public key, and the request signature is verified against that aud key. A peer that captures both cannot mint a new request without the private key, and the captured signature is bound to one operation, resource, body and timestamp.

An earlier revision of this PR capped how many unproven candidates were tried. That was removed: a peer sitting first in DHT distance order receives the credentials regardless of any cap, so the cap only cut failover to legitimate members — an availability cost for no confidentiality gain.

The real remaining gap is member discovery itself — see Round 4, which closes part of it. What is left is #63, and docs/design.md states it rather than implying it is solved.

Round 4 — the publisher of a membership event is authenticated (8201797)

Round 3 leaned on the local ContentNetwork record being a better basis than a DHT guess. Checking that assumption showed it did not hold: ContentCreated and ContentNetworkManagerAdded were applied with no origin check at all, so any peer could plant a record on this node by naming it in member_nodes — and that record then read back as the trusted kind, which is what decides whether a 403 ends the member loop. ContentUpdated already called verify_source_peer_id; these two arms did not.

The event's publisher is now bound:

• ContentCreated must come from the creator_node_id it names. create_content always publishes with creator_node_id == local_node_id, so every legitimate event satisfies this.
• A membership change must come from a node already in the network it changes. ContentNetworkManagerAdded names no publisher (added_node_id is the node being added, usually us), so membership is the check that fits. With no local record there is nothing to check against and nothing to overwrite, so it is accepted as bootstrap.

ReceivedEvent::source now carries gossipsub's Message::source — the authenticated publisher — instead of propagation_source, the peer that forwarded it. The mesh forwards over multiple hops, so binding on the forwarder would reject honest multi-hop delivery and accept forged origins. Under MessageAuthenticity::Signed + ValidationMode::Strict the author field is required and signature-verified before delivery, so a forwarder cannot alter it.

MemberProvenance::Attested is renamed to LocalRecord. The old name claimed a cryptographic attestation that does not exist: events carry no owner signature, so the member set is the publisher's own claim even now that the publisher is authenticated. Two gaps remain, both needing owner-signed membership — a protocol change tracked in #63: the first record for a content has no prior membership to check against, and an authenticated member can still declare a member set of its choosing.

Round 5 — the remaining two arms, and the verdict that rested on them (746d1ad)

Round 4 hardened the two arms a review had named and stopped there. Enumerating all six arms of handle_sync_event found two more that mutate the ContentNetwork record with no publisher authorization:

• ContentNetworkManagerRemoved had no check whatsoever. Any peer could delete our record by naming us as removed_node_id.
• ContentDeleted called verify_source_peer_id against deleted_by_node_id — but the event names its own publisher, so any authenticated peer satisfied it and could delete our record.

Both now require the publisher to be a member of the network being changed, the same rule as the Added arm.

auth_verdict_is_authoritative now always returns false. It returned true for LocalRecord on the grounds that a listed member had evaluated the caller against the real access policy. That reasoning does not survive the above: the first record for a content is accepted with no prior membership to check against, so an attacker who wins that race lands in the candidate list, and its 403 would end the failover loop — a permanent denial of service against a legitimate caller, which is precisely the attack the DhtGuess case already guards against. The denial is still kept and returned when no candidate produces anything better; only the early exit is gone. The cost is bounded — one extra round trip per remaining candidate on a genuine denial — and errs toward availability, which is the right direction: the caller is refused either way, just later. Restoring the early exit needs owner-signed membership (#63).

provenance is kept and marked #[allow(dead_code)] rather than deleted — it is what #63 will read to restore the distinction.

The test named attested_member_auth_verdict_is_final passed a hardcoded true, so it never exercised the policy at all, only record_relay_read_error's mechanics. Renamed to say what it actually checks, and to record that no caller passes true today.

Verification

• cargo test --workspace: all green, no failing test binaries.
• Rust 1.97 cargo clippy --workspace --all-targets --deny warnings: clean.
• Every new negative test in Rounds 4 and 5 was confirmed to fail with its check disabled — none are vacuous.
• Rounds 4 and 5 verified on four real nodes (merged into feat(read-integrity): verified read path — payload authenticity, sender-authenticated CEK envelopes, single-use mutation signatures #56, which has the 4-node script): e2e 8/8, auth script 9/9, content propagated to all four, and zero membership-rejection warnings across all four node logs — the tightened checks accept every legitimate event in a real multi-hop mesh.
• Deployed to the 4-node AWS environment and verified end-to-end through the Drive UI (feat(example-ui): minimal Drive-like UI on monas-sdk/gateway #47) with Playwright: create-in-folder (the old 408 repro), preview with version history, verify-integrity via relayed version read on a non-member node, edit, HPKE share/revoke, delete. (This predates Round 4.)

Note: this branch's start-local-nodes.sh still starts 3 nodes, so the local e2e script cannot satisfy min_replication_factor = 3 (the creator is excluded from the member set, leaving 2). The 4-node standardisation lives in #56; the full local e2e passes there. This is pre-existing on this branch and not introduced by the review fixes.

Note on edd429f (also in this PR): it adds two public SDK API fields — VerifyIntegrityInput.local_content_id (verify against the locally stored ciphertext; the state node never sees plaintext) and RevokeShareInput.remote_content_id (the state node only knows the series id). Both optional, backward compatible.

🤖 Generated with Claude Code

[somasekimoto]

Added edd429f with two more SDK fixes found while completing the UI E2E against the deployed 4-node environment:

1. verify_integrity compared the caller's plaintext with the state node's stored bytes — the state node only ever holds the ciphertext the SDK sent, so the check could never return valid. VerifyIntegrityInput gains an optional local_content_id; when present the SDK byte-compares the state node's version data against its locally stored ciphertext (new ContentService::fetch_encrypted).
2. revoke_share addressed the post-revoke re-encryption PUT with the SDK-local version id, which the state node doesn't know — the sync failed and the revoke rolled back. RevokeShareInput gains remote_content_id (same local/remote split as UpdateContentInput).

Final verification: full Drive-UI Playwright E2E against the redeployed nodes — 16/16 steps green, zero error toasts (account → folder → create-in-folder → history via relay → integrity valid → edit → share w/ HPKE decrypt proof → revoke → delete).

[Yu-da-1]

[セキュリティ] DHTフォールバックで得た未検証ピアに実credentialを転送し、応答を無検証で信頼している

問題

resolve_members は、ローカルに ContentNetwork レコードがない場合、Kademlia DHT の近接ピア (find_closest_peers) を「そのコンテンツのメンバー」として扱います。ここには暗号学的な検証やメンバーシップの確認が一切ありません。

let mut members: Vec<String> = match local_record {
    Some(network) => network.member_nodes_as_strings(),
    None => {
        let key = compute_dht_key(content_id);
        self.peer_network
            .find_closest_peers(key, self.min_replication_factor + 1)
            .await
            ...
    }
};

relay_read_data はこの members に対して、呼び出し元の本物の auth_token/request_signature をそのまま転送し（L637）、最初に成功した応答をそのまま信頼します（L640 Ok(result) => return Ok(result)）。応答データに対する署名検証・completeness検証はどこにも存在しません（libp2p_network.rs の応答ハンドラでも素通し、relay_read_content/relay_read_history の実装含め sign/verify/Signature は0件でした）。

再現方法

実は、既存のハッピーパステスト自体がこの挙動を証明しています:

monas/monas-state-node/src/application_service/state_node_service.rs

Lines 2997 to 3030 in 207f0af

	async fn test_relay_read_data_returns_member_payload() {
	// Happy path: the member serves the read and the payload comes back.
	let node_registry = MockNodeRegistry::new();
	let content_repo = Arc::new(RwLock::new(MockContentNetworkRepository::new()));
	let peer_network = Arc::new(
	MockPeerNetwork::new()
	.with_local_peer_id("node-1")
	.with_closest_peers(vec!["node-2".to_string()])
	.with_relay_read_data(b"cipher".to_vec(), "v1"),
	);
	let event_publisher = MockEventPublisher::new();
	let crdt_repo = Arc::new(MockContentRepository::new());
	let service: TestService = StateNodeService::new(
	node_registry,
	content_repo,
	peer_network,
	event_publisher,
	crdt_repo,
	"node-1".to_string(),
	);
	let result = service
	.relay_read_data(
	"content-1",
	None,
	&test_token(),
	Some(&test_request_signature()),
	None,
	)
	.await
	.expect("relayed read should succeed");
	assert_eq!(result, (b"cipher".to_vec(), "v1".to_string()));
	}

async fn test_relay_read_data_returns_member_payload() {
    let content_repo = Arc::new(RwLock::new(MockContentNetworkRepository::new())); // ← "content-1" のContentNetworkレコードは無い
    let peer_network = Arc::new(
        MockPeerNetwork::new()
            .with_local_peer_id("node-1")
            .with_closest_peers(vec!["node-2".to_string()]) // ← DHTフォールバックで返る「メンバー」
            .with_relay_read_data(b"cipher".to_vec(), "v1"),
    );
    ...
    let result = service.relay_read_data("content-1", None, &test_token(), Some(&test_request_signature()), None).await
        .expect("relayed read should succeed");
    assert_eq!(result, (b"cipher".to_vec(), "v1".to_string()));
}

このテストでは content-1 の正規メンバーが誰かを一度も検証せず、DHTで返ってきた node-2 の回答をそのまま採用しています。node-2 を攻撃者ノードに、with_relay_read_data の内容を攻撃者が用意した任意のバイト列に置き換えても、コード上は全く同じパスを通り、同じように成功します。つまり以下を機械的に示せます:

1. content-1 のContentNetworkレコードを持たないノードAが読み取りリクエストを受ける
2. resolve_members がDHT近接性だけで選ばれた「メンバー」を返す（実際にそのコンテンツを保持しているかは未検証）
3. ノードAはそのピアに、リクエスト元の生きた auth_token/request_signature を転送する（別ホストへのcredential漏洩）
4. そのピアが返す (data, version) をノードAは無条件に信頼し、クライアントへ返す

再現テストとして、以下のように既存パターンをそのまま流用して「攻撃者ノード」を明示した形にできます（差分は node-2 → attacker-node のリネームと意図を示すコメントのみ）:

#[tokio::test]
async fn test_relay_read_data_trusts_unverified_dht_fallback_peer() {
    // resolve_members が DHT フォールバックで返すピアは、実際にそのコンテンツの
    // 正規メンバーであることが一度も検証されない。この攻撃者ノードが返す
    // 任意のペイロードが、そのまま relay_read_data の戻り値として信頼されることを示す。
    let content_repo = Arc::new(RwLock::new(MockContentNetworkRepository::new())); // 正規メンバー情報なし
    let peer_network = Arc::new(
        MockPeerNetwork::new()
            .with_local_peer_id("node-1")
            .with_closest_peers(vec!["attacker-node".to_string()])
            .with_relay_read_data(b"FABRICATED_DATA".to_vec(), "fake-version-cid"),
    );
    let service: TestService = StateNodeService::new(
        MockNodeRegistry::new(), content_repo, peer_network,
        MockEventPublisher::new(), Arc::new(MockContentRepository::new()),
        "node-1".to_string(),
    );

    let result = service
        .relay_read_data("content-1", None, &test_token(), Some(&test_request_signature()), None)
        .await
        .expect("relayed read succeeds even though attacker-node was never a verified member");

    // 検証も membership チェックも無いまま、攻撃者の偽データがそのまま返る。
    assert_eq!(result, (b"FABRICATED_DATA".to_vec(), "fake-version-cid".to_string()));
}

「versionはCID（ハッシュ）だから偽装は自動検出できる」への補足

version/genesis_cid は crsl-lib の Node::content_id()（SHA-256ベースのCID、crdt_repository.rs:527-529）です。正規メンバーの内部ストレージは content-addressed なので正規メンバー自身は嘘をつけませんが、これは relay 境界（ネットワーク層）には及びません。受信側 (relay_read_data, libp2p_network.rs の relay_read_content) は返ってきた (data, version) に対して一度もハッシュ再計算・検証をしていません。ワイヤプロトコル上は ContentResponse::Data{data, version} を直接送るだけなので、攻撃者は正規のcrsl-lib DAGを経由する必要がありません。ハッシュ整合性チェックを追加しても「Mの回答の内部整合性」しか証明できず、「そのversionがX系列の正当な履歴（genesisからの親子チェーン）に属する」ことまでは証明できない点にも注意が必要です。

提案する修正

• DHTフォールバックで得たピアは、relay対象にする前に「本当にそのコンテンツを保持する正当な権利があるか」を検証する（例: ContentNetwork 作成時の正当なメンバー証明、または応答データが genesis からの正当な履歴に属することの検証）。
• 少なくとも、DHTフォールバック経由の未検証ピアには実 auth_token/request_signature を転送しない。

[somasekimoto]

ディープレビュー(2026-07-26)の blocker を実コードで検証し、本 PR のスコープでは解決しない設計課題として issue 化しました(セルフメモ)。

• [Security] relay read: 未証明の DHT 候補を member として扱い、credential 転送と認可否定の受理を行っている #63: 未証明の DHT 候補を member として扱い、credential 転送・応答の無検証受理・未証明ピアの認可否定の権威化を行っている(blocker 1 / 3)
• [Security] access policy 欠落時に read を許可している(fail-open)。genesis と policy が原子的でない #64: access policy 欠落時に read を許可する fail-open。genesis と owner policy が原子的でない(blocker 2)

いずれも「認証された member discovery」「genesis への policy 原子的包含」という設計変更が必要で、本 PR(relay 到達性と member 側認可の修正)の範囲を超えます。

なお #63 の影響範囲は PR #56 のリクエスト署名統一により縮小しています。署名が operation / resource / timestamp / body digest に束縛されたため、転送された署名を別 content や別操作へ転用することはできなくなりました(残るのは同一 content・同一操作の 5 分以内の再実行)。

AES-CTR → AES-GCM への移行が design.md に反映されていない点も指摘を受けたので確認します。

[somasekimoto]

@Yu-da-1 さん

指摘ありがとうございます。既存のハッピーパステストがそのまま攻撃の証明になっている、という示し方が特に的確でした。実際そのテストは今もリネームなしで通ります。

指摘を 3 つに分けて、それぞれ現状を書きます。2 つは未解決のままです。

1. 未検証ピアの応答を無条件に信頼する → #56 で対応、ただし保証範囲は限定的

PR #56 でクライアント側の検証を入れました。返ってきた Node の CID を再計算して要求した版 CID と照合し、AES-GCM で復号します。CEK を持たない相手は復号可能な偽 payload を作れません。

ただしご指摘の但し書きがそのまま当たっています。「ハッシュ整合性チェックを追加しても『M の回答の内部整合性』しか証明できず、『その version が X 系列の正当な履歴に属する』ことまでは証明できない」— その通りで、#56 が保証するのは payload の真正性までです。正規の writer が書いたか、canonical head か、本当に最新かは保証しません。

一度クライアント側に単調性チェック(last-seen 版より後退したら拒否)を実装しましたが、撤去しました。偽の parents を詰めた版で bypass できて防御にならない一方、結果整合性のもとでは正当な sync 遅延と攻撃を応答単体で区別できず、正規の read を壊す誤検知が残るためです。版の真正性には owner/writer 署名の trust anchor が必要で、crsl-lib の Node フォーマットに及ぶプロトコル変更になります。issue #59 で追跡しています。

なお relay 境界そのものは今も無検証で、検証はクライアント側です。これは「state node を信頼しない」設計に沿った配置ですが、relay が嘘をつけないことを意味しません。

2. 未検証ピアへの credential 転送 → 転送は続けます

ここはご提案と違う結論にしました。理由を書きます。

relay は access policy を持たないので、認可判断は member 側が実 policy に対して行います。credential を転送しなければ member 側で認可できず、この経路自体が成立しません。

そのうえで転送が安全と判断したのは、認可が Proof of Possession だからです。token は自己完結型の鍵 ID(公開鍵そのもの)か委譲 JWT で、後者の aud もまた自己完結型の鍵 ID です。リクエスト署名はその aud の鍵に対して検証されるので、token と署名の両方を傍受した相手も aud の秘密鍵を持たない以上、新しいリクエストを作れません。傍受した署名自体も操作・リソース・body digest・timestamp に束縛され、mutation はさらに使い切りです。

途中で「未検証候補は 2 件まで」という上限を実装しましたが、撤去しました。DHT 距離順で先頭に来る相手は上限があろうと credential を受け取るので機密性は改善せず、正当な member への failover を減らして可用性を落とすだけだったためです。

3. 未検証ピアを member 扱いする → 部分的に対応、根本は未解決

resolve_members が出自を報告するようにして、扱いを分けました。さらに調べたところ、ご指摘より一段悪い状態が見つかりました — ローカルの ContentNetwork レコード側も安全ではありませんでした。ContentCreated などの membership イベントが発行元を一切検証せずに適用されていたので、任意のピアが member_nodes にこちらを名指しするだけでレコードを植え付けられました。

現在は gossipsub の Message::source(署名検証済みの発行元)に束縛し、membership を変える全イベントで発行元を認可しています。また 401/403 による早期打ち切りは、出自によらず全廃しました — レコード自体が最初の 1 通で植え付けられる以上、そこに載った peer の 403 も最終判断にはできないためです。

ただし根本は未解決です。 発行元は認証できても、member 集合そのものは発行元の自己申告のままです。ご提案の「正当なメンバー証明」= owner 署名付き membership が必要で、issue #63 で追跡しています。

まとめ

指摘	状態
応答を無検証で信頼	#56 で payload 真正性まで。版・writer・latest は #59
credential 転送	転送を継続(PoP なので安全と判断)
未検証ピアを member 扱い	発行元認証と verdict 非採用まで。member 証明は #63

design.md に保証範囲を明記し、未達の部分は「解決済みに見える書き方」を避けました。2 と 3 についてご異論があれば伺いたいです。特に 2 は判断が分かれうる箇所だと思っています。