diff --git a/docs/agents/reticulum.md b/docs/agents/reticulum.md index e6e63e787..cd7ae28bb 100644 --- a/docs/agents/reticulum.md +++ b/docs/agents/reticulum.md @@ -8,7 +8,7 @@ Deep subsystem reference for AI assistants. Open this when a task touches the Re - **Deep links / QR:** OS scheme is **`lxm://`** (not `mesh-client://`); `MeshClientDeepLinkHost`, `meshClientDeepLink.ts` (`lxmPaperMessage` kind + `looksLikeLxmPaperBlob`; Games `lxm://game/` / Ratspeak `lrgp:` → `lxmGameSession`), `handleReticulumQrIngest.ts` (shared Network/Chat/OS paper + in-app contact ingest), `applyLxmPaperIngest` → `POST /api/v1/lxmf/paper/ingest`, `QrIngestControl` / `QrCodeImage`. OS contact / MeshCore imports confirm before upsert; **paper OS deep links ingest without confirm**; Games session links open Reticulum Games tab via `openReticulumGameSession`. - **Decommissioned hubs:** `src/shared/reticulumDecommissionedHubs.ts` (Amsterdam only) — stack-start auto-disable + **Add default backbones** disables matching enabled TCP rows; UI badge + enable-block in `ReticulumInterfacesPanel.tsx` (`isDecommissionedReticulumTcpInterfaceRow`); keep TS↔Rust synced via `pnpm run check:reticulum-decommissioned-hubs`. Default backbone picker + region-grouped interface list (Primary & Global / North America / Europe / Asia & Oceania / Specialty / User Defined) in `reticulumDefaultHubPresets.ts` + `ReticulumDefaultHubsPickerModal.tsx`; muted disabled rows + checkbox bulk delete; `countEnabledDefaultHubPresets` / >3 enable warning - **BLE RNode RSSI:** `useReticulumBleRnodeRssiMap` gates on sidecar **running** (not api-ready), burst-then-steady scans via nested `acquireReticulumBleScan`, clears sticky targets immediately when all BLE RNodes are disabled -- **Propagation mode / sync:** Network → Propagation nodes owns Off/Auto/Manual (default **Off**; persisted values including legacy App-panel `auto` are honored). Auto one-time syncs the best Discovered PN by destination hash (no Add, no Preferred write) via `startPropagationSyncCascade` + sidecar `destination_hash` sync, then configured remotes, then local-prop (skips remotes when no enabled interfaces); runtime hook `useReticulumPropagationAutoSync`. Manual uses Preferred, else picks the best configured remote **for that sync only** (no Preferred write), then the remaining remotes, then local-prop. Off = **no PN support**: `startPropagationSyncCascade` returns early (per-row Sync is disabled in UI), `hasEffectiveReticulumPropagationTarget` / `hasReticulumPnCascadeCapacity` are false, `ReticulumPropagationNotice` is hidden, and the sidecar disarms the outbound PN plus empties cascade candidates (`propagation_mode` in `mesh_client_stack.json`, `POST /api/v1/propagation/mode`, `candidates_for_propagation_mode`); renderer pushes the mode on change and on sidecar-ready. `reticulumPropagationStore` / `reticulumPropagationSync.ts` — Complete on HaveAll, Establishing stall (~45s) + hard ceiling (~180s), auto-sync interval from last success with failure cooldown, error keys for identity / non-PN / peering stamp; stamps `lastPropagationSyncAttemptAt` / `activePropagationSyncAttemptAt` for WS correlation. **Nothing-to-sync is not a failure:** when the cascade contacts no node it writes `syncNoTarget` / `syncLocalLoading` (never overwriting a real error from an attempted node), the local row reports sidecar `status: "loading"` while the messagestore reads (`local_propagation_status` + `PropagationBridge::messagestore_load_pending`, per-row Sync disabled), and the 30 s tick calls `refreshFromSidecar` while `hasPropagationCascadeCandidate` is false so a fresh stack recovers on its own — `refreshFromSidecar` must **not** clear the active attempt while `sync.active`. Debug snapshot `propagationClient` exposes mode/preferred/autoTarget/resolvedSyncTargetId. **Auto also deposits on Discovered PNs:** sidecar `auto_discovered_candidates` (`pn_cascade.rs`, Auto only, cap 3, hop-sorted, skips inactive / self / already-configured / over `max_peering_cost`) appends after configured remotes and before local-prop, rebuilt from the shared `rebuild_pn_cascade_candidates` helper in `live.rs` (called by `refresh_pn_cascade_candidates` **and** the PN announce handler); `hasEffectiveReticulumPropagationTarget` / `hasReticulumPnCascadeCapacity` therefore count discovered rows in Auto, so the Chat notice hides and the link-timeout failure bridge holds off. **Chat notice dismiss:** `chatNoticeDismissed` (`mesh-client:reticulumPropagationNoticeDismissed`) with **Don't show again** on the banner and **Show propagation reminder in Chat** in the Network section. **Named sync target:** `startSync` stamps `syncTargetId`; progress line, inline error, and Sync toasts resolve it with `resolveReticulumPropagationTargetLabel`; the cascade clears it when nothing was contacted so `syncNoTarget` / `syncLocalLoading` stay unprefixed. **Attempts settle before the cascade advances:** `startSync` returns `accepted` | `deferred` | `failed` (not a boolean) — only sidecar _acceptance_ starts `awaitPropagationSyncSettled` (terminal WS frame or stall/ceiling watchdog). `failed` advances with ~15 min session-memory omit via `reticulumPropagationSyncBackoff.ts`; `deferred` (`PROPAGATION_SYNC_OUTBOUND_BUSY` — outbound deposit owns the PN link) advances **without** backoff so the next tick may retry; `cancelled` (user Cancel) stops; `success` ends the run. Remote steps are capped by `PROPAGATION_CASCADE_BUDGET_MS` (5 min) then fall through to local-prop; each remote attempt is capped by `PROPAGATION_CASCADE_ATTEMPT_TIMEOUT_MS` (~60s); local fallback refreshes nodes when local looks disabled; the cascade is single-flight (`resetPropagationSyncCascadeState` is the test seam) so overlapping 30 s ticks join one run while an explicit per-row Sync supersedes it. Auto `/api/v1/interfaces` probe **fails open** (assumes interfaces enabled) so a broken proxy still tries remotes before local. +- **Propagation mode / sync:** Network → Propagation nodes owns Off/Auto/Manual (default **Off**; persisted values including legacy App-panel `auto` are honored). Auto one-time syncs the best Discovered PN by destination hash (no Add, no Preferred write) via `startPropagationSyncCascade` + sidecar `destination_hash` sync, then configured remotes, then local-prop (skips remotes when no enabled interfaces); runtime hook `useReticulumPropagationAutoSync`. Manual uses Preferred, else picks the best configured remote **for that sync only** (no Preferred write), then the remaining remotes, then local-prop. Off = **no PN support**: `startPropagationSyncCascade` returns early (per-row Sync is disabled in UI), `hasEffectiveReticulumPropagationTarget` / `hasReticulumPnCascadeCapacity` are false, `ReticulumPropagationNotice` is hidden, and the sidecar disarms the outbound PN plus empties cascade candidates (`propagation_mode` in `mesh_client_stack.json`, `POST /api/v1/propagation/mode`, `candidates_for_propagation_mode`); renderer pushes the mode on change and on sidecar-ready. `reticulumPropagationStore` / `reticulumPropagationSync.ts` — Complete on HaveAll, Establishing stall (~45s) + hard ceiling (~180s), auto-sync interval from last success with failure cooldown, error keys for identity / non-PN / peering stamp; stamps `lastPropagationSyncAttemptAt` / `activePropagationSyncAttemptAt` for WS correlation. **Nothing-to-sync is not a failure:** when the cascade contacts no node it writes `syncNoTarget` / `syncLocalLoading` (never overwriting a real error from an attempted node), the local row reports sidecar `status: "loading"` while the messagestore reads (`local_propagation_status` + `PropagationBridge::messagestore_load_pending`, per-row Sync disabled), and the 30 s tick calls `refreshFromSidecar` while `hasPropagationCascadeCandidate` is false so a fresh stack recovers on its own — `refreshFromSidecar` must **not** clear the active attempt while `sync.active`. Debug snapshot `propagationClient` exposes mode/preferred/autoTarget/resolvedSyncTargetId. **Auto also deposits on Discovered PNs:** sidecar `auto_discovered_candidates` (`pn_cascade.rs`, Auto only, cap 3, hop-sorted, skips inactive / self / already-configured / over `max_peering_cost`) appends after configured remotes and before local-prop, rebuilt from the shared `rebuild_pn_cascade_candidates` helper in `live.rs` (called by `refresh_pn_cascade_candidates` **and** the PN announce handler); `hasEffectiveReticulumPropagationTarget` / `hasReticulumPnCascadeCapacity` therefore count discovered rows in Auto, so the Chat notice hides and the link-timeout failure bridge holds off. **Chat notice dismiss:** `chatNoticeDismissed` (`mesh-client:reticulumPropagationNoticeDismissed`) with **Don't show again** on the banner and **Show propagation reminder in Chat** in the Network section. **Named sync target:** `startSync` stamps `syncTargetId`; progress line, inline error, and Sync toasts resolve it with `resolveReticulumPropagationTargetLabel`; the cascade clears it when nothing was contacted so `syncNoTarget` / `syncLocalLoading` stay unprefixed. **Attempts settle before the cascade advances:** `startSync` returns `accepted` | `deferred` | `failed` (not a boolean) — only sidecar _acceptance_ starts `awaitPropagationSyncSettled` (terminal WS frame or stall/ceiling watchdog). `failed` advances with ~15 min session-memory omit via `reticulumPropagationSyncBackoff.ts`; `deferred` (`PROPAGATION_SYNC_OUTBOUND_BUSY` — outbound deposit owns the PN link) advances **without** backoff so the next tick may retry; `cancelled` (user Cancel) stops; `success` ends the run. Remote steps are capped by `PROPAGATION_CASCADE_BUDGET_MS` (5 min) then fall through to local-prop; each remote attempt is capped by `PROPAGATION_CASCADE_ATTEMPT_TIMEOUT_MS` (~60s); local fallback refreshes nodes when local looks disabled; the cascade is single-flight (`resetPropagationSyncCascadeState` is the test seam) so overlapping 30 s ticks join one run while an explicit per-row Sync supersedes it. Auto `/api/v1/interfaces` probe **fails open** (assumes interfaces enabled) so a broken proxy still tries remotes before local. **Retrieval vs peer sync:** the WS progress bar / HaveAll reflects the `/offer` peer sync (`PropagationSyncTask`, inventory replication only). The half that actually pulls your mail into Chat is the **client `/get` download** (`PropagationClient`), wired in `propagation_download.rs` + `propagation_bridge.rs::poll_client_download` and driven by `live.rs::spawn_client_download_driver` (list → get → purge; decrypt with local identity → router delivery callback). `local-prop` Sync uses `PropagationBridge::drain_local_inbox` (in-process node `/get` replay). Logs: real inbox retrieval is `propagation-retrieve` (`retrieve_mode=get|local`, `listed`/`downloaded`/`delivered`); the peer-offer outcome is `propagation-sync … peer_outcome=have_all|transfer` (**not** retrieval). - **PN hosting:** Network **Advanced PN hosting** / `ReticulumPnHostingDangerZone`; shared `pnHostingPolicy.ts` + sidecar `pn_hosting_policy.rs` / `pn_hosting_apply.rs`; `POST /api/v1/propagation/hosting-policy`; rsLXMF policy-setters overlay ([ratspeak/rsLXMF#6](https://github.com/ratspeak/rsLXMF/pull/6)). Messagestore loads in background on live attach; enabled `local-prop` serve/announce waits until load completes. - **Interface modes:** rnsd `mode` via `reticulumInterfaceMode.ts` + sidecar `normalize_interface_mode` (keep catalogs in sync — `pnpm run check:reticulum-interface-modes` in pre-commit/`release.sh`); add defaults TCP/UDP/I2P → `boundary`, RNode → `access_point`; UI in `ReticulumInterfacesPanel`; default hub presets add/repair missing mode to `boundary` (do not overwrite valid non-boundary). See [../reticulum.md#interface-modes](../reticulum.md#interface-modes). - **Share instance defaults:** missing keys bootstrap to `share_instance = No` / `instance_name = mesh-client` (does not overwrite explicit Yes/`default`); SharedInstanceClient banner + `disable_share_instance` repair; offline lint via `reticulum:validateConfig` / Network **Check config** / `pnpm run reticulum:config:check` diff --git a/docs/reticulum.md b/docs/reticulum.md index cc7d6812a..d907ab483 100644 --- a/docs/reticulum.md +++ b/docs/reticulum.md @@ -275,7 +275,7 @@ When multiple enabled local RNode interfaces are connected, the interface list s - **Config validate:** Electron IPC `reticulum:validateConfig` → one-shot sidecar `validate-config --json` against `userData/reticulum/config` - **Announces:** interval (`announce_interval_sec`, 0–86400; default **3600** s / 1 h when unset; `0` = startup-only) persisted in rnsd config. The live sidecar sends an **LXMF delivery** announce shortly after stack start and on that interval (Ratspeak/lxmd parity). **Announce now** (`POST /api/v1/announces`) forces an immediate delivery announce. **Clear announces** (`DELETE /api/v1/announces`) clears the stub peer cache; the live path table may refill on the next peer refresh. Per-interface `announce_interval_min` (RMAP/discoverable interfaces) is separate. - **Inbound LXMF:** the sidecar registers `lxmf.delivery` with the transport (`RegisterDestination` + `LinkManager`) and feeds decrypted link/resource payloads into the delivery callback (WS `lxmf_message`). Without this registration, peer DMs never appear in Chat even when paths exist. -- **Propagation:** **Propagation mode** (Network → Propagation nodes; **Off** default / **Auto** / **Manual**) — **Off** means **no propagation support**: no sync (periodic, bottom **Sync**, or per-node **Sync messages**) and no outbound Direct→PN cascade, so nothing is deposited on a remote PN or the local inbox; a saved Preferred row stays on disk and is re-armed only when you pick Auto/Manual (renderer pushes the mode to the sidecar via `POST /api/v1/propagation/mode`, persisted as `propagation_mode` in `mesh_client_stack.json`). **Auto** one-time syncs the best **Discovered** PN by destination hash (does **not** add it to the configured list or change Preferred), then tries configured remotes, then local-prop (skips remotes when no enabled interfaces). Auto also **deposits** outbound LXMF on Discovered PNs: `auto_discovered_candidates` (`pn_cascade.rs`) appends up to `MAX_AUTO_DISCOVERED_PN_CANDIDATES` (3) heard nodes — hop-sorted, skipping inactive announces, the self hash, already-configured hashes, and `peering_cost` above the hosting policy `max_peering_cost` — **after** the added remotes and **before** local-prop. Nothing is persisted; the announce handler and `refresh_pn_cascade_candidates` share one rebuild helper in `live.rs`, so a newly heard PN becomes cascade-eligible without a stack restart. Because Auto really uses them, the Chat **“No propagation node is configured”** banner hides in Auto as soon as one PN is discovered; Manual only counts nodes you added. The banner also has **Don't show again**, backed by **Show propagation reminder in Chat** in Network → Propagation nodes (`chatNoticeDismissed`, persisted in `mesh-client:reticulumPropagationNoticeDismissed`). **Manual** syncs Preferred; with no Preferred it picks the closest added remote **for that sync only** (no Preferred write), then falls back to the other added remotes, then local-prop. Every cascade step **waits for that attempt to settle** before deciding what to do next: `startSync` only reports that the sidecar _accepted_ the request, so the renderer awaits the terminal `propagation_sync` frame (or the stall / ceiling watchdog) through `awaitPropagationSyncSettled` and moves to the next candidate on failure — a node that accepts and then never establishes no longer ends the cascade at its first step. A user **Cancel** stops the chain instead of advancing. The remote half of the chain is capped by `PROPAGATION_CASCADE_BUDGET_MS` (5 min), after which it goes straight to the local inbox, and a target that just failed is omitted for 15 minutes (`reticulumPropagationSyncBackoff.ts`, session memory) so a hop-closest dead PN cannot monopolize every tick; each remote attempt is capped at ~60s (`PROPAGATION_CASCADE_ATTEMPT_TIMEOUT_MS`) before the cascade advances, and local fallback re-reads sidecar nodes when local looks disabled. Because a cascade can now span several attempts, overlapping 30 s auto-sync ticks join the single in-flight run; an explicit per-row **Sync** supersedes it. **Local-only PN** is a supported setup for an always-on machine: enable local-prop with no remotes and every cascade/sync settles in the local inbox. Preferred node for offline DMs, per-node **Sync messages**, add remote propagation nodes by 32-character `lxmf.propagation` hash or from the **Discovered on network** list (heard PN announces; explicit **Add / Add & prefer** for list management), **rename** / **delete** remote nodes, optional **local PN hosting** (announce + `/offer`/`/get` + Link Resource deposit ingress with stamp validation into the local store, plus outbound peer inventory sync when hosting + autopeer/static peers are on), Network **Advanced PN hosting** policy (`peering_cost`, `max_peering_cost`, autopeer, stamps, storage), Add-time `/offer` probe, **auto-sync interval** (`auto_sync_interval_sec`; `0` disables periodic sync; interval measured from last _successful_ sync with a short failure cooldown). Local-prop messagestore load is **deferred** off the live-ready path; serve/announce waits until that load finishes so peers are not syncing an empty store — while that load runs, `list_propagation` reports the local row as `status: "loading"` (not just disabled), the row renders **loading…**, and its per-node **Sync messages** is disabled. When a sync cascade finds nothing to contact (no Discovered PN, no added remotes, local inbox off or still loading) it reports **why** — `reticulumPropagation.syncLocalLoading` or `reticulumPropagation.syncNoTarget` — instead of the generic "node may be unreachable"; a real per-node error from an attempted node is never overwritten. Every sync attempt stamps `syncTargetId`, so the progress line, the inline error, and the Sync toasts **name the node** being tried (`resolveReticulumPropagationTargetLabel` — configured row name, announce name, or hash prefix); the cascade clears it when it contacted nobody, so "nothing to sync with" is never blamed on a node. The 30 s auto-sync tick re-reads `/api/v1/propagation` while no cascade candidate exists, so a fresh stack starts syncing on its own once an announce lands or the local store finishes loading. Remote sync **always sends an LXMF delivery announce** then settles briefly (~2s) before Establishing so the PN has a reverse path for LRPROOF, **re-requests the forward path** (does not reuse a possibly stale hop count), pins/persists PN identity during Establishing (avoids announce-flood eviction), resolves identity+path before Establishing, rejects non-PN destinations (`PROPAGATION_TARGET_NOT_PN`), requires a peering stamp when cost > 0, treats HaveAll/Complete as success (not failure), surfaces `NoLinkProof` when establish stalls without a proof, and the renderer cancels Establishing-only stalls (~45s) plus a hard ceiling (~180s) via `reticulumPropagationSync.ts` without overwriting sidecar failure keys. After Sync Completes, the renderer runs inbound LXMF catch-up so Chat does not wait for the periodic ring poll. Correlatable deposit/retrieve logs use targets `propagation-deposit` / `propagation-retrieve` (`message_hash`, `transient_id`, `pn_hash`, and on Completes `cascade_step` + `delivery_method` so the actual deposit island is auditable). Developer support bundles always include `reticulum/mesh_client_stack.json` and `reticulum/lxmf-outbound.log` (placeholder when absent), and `debug-snapshot.json` carries a `propagationClient` slice (`mode`, `preferredId`, `resolvedSyncTargetId`, `autoTarget`, `lastSyncError`). +- **Propagation:** **Propagation mode** (Network → Propagation nodes; **Off** default / **Auto** / **Manual**) — **Off** means **no propagation support**: no sync (periodic, bottom **Sync**, or per-node **Sync messages**) and no outbound Direct→PN cascade, so nothing is deposited on a remote PN or the local inbox; a saved Preferred row stays on disk and is re-armed only when you pick Auto/Manual (renderer pushes the mode to the sidecar via `POST /api/v1/propagation/mode`, persisted as `propagation_mode` in `mesh_client_stack.json`). **Auto** one-time syncs the best **Discovered** PN by destination hash (does **not** add it to the configured list or change Preferred), then tries configured remotes, then local-prop (skips remotes when no enabled interfaces). Auto also **deposits** outbound LXMF on Discovered PNs: `auto_discovered_candidates` (`pn_cascade.rs`) appends up to `MAX_AUTO_DISCOVERED_PN_CANDIDATES` (3) heard nodes — hop-sorted, skipping inactive announces, the self hash, already-configured hashes, and `peering_cost` above the hosting policy `max_peering_cost` — **after** the added remotes and **before** local-prop. Nothing is persisted; the announce handler and `refresh_pn_cascade_candidates` share one rebuild helper in `live.rs`, so a newly heard PN becomes cascade-eligible without a stack restart. Because Auto really uses them, the Chat **“No propagation node is configured”** banner hides in Auto as soon as one PN is discovered; Manual only counts nodes you added. The banner also has **Don't show again**, backed by **Show propagation reminder in Chat** in Network → Propagation nodes (`chatNoticeDismissed`, persisted in `mesh-client:reticulumPropagationNoticeDismissed`). **Manual** syncs Preferred; with no Preferred it picks the closest added remote **for that sync only** (no Preferred write), then falls back to the other added remotes, then local-prop. Every cascade step **waits for that attempt to settle** before deciding what to do next: `startSync` only reports that the sidecar _accepted_ the request, so the renderer awaits the terminal `propagation_sync` frame (or the stall / ceiling watchdog) through `awaitPropagationSyncSettled` and moves to the next candidate on failure — a node that accepts and then never establishes no longer ends the cascade at its first step. A user **Cancel** stops the chain instead of advancing. The remote half of the chain is capped by `PROPAGATION_CASCADE_BUDGET_MS` (5 min), after which it goes straight to the local inbox, and a target that just failed is omitted for 15 minutes (`reticulumPropagationSyncBackoff.ts`, session memory) so a hop-closest dead PN cannot monopolize every tick; each remote attempt is capped at ~60s (`PROPAGATION_CASCADE_ATTEMPT_TIMEOUT_MS`) before the cascade advances, and local fallback re-reads sidecar nodes when local looks disabled. Because a cascade can now span several attempts, overlapping 30 s auto-sync ticks join the single in-flight run; an explicit per-row **Sync** supersedes it. **Local-only PN** is a supported setup for an always-on machine: enable local-prop with no remotes and every cascade/sync settles in the local inbox. Preferred node for offline DMs, per-node **Sync messages**, add remote propagation nodes by 32-character `lxmf.propagation` hash or from the **Discovered on network** list (heard PN announces; explicit **Add / Add & prefer** for list management), **rename** / **delete** remote nodes, optional **local PN hosting** (announce + `/offer`/`/get` + Link Resource deposit ingress with stamp validation into the local store, plus outbound peer inventory sync when hosting + autopeer/static peers are on), Network **Advanced PN hosting** policy (`peering_cost`, `max_peering_cost`, autopeer, stamps, storage), Add-time `/offer` probe, **auto-sync interval** (`auto_sync_interval_sec`; `0` disables periodic sync; interval measured from last _successful_ sync with a short failure cooldown). Local-prop messagestore load is **deferred** off the live-ready path; serve/announce waits until that load finishes so peers are not syncing an empty store — while that load runs, `list_propagation` reports the local row as `status: "loading"` (not just disabled), the row renders **loading…**, and its per-node **Sync messages** is disabled. When a sync cascade finds nothing to contact (no Discovered PN, no added remotes, local inbox off or still loading) it reports **why** — `reticulumPropagation.syncLocalLoading` or `reticulumPropagation.syncNoTarget` — instead of the generic "node may be unreachable"; a real per-node error from an attempted node is never overwritten. Every sync attempt stamps `syncTargetId`, so the progress line, the inline error, and the Sync toasts **name the node** being tried (`resolveReticulumPropagationTargetLabel` — configured row name, announce name, or hash prefix); the cascade clears it when it contacted nobody, so "nothing to sync with" is never blamed on a node. The 30 s auto-sync tick re-reads `/api/v1/propagation` while no cascade candidate exists, so a fresh stack starts syncing on its own once an announce lands or the local store finishes loading. Remote sync runs **two independent operations** against the PN: the **`/offer` peer sync** (`PropagationSyncTask` — inventory replication that drives the progress bar and the `have_all`/`transfer` outcome) and the **client `/get` download** (`PropagationClient` in `propagation_download.rs` / `propagation_bridge.rs`, driven from `live.rs::spawn_client_download_driver`) which is the half that actually **retrieves your own mail into Chat** (list → get → purge, Python `request_messages_from_propagation_node` parity). Retrieved blobs are decrypted with the local identity, unpacked, and delivered through the normal router callback (so WS `lxmf_message`, the recent ring, and catch-up fire unchanged). `local-prop` Sync has no remote link, so it replays the in-process node's own list → serve → purge (`PropagationBridge::drain_local_inbox`) to drain your inbox. Remote sync **always sends an LXMF delivery announce** then settles briefly (~2s) before Establishing so the PN has a reverse path for LRPROOF, **re-requests the forward path** (does not reuse a possibly stale hop count), pins/persists PN identity during Establishing (avoids announce-flood eviction), resolves identity+path before Establishing, rejects non-PN destinations (`PROPAGATION_TARGET_NOT_PN`), requires a peering stamp when cost > 0, treats HaveAll/Complete as success (not failure), surfaces `NoLinkProof` when establish stalls without a proof, and the renderer cancels Establishing-only stalls (~45s) plus a hard ceiling (~180s) via `reticulumPropagationSync.ts` without overwriting sidecar failure keys. After Sync Completes, the renderer runs inbound LXMF catch-up so Chat does not wait for the periodic ring poll. Correlatable logs: `propagation-deposit` (`message_hash`, `transient_id`, `pn_hash`, and on Completes `cascade_step` + `delivery_method`) for outbound; `propagation-retrieve` for the client `/get` download (`retrieve_mode=get`, `listed` / `downloaded` / `delivered` counts, `retrieve_mode=local` for local-prop) and per delivered message. The peer-offer side now logs under `propagation-sync` with `peer_outcome=have_all|transfer` — it is **not** inbox retrieval. Developer support bundles always include `reticulum/mesh_client_stack.json` and `reticulum/lxmf-outbound.log` (placeholder when absent), and `debug-snapshot.json` carries a `propagationClient` slice (`mode`, `preferredId`, `resolvedSyncTargetId`, `autoTarget`, `lastSyncError`). --- diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index c3ce86a8f..24740970b 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -1337,10 +1337,12 @@ Bond-stale **TX queue full** hints (`txQueueDropsHintBleBondStale`) point at the **Cause (any-node model)**: LXMF does **not** require both parties to prefer the same PN. Deposit on PN A and retrieve via Sync from PN B is valid when autopeer/static peering moves inventory. Empty Chat after Sync is usually a fabric/retrieve/ingest gap (mail never reached the synced node, stamp/admission drop on a host PN, or inbound ring not catch-up’d into Chat) — not “wrong preferred PN.” +**Progress bar ≠ retrieval.** Sync runs two independent operations against the PN: the **`/offer` peer sync** (inventory replication — this is what drives the progress bar and the `have_all`/`transfer` outcome) and the **client `/get` download** (the part that actually pulls _your_ mail into Chat). A Sync reaching **Complete / HaveAll** only tells you the peer-offer finished; it does **not** mean anything was retrieved. Look for the `/get` retrieve counts, not the peer-offer outcome. + **Do not** tell users they must share the same preferred PN. Prefer log correlation instead: 1. Sender Device log: `propagation-deposit` with `message_hash`, `transient_id`, `pn_hash` (deposit Completes). -2. Recipient (or Host PN) log: `propagation-retrieve` with matching `message_hash` / `transient_id` after Sync, plus `retrieve_mode=have_all|transfer`. +2. Recipient log (the real retrieval): `propagation-retrieve … retrieve_mode=get pn_hash=… listed=N downloaded=N delivered=N` (the client `/get` download; `listed=0` is a valid empty-inbox success). Per-message `propagation-retrieve` with matching `message_hash` / `transient_id` fires as each downloaded message hits the delivery callback. `local-prop` Sync logs `retrieve_mode=local`. The peer-offer side logs `propagation-sync … peer_outcome=have_all|transfer` — that is **not** retrieval. 3. Renderer: `[catchUpRecentInboundLxmf] … reason=propagation_sync` or `propagation-retrieve catch-up after sync Completes count=N` (`count=0 (empty ring)` means Sync Completes with no new inbound for Chat). 4. Confirm remote Sync Completes and that Host PN (if used) shows `[propagation-deposit] local PN accepted stamped propagated blob`. diff --git a/reticulum-sidecar/src/stack/live.rs b/reticulum-sidecar/src/stack/live.rs index a199300a8..fae5b7a89 100644 --- a/reticulum-sidecar/src/stack/live.rs +++ b/reticulum-sidecar/src/stack/live.rs @@ -68,6 +68,7 @@ use super::pn_hosting_apply::{apply_pn_hosting_policy_to_node, apply_pn_hosting_ use super::pn_hosting_policy::PnHostingPolicy; use super::propagation_announce::PropagationAnnounceLoop; use super::propagation_bridge::PropagationBridge; +use super::propagation_download::ClientDownloadPoll; use super::propagation_serve::PropagationServeHandle; use super::rncp_transfer::RncpTransferManager; use super::rnsh_session::RnshSessionManager; @@ -3665,6 +3666,11 @@ impl LiveBridge { } return Err("propagation sync unavailable".into()); } + // Retrieval half of Sync (Python `request_messages_from_propagation_node`): + // pull our own store-and-forward mail from this PN via the client `/get` + // download and deliver it into Chat. Runs alongside the `/offer` peer sync + // above, which only replicates inventory and never fills our inbox. + self.spawn_client_download_driver(hash, dest_hex.clone(), Arc::clone(&cancel), run_id); let outbound = Arc::clone(&self.outbound); let on_terminal: Arc = Arc::new(move || { if let Ok(mut driver) = outbound.lock() { @@ -3682,6 +3688,150 @@ impl LiveBridge { Ok(()) } + /// Drive the client `/get` download to completion off the sync request path. + /// + /// Ticks [`PropagationBridge::poll_client_download`] on a short interval, + /// feeding it the current known-identity map (for link-proof validation) and, + /// on a terminal Complete, delivering each decoded message through the router + /// delivery callback — the same path Direct/opportunistic inbound uses, so WS + /// `lxmf_message`, the recent ring, and renderer catch-up all fire unchanged. + fn spawn_client_download_driver( + &self, + pn_hash: [u8; 16], + pn_hex: String, + cancel: Arc, + run_id: u64, + ) { + if !self.propagation.start_client_download(pn_hash) { + tracing::debug!( + target: "propagation-retrieve", + pn_hash = %pn_hex, + "client /get download not started (already active or unavailable)" + ); + return; + } + let bridge = Arc::clone(&self.propagation); + let router = Arc::clone(&self.router); + let outbound = Arc::clone(&self.outbound); + let active_run_id = Arc::clone(&self.sync_run_id); + tokio::spawn(async move { + // Client tick cadence; the client's own 120s timeout bounds a stuck link. + const POLL_INTERVAL: Duration = Duration::from_millis(500); + const DOWNLOAD_WATCHDOG: Duration = Duration::from_secs(180); + let mut interval = tokio::time::interval(POLL_INTERVAL); + let started = Instant::now(); + loop { + interval.tick().await; + // Superseded by a newer sync run: that run now owns the client, + // so exit without touching shared client state. + if !PropagationBridge::is_current_sync_run( + active_run_id.load(Ordering::SeqCst), + run_id, + ) { + break; + } + // This run was cancelled: cancel the client only while we are + // still the active run (lifecycle-lock guarded via run_if_current). + if cancel.load(Ordering::SeqCst) { + bridge.run_if_current(&active_run_id, run_id, || { + bridge.cancel_client_download(); + }); + break; + } + if started.elapsed() > DOWNLOAD_WATCHDOG { + tracing::info!( + target: "propagation-retrieve", + pn_hash = %pn_hex, + "client /get download watchdog timeout" + ); + bridge.run_if_current(&active_run_id, run_id, || { + bridge.cancel_client_download(); + }); + break; + } + let known = outbound + .lock() + .ok() + .map(|d| d.known_identities_for_propagation()) + .unwrap_or_default(); + match bridge.poll_client_download(&known) { + ClientDownloadPoll::Idle => break, + // Keep polling on the next interval tick. + ClientDownloadPoll::InProgress => {} + ClientDownloadPoll::Failed => { + tracing::info!( + target: "propagation-retrieve", + pn_hash = %pn_hex, + "client /get download failed" + ); + // Consume the terminal failed state → Idle so a later + // sync can start a fresh retrieval. + bridge.cancel_client_download(); + break; + } + ClientDownloadPoll::Complete { + messages, + listed, + downloaded, + } => { + let delivered = messages.len(); + { + let router = router.lock().await; + if let Some(ref cb) = router.delivery_callback { + for msg in &messages { + cb(msg); + } + } + } + // Empty list is success (result=0), not failure. + tracing::info!( + target: "propagation-retrieve", + pn_hash = %pn_hex, + listed, + downloaded, + delivered, + retrieve_mode = "get", + "client /get download Completes" + ); + break; + } + } + } + }); + } + + /// Drain the in-process (`local-prop`) PN store of our own mail into Chat. + /// + /// `local-prop` Sync has no remote link to `/get` against, so we replay the + /// local node's own list → serve → purge and deliver each decoded message + /// through the router callback (same ingest as remote retrieval). Returns the + /// number of messages delivered. + pub async fn drain_local_propagation_inbox(&self) -> usize { + // The drain does blocking file I/O + per-message decryption; run it off + // the async worker so it cannot stall the runtime. + let bridge = Arc::clone(&self.propagation); + let (messages, listed) = tokio::task::spawn_blocking(move || bridge.drain_local_inbox()) + .await + .unwrap_or_else(|_| (Vec::new(), 0)); + let delivered = messages.len(); + if delivered > 0 { + let router = self.router.lock().await; + if let Some(ref cb) = router.delivery_callback { + for msg in &messages { + cb(msg); + } + } + } + tracing::info!( + target: "propagation-retrieve", + listed, + delivered, + retrieve_mode = "local", + "local-prop inbox drain Completes" + ); + delivered + } + /// Resolve identity hashes + peering stamp for a remote LXMF PN `/offer`. /// /// PNs with peering_cost > 0 reject empty keys (`ErrorInvalidKey`). When cost is 0, @@ -3791,6 +3941,7 @@ impl LiveBridge { slot.store(true, Ordering::SeqCst); } self.propagation.cancel_sync(); + self.propagation.cancel_client_download(); if let Ok(mut driver) = self.outbound.lock() { driver.clear_propagation_identity_pins(); driver.set_propagation_sync_target(None); diff --git a/reticulum-sidecar/src/stack/mod.rs b/reticulum-sidecar/src/stack/mod.rs index a06daf3c1..bfa874ebf 100644 --- a/reticulum-sidecar/src/stack/mod.rs +++ b/reticulum-sidecar/src/stack/mod.rs @@ -52,6 +52,8 @@ mod propagation_announce; #[cfg(feature = "rns-stack")] mod propagation_bridge; #[cfg(feature = "rns-stack")] +mod propagation_download; +#[cfg(feature = "rns-stack")] mod propagation_serve; #[cfg(feature = "rns-stack")] mod rncp_transfer; @@ -1499,8 +1501,13 @@ impl StackHandle { let sync_self = is_local || prop_hash.eq_ignore_ascii_case(&lxmf) || (!local_prop_hash.is_empty() && prop_hash.eq_ignore_ascii_case(&local_prop_hash)); - // Local inbox lives in this process — settle without a self LinkRequest. + // Local inbox lives in this process — settle without a self LinkRequest, + // but still drain our own mail out of the local PN store into Chat. if is_local { + #[cfg(feature = "rns-stack")] + if let Some(live) = self.live.get() { + live.drain_local_propagation_inbox().await; + } self.emit_event( "propagation_sync", serde_json::json!({ diff --git a/reticulum-sidecar/src/stack/propagation_bridge.rs b/reticulum-sidecar/src/stack/propagation_bridge.rs index c39754981..69d33438b 100644 --- a/reticulum-sidecar/src/stack/propagation_bridge.rs +++ b/reticulum-sidecar/src/stack/propagation_bridge.rs @@ -6,14 +6,19 @@ use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; use std::sync::{Arc, Mutex, MutexGuard, PoisonError}; use std::time::{Duration, Instant}; +use lxmf_core::message::LxMessage; use lxmf_core::peer::OutboundOfferPolicy; +use lxmf_core::propagation_client::{PropagationClient, PropagationClientState}; use lxmf_core::propagation_node::{PropagationNode, PropagationNodeConfig}; use lxmf_core::propagation_sync::{PeerSyncTerminalState, PropagationSyncTask, SyncTaskState}; use lxmf_core::router::LxmRouter; +use rns_identity::destination::Destination; use rns_identity::identity::Identity; use rns_transport::messages::TransportMessage; use tokio::sync::{Notify, broadcast, mpsc}; +use super::propagation_download::{ClientDownloadPoll, decode_downloaded_propagated_blob}; + /// Completed host-peer peering PoW (stamp, value) awaiting apply onto `LxmPeer`. type PeeringKeyResult = ([u8; 16], [u8; 32], u32); @@ -24,6 +29,12 @@ pub struct PropagationBridge { local_dest_hash: [u8; 16], local_node: Arc>, sync_task: Mutex, + /// Client `/get` pull for retrieving our own store-and-forward mail from a + /// remote PN into Chat (Python `request_messages_from_propagation_node`). + /// Distinct from `sync_task`, which is the `/offer` peer-replication path. + client: Mutex, + /// Local identity clone used to decrypt downloaded propagated blobs. + identity: Identity, local_serving: AtomicBool, /// Terminal result of background `load_messagestore_from_disk` (`None` while in flight). messagestore_result: Mutex>>, @@ -66,15 +77,25 @@ impl PropagationBridge { PropagationNode::with_storage_unloaded(node_config, local_dest_hash, storage_dir) .map_err(|e| format!("propagation storage init: {e}"))?, )); - let mut sync_task = PropagationSyncTask::with_shared_node(transport_tx, local_node.clone()); + let mut sync_task = + PropagationSyncTask::with_shared_node(transport_tx.clone(), local_node.clone()); let signing_key = identity .get_signing_key() .ok_or_else(|| "propagation sync: identity has no signing key".to_string())?; sync_task.set_identity(identity.get_public_key(), signing_key); + // Client `/get` pull uses the same identity to identify on the PN link + // and to decrypt downloaded blobs addressed to our `lxmf.delivery` hash. + let client = PropagationClient::new( + transport_tx, + Some(identity.get_public_key()), + identity.get_signing_key(), + ); Ok(Self { local_dest_hash, local_node, sync_task: Mutex::new(sync_task), + client: Mutex::new(client), + identity: identity.clone(), local_serving: AtomicBool::new(false), messagestore_result: Mutex::new(None), messagestore_notify: Notify::new(), @@ -377,6 +398,171 @@ impl PropagationBridge { last_finished_ok != Some(false) } + /// Start a client `/get` download of our own mail from `pn_hash`. + /// + /// This is the retrieval half of Sync (Python + /// `request_messages_from_propagation_node`): list → get → purge over an + /// `lxmf.propagation.client` link. Returns false when a download is already + /// in flight or the client refuses to start. + pub fn start_client_download(&self, pn_hash: [u8; 16]) -> bool { + let Ok(mut client) = self.client.lock() else { + return false; + }; + client.set_propagation_node(pn_hash); + client.start_download() + } + + /// Cancel any in-flight client download (best-effort). The next + /// [`Self::start_client_download`] re-arms from Idle. + pub fn cancel_client_download(&self) { + if let Ok(mut client) = self.client.lock() { + // Consuming the terminal snapshot returns the client to Idle so the + // next download can start; also drops any half-received blobs. + let _ = client.acknowledge_transfer(); + let _ = client.take_received_messages(); + } + } + + /// Drive the client download one step: drain inbound events, advance the + /// state machine, and on a terminal Complete decode the downloaded blobs + /// into inbound [`LxMessage`]s ready for the router delivery callback. + pub(crate) fn poll_client_download( + &self, + known_identities: &HashMap, + ) -> ClientDownloadPoll { + let Ok(mut client) = self.client.lock() else { + return ClientDownloadPoll::Failed; + }; + if matches!(client.state(), PropagationClientState::Idle) { + return ClientDownloadPoll::Idle; + } + client.drain_events(known_identities); + client.tick(); + match client.state() { + PropagationClientState::Idle => ClientDownloadPoll::Idle, + PropagationClientState::Failed => ClientDownloadPoll::Failed, + PropagationClientState::Complete => { + let listed = client.available_messages().len(); + let downloaded = client.received_count(); + let blobs = client.take_received_messages(); + // Consume the terminal snapshot → Idle so the next + // start_client_download can proceed without a cancel first. + let _ = client.acknowledge_transfer(); + drop(client); + let messages = blobs + .iter() + .filter_map(|blob| decode_downloaded_propagated_blob(&self.identity, blob)) + .collect::>(); + ClientDownloadPoll::Complete { + messages, + listed, + downloaded, + } + } + _ => ClientDownloadPoll::InProgress, + } + } + + /// Drain our own store-and-forward mail out of the **local** PN store into + /// inbound [`LxMessage`]s for Chat, without a network link. + /// + /// `local-prop` Sync hosts our inbox in-process, so there is no remote + /// `/get` to run. We replay the node's own `/get` list → serve → purge + /// against our `lxmf.delivery` hash (identical ownership gate + stamp strip + /// the server applies to remote clients), decode each blob with the local + /// identity, and hand them back for delivery. Returns `(messages, listed)`. + pub(crate) fn drain_local_inbox(&self) -> (Vec, usize) { + use rmpv::Value; + + let our_delivery = + Destination::hash_from_name_and_identity("lxmf.delivery", Some(&self.identity.hash)); + + // Hold the node lock only for the list + serve reads, then release it + // before per-message decryption/decode (CPU work off the shared lock). + let (blobs, listed) = { + let Ok(mut node) = self.local_node.lock() else { + return (Vec::new(), 0); + }; + + // Phase 1: list our available transient IDs. + let list_req = Self::encode_value(&Value::Array(vec![Value::Nil, Value::Nil])); + let tids = Self::decode_binary_array( + &node + .handle_get_request(&list_req, &our_delivery) + .into_response(), + ); + if tids.is_empty() { + return (Vec::new(), 0); + } + + // Phase 2: fetch every listed message (server strips the stamp). + let wants: Vec = tids.iter().map(|t| Value::Binary(t.clone())).collect(); + let get_req = Self::encode_value(&Value::Array(vec![ + Value::Array(wants), + Value::Array(Vec::new()), + ])); + let blobs = Self::decode_binary_array( + &node + .handle_get_request(&get_req, &our_delivery) + .into_response(), + ); + (blobs, tids.len()) + }; + + // Decode without holding the node lock. + let messages = blobs + .iter() + .filter_map(|blob| decode_downloaded_propagated_blob(&self.identity, blob)) + .collect::>(); + let skipped = blobs.len().saturating_sub(messages.len()); + if skipped > 0 { + tracing::warn!( + target: "propagation-retrieve", + skipped, + served = blobs.len(), + "local-prop drain skipped undecodable blob(s)" + ); + } + + // Phase 3: purge only what we successfully decoded, keyed by the exact + // transient id `compute_propagation_transient_id` stamped on each + // message. Undecodable blobs stay in the store for a later retry. + let purge_ids: Vec = messages + .iter() + .filter_map(|msg| msg.transient_id.map(|tid| Value::Binary(tid.to_vec()))) + .collect(); + if !purge_ids.is_empty() { + if let Ok(mut node) = self.local_node.lock() { + let purge_req = + Self::encode_value(&Value::Array(vec![Value::Nil, Value::Array(purge_ids)])); + let _ = node.handle_get_request(&purge_req, &our_delivery); + } + } + + (messages, listed) + } + + /// Encode an rmpv value to msgpack bytes (the `/get` request wire form). + fn encode_value(value: &rmpv::Value) -> Vec { + let mut buf = Vec::new(); + // Writing into a Vec is infallible. + let _ = rmpv::encode::write_value(&mut buf, value); + buf + } + + /// Decode a msgpack array of binaries (the `/get` list and serve responses). + fn decode_binary_array(bytes: &[u8]) -> Vec> { + let Ok(value) = rmpv::decode::read_value(&mut &bytes[..]) else { + return Vec::new(); + }; + let Some(arr) = value.as_array() else { + return Vec::new(); + }; + arr.iter() + .filter_map(|v| v.as_slice().map(<[u8]>::to_vec)) + .collect() + } + /// Whether this emitter still owns the active sync run (generation match). pub fn is_current_sync_run(active_run_id: u64, run_id: u64) -> bool { active_run_id == run_id @@ -461,14 +647,17 @@ impl PropagationBridge { } if ok { let peak = self.last_peak_progress(); - // Peak ≥ Transferring (70) means WantSome/WantAll pulled blobs; lower ≈ HaveAll. - let retrieve_mode = if peak >= 70.0 { "transfer" } else { "have_all" }; + // This is the `/offer` peer-sync (inventory replication) outcome, NOT + // inbox retrieval — HaveAll means the peer wanted nothing, transfer means + // we pushed blobs to it. Real inbox retrieval is logged by the client + // `/get` download path (`propagation-retrieve`) in live.rs. + let peer_outcome = if peak >= 70.0 { "transfer" } else { "have_all" }; tracing::info!( - target: "propagation-retrieve", + target: "propagation-sync", pn_hash = %hex::encode(peer_hash), peak_progress = peak, - retrieve_mode, - "remote/host PN sync Completes" + peer_outcome, + "remote/host PN peer sync Completes" ); } else { let peak = self.last_peak_progress(); @@ -588,13 +777,13 @@ impl PropagationBridge { if !active && (progress >= 99.0 || finished_ok.is_some()) { if finished_ok == Some(true) { let peak = bridge.last_peak_progress(); - let retrieve_mode = if peak >= 70.0 { "transfer" } else { "have_all" }; + let peer_outcome = if peak >= 70.0 { "transfer" } else { "have_all" }; tracing::info!( - target: "propagation-retrieve", + target: "propagation-sync", progress, peak_progress = peak, - retrieve_mode, - "propagation sync completed successfully" + peer_outcome, + "propagation peer sync completed successfully" ); } else if let Some(ref msg) = fail_message { tracing::info!( @@ -631,6 +820,96 @@ impl PropagationBridge { #[cfg(test)] mod tests { use super::*; + use lxmf_core::constants::DeliveryMethod; + + /// local-prop loopback (plan acceptance gate): deposit stamped mail addressed + /// to our own `lxmf.delivery` hash into the local PN store, then confirm + /// `drain_local_inbox` runs the node's `/get` list → serve (stamp strip) → + /// purge, decrypts with the local identity, and yields the message once. + #[test] + fn drain_local_inbox_delivers_then_purges_own_mail() { + let dir = + std::env::temp_dir().join(format!("mesh-prop-drain-loopback-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).expect("tmpdir"); + let (tx, _rx) = mpsc::channel(8); + let recipient = Identity::new(); + let bridge = PropagationBridge::new( + tx, + [0xab; 16], + dir.clone(), + &recipient, + &super::super::pn_hosting_policy::PnHostingPolicy::default(), + ) + .expect("bridge"); + + // Deposit a stamped blob addressed to the bridge's own lxmf.delivery hash. + let sender = Identity::new(); + let blob = super::super::propagation_download::build_client_download_blob( + &sender, + &recipient, + "loopback mail", + ); + { + let mut node = bridge.local_node.lock().expect("node lock"); + // stamp_value high enough to clear any policy min_stamp_cost. + assert!(node.accept_stamped_propagated_blob(&blob, &[0u8; 32], u8::MAX)); + } + + let (messages, listed) = bridge.drain_local_inbox(); + assert_eq!(listed, 1, "one message listed for our delivery hash"); + assert_eq!(messages.len(), 1, "one message decoded"); + assert_eq!(messages[0].content, "loopback mail"); + assert!(messages[0].incoming, "delivered mail is inbound"); + assert_eq!(messages[0].method, DeliveryMethod::Propagated); + + // Phase-3 purge must have removed the entry: a second drain is empty. + let (again, listed_again) = bridge.drain_local_inbox(); + assert!(again.is_empty(), "purged mail is not re-delivered"); + assert_eq!(listed_again, 0); + + let _ = std::fs::remove_dir_all(&dir); + } + + /// A blob addressed to someone else must never leak into our inbox drain + /// (server ownership gate + decrypt both reject it). + #[test] + fn drain_local_inbox_ignores_mail_for_other_recipients() { + let dir = + std::env::temp_dir().join(format!("mesh-prop-drain-foreign-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).expect("tmpdir"); + let (tx, _rx) = mpsc::channel(8); + let us = Identity::new(); + let bridge = PropagationBridge::new( + tx, + [0xab; 16], + dir.clone(), + &us, + &super::super::pn_hosting_policy::PnHostingPolicy::default(), + ) + .expect("bridge"); + + let sender = Identity::new(); + let other_recipient = Identity::new(); + let blob = super::super::propagation_download::build_client_download_blob( + &sender, + &other_recipient, + "not for us", + ); + { + let mut node = bridge.local_node.lock().expect("node lock"); + assert!(node.accept_stamped_propagated_blob(&blob, &[0u8; 32], u8::MAX)); + } + + let (messages, listed) = bridge.drain_local_inbox(); + assert!( + messages.is_empty() && listed == 0, + "mail addressed to another identity must not drain into our inbox" + ); + + let _ = std::fs::remove_dir_all(&dir); + } #[test] fn should_emit_terminal_success_skips_explicit_failure() { @@ -790,9 +1069,25 @@ mod tests { bridge.contains("start_sync_with_policy"), "bridge must expose policy sync for host peer loop" ); + // Inbox retrieval is the client `/get` download, driven from live.rs; the + // bridge only logs peer-sync (`/offer`) outcomes, never inbox retrieval. + assert!( + bridge.contains("peer_outcome"), + "peer sync Completes must log peer-sync (not retrieve) telemetry" + ); + assert!( + live.contains("propagation-retrieve"), + "client /get download must log inbox retrieve telemetry" + ); + // The `/get` pull must stay wired: PropagationClient owned by the bridge, + // driven from live.rs, so PN→inbox retrieval cannot silently regress. + assert!( + bridge.contains("PropagationClient") && bridge.contains("poll_client_download"), + "bridge must own the PropagationClient `/get` pull" + ); assert!( - bridge.contains("propagation-retrieve"), - "sync Completes must log retrieve telemetry" + live.contains("spawn_client_download_driver") && live.contains("start_client_download"), + "live sync must drive the client `/get` download" ); } diff --git a/reticulum-sidecar/src/stack/propagation_download.rs b/reticulum-sidecar/src/stack/propagation_download.rs new file mode 100644 index 000000000..b16cb62a8 --- /dev/null +++ b/reticulum-sidecar/src/stack/propagation_download.rs @@ -0,0 +1,139 @@ +//! Client-side propagation-node retrieval (`/get` pull) decode helpers. +//! +//! The user-facing **Sync** flow downloads store-and-forward mail addressed to +//! our own `lxmf.delivery` identity from a propagation node via +//! [`lxmf_core::propagation_client::PropagationClient`] (`/get` list → get → +//! purge, Python `LXMRouter.request_messages_from_propagation_node` parity). +//! +//! A downloaded entry (client-download form, propagation stamp already stripped +//! by the serving node) is `dest_hash(16) || encrypted_data`, where +//! `encrypted_data` is the RNS-encrypted `src_hash(16) || signature(64) || +//! msgpack_payload`. We decrypt with the local identity, reassemble the wire +//! message, and hand the resulting [`LxMessage`] to the router delivery +//! callback exactly like a Direct/opportunistic inbound message. + +use lxmf_core::constants::{DESTINATION_LENGTH, DeliveryMethod}; +use lxmf_core::message::LxMessage; +use rns_identity::identity::Identity; + +/// Outcome of one [`PropagationBridge::poll_client_download`] tick. +/// +/// [`PropagationBridge::poll_client_download`]: super::propagation_bridge::PropagationBridge::poll_client_download +#[derive(Debug)] +pub(crate) enum ClientDownloadPoll { + /// No download is active (never started, or already consumed). + Idle, + /// Download in progress (link/list/get/purge not yet terminal). + InProgress, + /// Download reached a terminal Complete. `messages` are decoded and ready + /// for the delivery callback; `listed` / `downloaded` are audit counts. + Complete { + messages: Vec, + listed: usize, + downloaded: usize, + }, + /// Download failed (link close, timeout, or malformed response). + Failed, +} + +/// Decode one client-downloaded propagation entry into an inbound [`LxMessage`]. +/// +/// `blob` is `dest_hash(16) || encrypted_data` (the serving node strips the +/// trailing propagation stamp for client downloads). Returns `None` when the +/// blob is too short, decryption fails (not addressed to us / wrong ratchet), +/// or the decrypted bytes are not a valid LXMF message. +pub(crate) fn decode_downloaded_propagated_blob( + identity: &Identity, + blob: &[u8], +) -> Option { + if blob.len() <= DESTINATION_LENGTH { + return None; + } + let (dest_hash, ciphertext) = blob.split_at(DESTINATION_LENGTH); + let plaintext = identity.decrypt(ciphertext, None, false).ok()?; + + let mut unpack_data = Vec::with_capacity(DESTINATION_LENGTH + plaintext.len()); + unpack_data.extend_from_slice(dest_hash); + unpack_data.extend_from_slice(&plaintext); + + let mut msg = LxMessage::unpack(&unpack_data).ok()?; + msg.incoming = true; + msg.method = DeliveryMethod::Propagated; + // Python computes the transient id over the (unstamped) lxmf_data; mirror it + // so retrieve telemetry correlates with the sender's deposit transient id. + msg.transient_id = Some(LxMessage::compute_propagation_transient_id(blob)); + Some(msg) +} + +/// Pack a propagated message for `recipient` in the client-download form the +/// serving node returns (`dest_hash || encrypted_data`, no stamp). Shared by the +/// download unit tests and the propagation-bridge local-prop loopback test. +#[cfg(test)] +pub(crate) fn build_client_download_blob( + sender: &Identity, + recipient: &Identity, + content: &str, +) -> Vec { + use rns_identity::destination::Destination; + + let dest_hash = + Destination::hash_from_name_and_identity("lxmf.delivery", Some(&recipient.hash)); + let src_hash = Destination::hash_from_name_and_identity("lxmf.delivery", Some(&sender.hash)); + let mut msg = LxMessage::new(dest_hash, src_hash, "", content, DeliveryMethod::Propagated); + msg.sign(&sender.get_signing_key().expect("sender signing key")) + .expect("sign"); + // pack_propagated_encrypted returns the msgpack propagation wrapper; we only + // need the inner lxmf_data entry (dest || encrypted_data) that a client + // download yields, so extract it to mirror the wire form. + let (wrapper, _tid) = { + let recipient_pub = recipient.get_public_key(); + let remote = Identity::from_public_key(&recipient_pub).expect("remote identity"); + msg.pack_propagated_encrypted(|plaintext| { + remote + .encrypt(plaintext, None) + .map_err(|e| lxmf_core::message::MessageError::PackFailed(e.to_string())) + }) + .expect("pack propagated encrypted") + }; + let (_ts, entries) = LxMessage::unpack_propagation_wrapper(&wrapper).expect("unpack wrapper"); + entries.into_iter().next().expect("one entry") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn decodes_downloaded_blob_addressed_to_us() { + let sender = Identity::new(); + let recipient = Identity::new(); + let blob = build_client_download_blob(&sender, &recipient, "hello via PN"); + + let decoded = decode_downloaded_propagated_blob(&recipient, &blob) + .expect("recipient can decode its own mail"); + assert_eq!(decoded.content, "hello via PN"); + assert!(decoded.incoming); + assert_eq!(decoded.method, DeliveryMethod::Propagated); + assert!(decoded.transient_id.is_some()); + } + + #[test] + fn rejects_blob_not_addressed_to_us() { + let sender = Identity::new(); + let recipient = Identity::new(); + let stranger = Identity::new(); + let blob = build_client_download_blob(&sender, &recipient, "not for you"); + + assert!( + decode_downloaded_propagated_blob(&stranger, &blob).is_none(), + "a foreign identity must not decode mail addressed to the recipient" + ); + } + + #[test] + fn rejects_too_short_blob() { + let identity = Identity::new(); + assert!(decode_downloaded_propagated_blob(&identity, &[0u8; 8]).is_none()); + assert!(decode_downloaded_propagated_blob(&identity, &[0u8; DESTINATION_LENGTH]).is_none()); + } +}