Two FileStorageProvider objects on one dataDir clobber the entire wallet file, money journals included

[MastaP]

Found while designing the tracked_addresses fix for #766. This is strictly larger than that issue and is not covered by #766, #768, #769 or #770.

The defect

FileStorageProvider holds the whole store as an in-memory snapshot loaded once at connect() (impl/nodejs/storage/FileStorageProvider.ts:70-112), and save() rewrites the entire file on every set() (:218-222, JSON.stringify(this.data)).

So with two provider objects over one dataDir:

B's next set() of any key erases every key A has written since B connected.

Not one key — all of them. Including:

• pv2g2:{network}:{pubkey}:delivery-journal — the only place a certified-but-undelivered token's bytes exist (see Money-critical: a pending delivery is stranded on intent completion, and a trust-base rotation freezes the fleet #768.1). Losing it loses a spent token with no error surface.
• pv2g2:* intent backstop, mint journal, sync cursors, the wallet-api refresh token.
• mnemonic / master_key after a password rotation.

IndexedDBStorageProvider is not exposed to this — every get/set goes to the DB rather than through an in-memory snapshot.

Why it isn't the same bug as #766 item 5

#766 item 5 (tracked_addresses lost update) is a single key with a wholesale write, fixable with a read-merge-write serialized inside one provider instance. That fix is correct and worth having, but it only covers one provider object shared by two Spheres — the normal browser wiring.

This is the other case: two provider objects, where no in-provider chain can help, because each has its own snapshot of the whole file and neither observes the other's writes.

Reach

Any Node consumer that constructs two FileStorageProviders on the same dataDir — the exact shape #766 is trying to make safe ("one Sphere per network, disposable, concurrent instances must not interfere"). docs/MIGRATION-TOKEN-REGISTRY.md:95-100 already notes two providers over one dataDir as an unresolved residual; this is what that residual actually costs.

Options

1. Re-read before write in save() — merge the on-disk object with the in-memory delta rather than overwriting. Costs a read per write; makes concurrent providers safe by construction.
2. Enforce one provider per dataDir — a lockfile or a process-level registry, failing loudly on a second open. Cheaper, and arguably the honest contract, but it turns a silent corruption into a startup error a consumer must handle.
3. Per-key files instead of one JSON blob. Biggest change; removes the class of bug entirely.

Option 2 is the smallest thing that stops data loss; option 1 is the smallest thing that keeps the current API working. Worth deciding before #766 is described as delivered, since "concurrent instances must not interfere" is not true while this stands.

---

Found 2026-09-02 against main @ 74f26e3. Not yet verified by an executed repro — the mechanism is read from source (connect() snapshot + whole-file save()); a two-provider test would confirm it in a few lines.

[MastaP]

Reviewers keep re-deriving this, so recording the shape and the refused partial fix

Two independent reviewers on #772 (Copilot and a local codex pass, twice) proposed the same
narrow fix: share FileStorageProvider's tracked-address write chain across provider
objects — keyed by backingStoreId, as LocalStorageProvider already does — and refresh
from disk inside it.

Declined, and the reason is worth pinning here rather than being re-argued each round.

The premise both reviewers start from is that two objects over one dataDir became a
supported configuration once #772 gave them the same backingStoreId. They did not.
backingStoreId scopes teardown — which live Spheres a Sphere.clear() must destroy —
and #772's release notes and migration guide both say the concurrent-write half is still
open and tracked here.

The narrow fix would not make the provider safe. saveTrackedAddresses merges and then
calls set(), and set() writes the entire wallet file from this.data, which is a
per-object in-memory cache of every key. So a sibling object's unrelated set() — a money
journal entry, a cursor, anything — rewrites the file from its own stale snapshot and rolls
the tracked registry back, however carefully the tracked write itself was serialized. The
contract suite's cross-object case would go green while the provider stayed unsafe, which is
exactly what the unsupported: note in
tests/unit/storage/tracked-addresses-providers.test.ts says. I implemented the change once
during the review rounds, then reverted it for this reason; the refusal now also lives in the
provider's own docstring so the next reader finds it without re-deriving it.

What the real fix has to be

Every mutating operation — not just saveTrackedAddresses — must refresh from the file and
write under coordination scoped to the file:

1. set / remove / clear re-read the file inside a per-backingStoreId chain before
   merging their change into this.data, or
2. the provider stops caching the whole store and reads/writes per key, or
3. an advisory file lock (proper-lockfile or an O_EXCL sidecar) covers the read-write
   pair, which is also the only option that survives a second process — the in-process
   chain cannot.

(1) is the smallest, but it puts a file read on the hot path of every money-journal write, so
it wants measurement rather than an overnight change. (3) is the only one that closes the
cross-process case, which is the realistic Node deployment shape.

Not the same as the browser providers

IndexedDBStorageProvider does its read-merge-put in one readwrite transaction, so it
is safe across objects and across tabs. LocalStorageProvider serializes per backing store
in one JS realm and does not cache the store. Neither rewrites a whole file from a private
snapshot, which is what makes this provider different in kind rather than degree.