From cea1c072536fcdeb722163bbecf1c4a256717828 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Wed, 2 Sep 2026 22:17:40 +0200 Subject: [PATCH 01/43] refactor(sphere)!: remove the Sphere.instance global; scope clear()/import() to their storage (#766) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit BREAKING: `Sphere.getInstance()`, `Sphere.isInitialized()` and the root export `getSphere` are removed. Root exports 126 -> 125. The consumer gate found ZERO users across all 29 sibling repos — every fleet `getInstance` is TokenRegistry's, every `getSphere` is repo-local, every `isInitialized` is a consumer's own field. The replacement is the instance the entry point already returns, plus `sphere.isReady`. Why they had to go rather than be deprecated: after a second Sphere is created and then destroyed, `getInstance()` returns null while the first is alive and serving money. A deprecation note does not stop a wrong answer being consumed. The static is replaced by a PRIVATE `WeakMap>` used only by clear()/import(). Keyed by object identity — StorageProvider.id is a class constant ('file-storage'), so comparing it would still have destroyed an unrelated instance. Private, so it is not a liveness API; WeakMap, so it is bounded by the provider's own lifetime. Registration happens at exactly the three publication points, preserving #767's invariant that a half-built Sphere is never reachable. - clear({storage}) now destroys only the Spheres built on THAT storage. It used to destroy whichever Sphere was constructed last, killing a live wallet on an unrelated provider and dropping every sphere.on() handler with no event and no error. - import()'s `needsClear` is likewise storage-scoped; it used to fire merely because some instance was live somewhere. The `exists(storage)` disjunct is preserved, which is the storage-wipe contract sphere and Boxy-Run actually depend on. - importFromLegacyFile returned `Sphere.getInstance()!` because importFromJSON DISCARDED the Sphere it built. Threading it out (additive, non-breaking) fixes the cause, not the symptom — an interleaved init used to make that path return the wrong object. Also folded in, both in the same entry-point preambles this touches: - #770.5: TokenRegistry.resetInstance() now calls the instance dispose() instead of only stopAutoRefresh(), which left `disposed` unset, the generation unchanged and the in-flight fetch running — so a load past its entry guard re-armed the interval on an instance getInstance() could no longer return. - #769.2: init now forwards `verification` to create/load. It was silently dropped, so a consumer opting into the worker pool at the documented entry point got the sequential verifier with no indication. (The issue also claimed `debug` was dropped; it is not — init configures the global logger itself.) - #766 item 4: the debug flag was one-way. `if (options.debug)` meant no second init could ever turn it off; all four entry points now honour an explicit false. createNodeProviders had the mirror-image bug — `?? false` silently disabled a flag the consumer had already set — and now matches createBrowserProviders in only overriding when told to. The logger stays process-global deliberately. 370 call sites across 27 files, most in providers constructed before any Sphere exists and shared between Spheres by design (ConnectClient has no owning Sphere at all). Threading a handle would break every provider constructor to gain per-instance control of a third of the messages, and the worst outcome of the global is noisy stdout — unlike the registry global, which produced a 10^8 balance error. Tests: singleton-hygiene resets deleted (they existed only because the static did); afterEach blocks that used getInstance() to find the leftover now hold the returned reference. Two real guards kept with the mechanism changed, not the assertion — the clear()-destroys-a-live-Sphere test seeds the private map instead of the static. New tests/integration/sphere-instance-scoping.test.ts proves the point in both directions, and was falsified against three separate mutations: restoring the old last-constructed-wins static reds the survives half, making liveOn return nothing reds the own-Sphere-is-destroyed half, and returning every Sphere regardless of storage reds all three. Construction order in those tests is load-bearing — A is built last, so it is exactly the instance the deleted static pointed at. --- core/Sphere.ts | 106 +++++---- impl/nodejs/index.ts | 5 +- index.ts | 2 +- registry/TokenRegistry.ts | 8 +- .../integration/nametag-normalization.test.ts | 4 - .../nametag-overwrite-guard.test.ts | 11 - .../sphere-instance-scoping.test.ts | 213 ++++++++++++++++++ .../sphere-payments-v2-wiring.test.ts | 18 +- tests/integration/wallet-clear.test.ts | 4 - tests/unit/core/Sphere.clear.test.ts | 28 +-- .../unit/core/Sphere.destroy-secrets.test.ts | 21 +- .../core/Sphere.network-delegation.test.ts | 16 +- .../unit/core/Sphere.registerNametag.test.ts | 4 - tests/unit/core/Sphere.status.test.ts | 14 +- 14 files changed, 343 insertions(+), 111 deletions(-) create mode 100644 tests/integration/sphere-instance-scoping.test.ts diff --git a/core/Sphere.ts b/core/Sphere.ts index 8f24608a..3c26b943 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -458,8 +458,12 @@ export interface AddressModuleSet { // ============================================================================= export class Sphere { - // Singleton - private static instance: Sphere | null = null; + // Live Spheres, keyed by the StorageProvider whose data they own. NOT a liveness API: + // it exists only so clear()/import() tear down the instances that actually use the + // storage they were handed, instead of whichever Sphere was constructed last. Keyed by + // object identity, so Spheres on different providers never see each other. Two + // providers over one dataDir/DB are still distinct keys — see #766. + private static readonly _liveByStorage = new WeakMap>(); // One-time best-effort cleanup of the orphaned vesting cache (prior versions). private static _orphanCacheCleaned = false; @@ -630,7 +634,10 @@ export class Sphere { */ static async init(options: SphereInitOptions): Promise { // Configure debug logging (also needed in main bundle context, same as TokenRegistry) - if (options.debug) logger.configure({ debug: true }); + // `undefined` leaves whatever the provider factory or consumer set; an explicit + // `false` MUST turn debug off. A truthy-only check made this process-global flag + // one-way — no second init could ever quieten it (#766). + if (options.debug !== undefined) logger.configure({ debug: options.debug }); // Fail-closed BEFORE any work: retired module options + the required // wallet-api composition (create/load re-check for direct callers). @@ -662,6 +669,7 @@ export class Sphere { market, communications: options.communications, password: options.password, + verification: options.verification, discoverAddresses: options.discoverAddresses, onProgress: options.onProgress, }); @@ -703,6 +711,7 @@ export class Sphere { market, communications: options.communications, password: options.password, + verification: options.verification, discoverAddresses: options.discoverAddresses, onProgress: options.onProgress, }); @@ -807,8 +816,8 @@ export class Sphere { /** * Own a registry for the duration of `bringUp`, disposing it if any of that rejects. * - * `bringUp` must cover EVERY fallible step from here until the Sphere is published to - * `Sphere.instance` — until then the caller receives nothing it could destroy, so a + * `bringUp` must cover EVERY fallible step from here until the Sphere is returned to + * its caller — until then nobody holds anything they could destroy, so a * registry left behind is unreachable and its hourly fetch runs for the life of the * process. Guarding a named subset of the steps is what failed twice: the guarded region * and the fallible region were separate things, and drifted. @@ -848,7 +857,10 @@ export class Sphere { * Create new wallet with mnemonic */ static async create(options: SphereCreateOptions): Promise { - if (options.debug) logger.configure({ debug: true }); + // `undefined` leaves whatever the provider factory or consumer set; an explicit + // `false` MUST turn debug off. A truthy-only check made this process-global flag + // one-way — no second init could ever quieten it (#766). + if (options.debug !== undefined) logger.configure({ debug: options.debug }); // Fail-closed BEFORE any storage write: retired module options + the // required wallet-api composition. @@ -953,7 +965,7 @@ export class Sphere { progress?.({ step: 'complete', message: 'Wallet created' }); }); - Sphere.instance = sphere; + Sphere.registerLive(sphere); return sphere; } @@ -961,7 +973,10 @@ export class Sphere { * Load existing wallet from storage */ static async load(options: SphereLoadOptions): Promise { - if (options.debug) logger.configure({ debug: true }); + // `undefined` leaves whatever the provider factory or consumer set; an explicit + // `false` MUST turn debug off. A truthy-only check made this process-global flag + // one-way — no second init could ever quieten it (#766). + if (options.debug !== undefined) logger.configure({ debug: options.debug }); // Fail-closed first: retired module options + the required wallet-api composition. Sphere.refuseRetiredModuleOptions(options); @@ -1035,7 +1050,7 @@ export class Sphere { progress?.({ step: 'complete', message: 'Wallet loaded' }); }); - Sphere.instance = sphere; + Sphere.registerLive(sphere); return sphere; } @@ -1043,7 +1058,10 @@ export class Sphere { * Import wallet from mnemonic or master key */ static async import(options: SphereImportOptions): Promise { - if (options.debug) logger.configure({ debug: true }); + // `undefined` leaves whatever the provider factory or consumer set; an explicit + // `false` MUST turn debug off. A truthy-only check made this process-global flag + // one-way — no second init could ever quieten it (#766). + if (options.debug !== undefined) logger.configure({ debug: options.debug }); // Fail-closed BEFORE the destructive clear below: retired module options + // the required wallet-api composition. @@ -1061,7 +1079,7 @@ export class Sphere { // Clear existing wallet if any. Skip if no active instance and wallet // doesn't exist — avoids a redundant IndexedDB delete/reopen that can race // with a subsequent initialize(). - const needsClear = Sphere.instance !== null || await Sphere.exists(options.storage); + const needsClear = Sphere.liveOn(options.storage).length > 0 || (await Sphere.exists(options.storage)); if (needsClear) { progress?.({ step: 'clearing', message: 'Clearing previous wallet data...' }); logger.debug('Sphere', 'Clearing existing wallet data...'); @@ -1189,7 +1207,7 @@ export class Sphere { logger.debug('Sphere', 'Import complete'); }); - Sphere.instance = sphere; + Sphere.registerLive(sphere); return sphere; } @@ -1211,9 +1229,12 @@ export class Sphere { // 1. Destroy Sphere instance — stops the payments vertical (quiescence), // then closes all connections. - if (Sphere.instance) { - logger.debug('Sphere', 'Destroying Sphere instance...'); - await Sphere.instance.destroy(); + // Scoped on purpose: destroying whichever Sphere was constructed last silently + // killed a live wallet on an unrelated provider, dropping every sphere.on() handler + // with no event and no error (#766). A Sphere on other storage is not our business. + for (const live of Sphere.liveOn(storage)) { + logger.debug('Sphere', 'Destroying Sphere instance on this storage...'); + await live.destroy(); logger.debug('Sphere', 'Sphere instance destroyed'); } @@ -1286,18 +1307,23 @@ export class Sphere { } catch { /* ignore — cleanup is best-effort */ } } - /** - * Get current instance - */ - static getInstance(): Sphere | null { - return Sphere.instance; + /** Record a fully-built Sphere against the storage it owns. See `_liveByStorage`. */ + private static registerLive(sphere: Sphere): void { + let live = Sphere._liveByStorage.get(sphere._storage); + if (!live) { + live = new Set(); + Sphere._liveByStorage.set(sphere._storage, live); + } + live.add(sphere); } - /** - * Check if initialized - */ - static isInitialized(): boolean { - return Sphere.instance?._initialized ?? false; + private static unregisterLive(sphere: Sphere): void { + Sphere._liveByStorage.get(sphere._storage)?.delete(sphere); + } + + /** Snapshot — callers iterate this while destroy() mutates the underlying Set. */ + private static liveOn(storage: StorageProvider): Sphere[] { + return Array.from(Sphere._liveByStorage.get(storage) ?? []); } /** @@ -1576,8 +1602,7 @@ export class Sphere { }); if (result.success) { - const sphere = Sphere.getInstance(); - return { success: true, sphere: sphere!, mnemonic: result.mnemonic }; + return { success: true, sphere: result.sphere, mnemonic: result.mnemonic }; } if (!password && result.error?.includes('Password required')) { @@ -1971,7 +1996,7 @@ export class Sphere { static async importFromJSON(options: Omit & { jsonContent: string; password?: string; - }): Promise<{ success: boolean; mnemonic?: string; error?: string }> { + }): Promise<{ success: boolean; sphere?: Sphere; mnemonic?: string; error?: string }> { const { jsonContent, password, ...baseOptions } = options; try { @@ -2011,20 +2036,20 @@ export class Sphere { // Import using mnemonic if available (preferred) if (mnemonic) { - await Sphere.import({ ...baseOptions, mnemonic, basePath }); - return { success: true, mnemonic }; + const sphere = await Sphere.import({ ...baseOptions, mnemonic, basePath }); + return { success: true, sphere, mnemonic }; } // Otherwise import using master key if (masterKey) { - await Sphere.import({ + const sphere = await Sphere.import({ ...baseOptions, masterKey, chainCode: data.wallet.chainCode, basePath, derivationMode: data.derivationMode || (data.wallet.isBIP32 ? 'bip32' : 'wif_hmac'), }); - return { success: true }; + return { success: true, sphere }; } return { success: false, error: 'No mnemonic or master key in wallet data' }; @@ -2156,7 +2181,11 @@ export class Sphere { } if (entry.hidden === hidden) return; - (entry as { hidden: boolean }).hidden = hidden; + // `updatedAt` moves with `hidden`: the registry merge (#766 item 5) resolves a + // conflicting entry by the greater `updatedAt`, so a stale flag left with its old + // timestamp would let another Sphere's snapshot win and silently undo this change. + (entry as { hidden: boolean; updatedAt: number }).hidden = hidden; + (entry as { hidden: boolean; updatedAt: number }).updatedAt = Date.now(); await this.persistTrackedAddresses(); const eventType = hidden ? 'address:hidden' : 'address:unhidden'; @@ -2643,7 +2672,11 @@ export class Sphere { } if (tracked.hidden !== hidden) { - (tracked as { hidden: boolean }).hidden = hidden; + // Bump `updatedAt` with `hidden` — the registry merge (#766 item 5) breaks a + // conflict by the greater `updatedAt`, so an unbumped flag can be overwritten + // by another Sphere's older snapshot. + (tracked as { hidden: boolean; updatedAt: number }).hidden = hidden; + (tracked as { hidden: boolean; updatedAt: number }).updatedAt = Date.now(); } } @@ -3537,9 +3570,7 @@ export class Sphere { this._disabledProviders.clear(); this.eventHandlers.clear(); - if (Sphere.instance === this) { - Sphere.instance = null; - } + Sphere.unregisterLive(this); } // =========================================================================== @@ -4147,5 +4178,4 @@ export const createSphere = Sphere.create.bind(Sphere); export const loadSphere = Sphere.load.bind(Sphere); export const importSphere = Sphere.import.bind(Sphere); export const initSphere = Sphere.init.bind(Sphere); -export const getSphere = Sphere.getInstance.bind(Sphere); export const sphereExists = Sphere.exists.bind(Sphere); diff --git a/impl/nodejs/index.ts b/impl/nodejs/index.ts index acc144e2..61bfa7a3 100644 --- a/impl/nodejs/index.ts +++ b/impl/nodejs/index.ts @@ -172,8 +172,9 @@ export function createNodeProviders(config?: NodeProvidersConfig): NodeProviders assertNetworkConsistency(network); // Configure global logger: top-level debug enables all, per-provider overrides are additive - const globalDebug = config?.debug ?? false; - sdkLogger.configure({ debug: globalDebug }); + // Only override when explicitly provided — `?? false` silently disabled a debug flag + // the consumer had already configured. Matches createBrowserProviders. + if (config?.debug !== undefined) sdkLogger.configure({ debug: config.debug }); if (config?.transport?.debug) sdkLogger.setTagDebug('Nostr', true); if (config?.oracle?.debug) sdkLogger.setTagDebug('Aggregator', true); if (config?.price?.debug) sdkLogger.setTagDebug('Price', true); diff --git a/index.ts b/index.ts index 0271a1f6..d390d76b 100644 --- a/index.ts +++ b/index.ts @@ -48,7 +48,7 @@ // Core // ============================================================================= -export { Sphere, createSphere, loadSphere, initSphere, getSphere, sphereExists, checkNetworkHealth, logger, SphereError, PartialSendConflictError, isSphereError, isPossiblyCommittedSendOutcome } from './core'; +export { Sphere, createSphere, loadSphere, initSphere, sphereExists, checkNetworkHealth, logger, SphereError, PartialSendConflictError, isSphereError, isPossiblyCommittedSendOutcome } from './core'; export { signMessage, verifySignedMessage, hashSignMessage, recoverPubkeyFromSignature, SIGN_MESSAGE_PREFIX } from './core/crypto'; export type { SphereCreateOptions, diff --git a/registry/TokenRegistry.ts b/registry/TokenRegistry.ts index 4bc5a295..5ad5ef17 100644 --- a/registry/TokenRegistry.ts +++ b/registry/TokenRegistry.ts @@ -258,9 +258,11 @@ export class TokenRegistry { * Stops auto-refresh if running. */ static resetInstance(): void { - if (TokenRegistry.instance) { - TokenRegistry.instance.stopAutoRefresh(); - } + // dispose(), not stopAutoRefresh(): the latter leaves `disposed` unset, the generation + // unchanged and the in-flight fetch running, so a load already past its entry guard + // re-arms the interval on an instance getInstance() can no longer return — an + // unstoppable timer plus a live abort timer. (#770) + TokenRegistry.instance?.dispose(); TokenRegistry.instance = null; } diff --git a/tests/integration/nametag-normalization.test.ts b/tests/integration/nametag-normalization.test.ts index 900480e3..b3e4a333 100644 --- a/tests/integration/nametag-normalization.test.ts +++ b/tests/integration/nametag-normalization.test.ts @@ -102,14 +102,10 @@ describe('Nametag normalization integration', () => { beforeEach(() => { cleanTestDir(); nostrRelayNametags.clear(); - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } storage = new FileStorageProvider({ dataDir: DATA_DIR }); }); afterEach(() => { - (Sphere as unknown as { instance: null }).instance = null; cleanTestDir(); nostrRelayNametags.clear(); }); diff --git a/tests/integration/nametag-overwrite-guard.test.ts b/tests/integration/nametag-overwrite-guard.test.ts index 1dd2d146..6b32c515 100644 --- a/tests/integration/nametag-overwrite-guard.test.ts +++ b/tests/integration/nametag-overwrite-guard.test.ts @@ -156,14 +156,10 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { beforeEach(() => { cleanTestDir(); clearRelay(); - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } storage = new FileStorageProvider({ dataDir: DATA_DIR }); }); afterEach(() => { - (Sphere as unknown as { instance: null }).instance = null; cleanTestDir(); clearRelay(); }); @@ -221,7 +217,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { expect(binding!.nametag).toBe('alice'); await sphere1.destroy(); - (Sphere as unknown as { instance: null }).instance = null; transport.publishIdentityBinding.mockClear(); transport.resolve.mockClear(); @@ -267,7 +262,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { expect(sphere1.identity!.nametag).toBe('bob'); await sphere1.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // 2. Simulate nametag loss: remove nametag from storage but keep binding on relay // Clear nametag from addressNametags in storage @@ -345,7 +339,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { expect(relayBindings.get(directAddr)!.nametag).toBe('carol'); await sphere1.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // 2. Reload with broken transport (resolve throws) const transport2 = createMockTransport({ resolveThrows: true }); @@ -388,7 +381,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { const _directAddr = sphere1.identity!.directAddress!; await sphere1.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // 2. Simulate nametag loss in local storage const identityKey = 'sphere_identity'; @@ -418,7 +410,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { expect(sphere2.identity!.nametag).toBe('dave'); await sphere2.destroy(); - (Sphere as unknown as { instance: null }).instance = null; transport.publishIdentityBinding.mockClear(); // 4. Second reload — nametag should be in local storage now, no need to recover @@ -457,7 +448,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { const directAddr = sphere1.identity!.directAddress!; await sphere1.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // 2. Simulate legacy event format on relay: // - binding exists (found by chainPubkey.slice(2)) @@ -515,7 +505,6 @@ describe('Nametag overwrite guard (syncIdentityWithTransport)', () => { expect(migrated!.nametag).toBe('legacy_user'); await sphere2.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // 5. Second reload — should find new-format event, no migration needed transport.publishIdentityBinding.mockClear(); diff --git a/tests/integration/sphere-instance-scoping.test.ts b/tests/integration/sphere-instance-scoping.test.ts new file mode 100644 index 00000000..64effeec --- /dev/null +++ b/tests/integration/sphere-instance-scoping.test.ts @@ -0,0 +1,213 @@ +/** + * #766 — Sphere lifecycle statics are storage-scoped. + * + * `Sphere.clear()` and `Sphere.import()` tear down the live Spheres registered against + * the StorageProvider they were HANDED, and nothing else. The process-global + * `Sphere.instance` they used to consult held whichever Sphere was constructed LAST, so + * clearing wallet B silently destroyed a live, unrelated wallet A: its payments vertical + * stopped, its providers disconnected and every `sphere.on()` handler was dropped, with + * no event and no error for the owner of A to observe. + * + * Construction order is load-bearing in these tests: B's Sphere is built FIRST and A's + * LAST, so A is exactly the instance the old static pointed at. Both directions are + * pinned — clearing an unrelated storage must NOT destroy A (tests 1 and 2), and + * clearing A's OWN storage still MUST (test 3). Scoping that forgot the second half + * would leave a Sphere alive on a KV that was just emptied under it. + */ + +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { Sphere } from '../../core/Sphere'; +import { FileStorageProvider } from '../../impl/nodejs/storage/FileStorageProvider'; +import type { TransportProvider } from '../../transport'; +import type { OracleProvider } from '../../oracle'; +import type { ProviderStatus } from '../../types'; +import { makePv2World, createEngineOracle, type Pv2World } from '../support/pv2-world'; + +const NET = 'testnet2' as const; + +const MNEMONIC_A = 'test test test test test test test test test test test junk'; +const MNEMONIC_B = + 'abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about'; +const MNEMONIC_C = + 'legal winner thank year wave sausage worth useful legal winner thank yellow'; + +function createMockTransport(): TransportProvider { + return { + id: 'mock-transport', + name: 'Mock Transport', + type: 'p2p' as const, + description: 'Mock transport', + setIdentity: vi.fn(), + connect: vi.fn().mockResolvedValue(undefined), + disconnect: vi.fn().mockResolvedValue(undefined), + isConnected: vi.fn().mockReturnValue(true), + getStatus: vi.fn().mockReturnValue('connected' as ProviderStatus), + sendMessage: vi.fn().mockResolvedValue('event-id'), + onMessage: vi.fn().mockReturnValue(() => {}), + subscribeToBroadcast: vi.fn().mockReturnValue(() => {}), + publishBroadcast: vi.fn().mockResolvedValue('broadcast-id'), + onEvent: vi.fn().mockReturnValue(() => {}), + resolve: vi.fn().mockResolvedValue(null), + resolveNametag: vi.fn().mockResolvedValue(null), + publishIdentityBinding: vi.fn().mockResolvedValue(true), + recoverNametag: vi.fn().mockResolvedValue(null), + } as unknown as TransportProvider; +} + +/** One wallet's worth of independent providers — its own dataDir, storage, transport. */ +interface Wallet { + dataDir: string; + storage: FileStorageProvider; + transport: TransportProvider; + oracle: OracleProvider; + world: Pv2World; + sphere?: Sphere; +} + +const wallets: Wallet[] = []; + +function makeWallet(label: string): Wallet { + const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), `sphere-scope-${label}-`)); + const wallet: Wallet = { + dataDir, + storage: new FileStorageProvider({ dataDir }), + transport: createMockTransport(), + oracle: createEngineOracle(), + world: makePv2World(NET), + }; + wallets.push(wallet); + return wallet; +} + +async function initWallet(wallet: Wallet, mnemonic: string): Promise { + const { sphere } = await Sphere.init({ + storage: wallet.storage, + transport: wallet.transport, + oracle: wallet.oracle, + walletApi: wallet.world.walletApi, + network: NET, + mnemonic, + }); + wallet.sphere = sphere; + return sphere; +} + +describe('Sphere lifecycle statics are scoped to the storage they are handed (#766)', () => { + beforeEach(() => { + // The registry's remote refresh must not reach the network from an integration test. + vi.stubGlobal( + 'fetch', + vi.fn(async () => ({ + ok: true, + status: 200, + statusText: 'OK', + json: async () => [], + text: async () => '[]', + } as unknown as Response)), + ); + }); + + afterEach(async () => { + for (const wallet of wallets.splice(0)) { + try { + await wallet.sphere?.destroy(); + } catch { /* already torn down by the test */ } + fs.rmSync(wallet.dataDir, { recursive: true, force: true }); + } + vi.unstubAllGlobals(); + }); + + it('clear() on another storage leaves a live Sphere on a different one fully alive', async () => { + // B first, A last: A is the instance the deleted process-global static held. + const b = makeWallet('b'); + await initWallet(b, MNEMONIC_B); + const a = makeWallet('a'); + const sphereA = await initWallet(a, MNEMONIC_A); + + const chainPubkeyBefore = sphereA.identity!.chainPubkey; + // Registered BEFORE the clear — destroy() drops every handler, so a handler that + // still fires afterwards is proof A's event bus was never torn down. + const activated: unknown[] = []; + sphereA.on('address:activated', (data) => activated.push(data)); + + await Sphere.clear({ storage: b.storage }); + + // A is untouched: still initialized, still holding its identity... + expect(sphereA.isReady).toBe(true); + expect(sphereA.identity).not.toBeNull(); + expect(sphereA.identity!.chainPubkey).toBe(chainPubkeyBefore); + // ...still able to hand out the payments vertical (the getter THROWS once stopped)... + expect(() => sphereA.payments).not.toThrow(); + expect(a.transport.disconnect).not.toHaveBeenCalled(); + expect(a.storage.isConnected()).toBe(true); + + // ...and its handlers still fire. + await sphereA.switchToAddress(1); + expect(activated).toHaveLength(1); + expect((activated[0] as { address: { index: number } }).address.index).toBe(1); + + // Sanity, so "A survived" can never be read as "clear() did nothing": B is the + // wallet that WAS cleared, and its Sphere is gone. + expect(b.sphere!.isReady).toBe(false); + }); + + it('import() onto another storage leaves a live Sphere on a different one fully alive', async () => { + // Same ordering: A is built last, so the old static pointed at it. + const b = makeWallet('b'); + await initWallet(b, MNEMONIC_B); + const a = makeWallet('a'); + const sphereA = await initWallet(a, MNEMONIC_A); + + const chainPubkeyBefore = sphereA.identity!.chainPubkey; + const activated: unknown[] = []; + sphereA.on('address:activated', (data) => activated.push(data)); + + // import() clears storage B first — via the same storage-scoped teardown. + const imported = await Sphere.import({ + storage: b.storage, + transport: b.transport, + oracle: b.oracle, + walletApi: b.world.walletApi, + network: NET, + mnemonic: MNEMONIC_C, + }); + b.sphere = imported; + + // Sanity: the import really happened and really replaced B's wallet. + expect(imported.isReady).toBe(true); + expect(imported.identity!.chainPubkey).not.toBe(chainPubkeyBefore); + + expect(sphereA.isReady).toBe(true); + expect(sphereA.identity).not.toBeNull(); + expect(sphereA.identity!.chainPubkey).toBe(chainPubkeyBefore); + expect(() => sphereA.payments).not.toThrow(); + expect(a.transport.disconnect).not.toHaveBeenCalled(); + expect(a.storage.isConnected()).toBe(true); + + await sphereA.switchToAddress(1); + expect(activated).toHaveLength(1); + }); + + it('clear() on a Sphere OWN storage still destroys it — scoped, not abandoned', async () => { + const b = makeWallet('b'); + await initWallet(b, MNEMONIC_B); + const a = makeWallet('a'); + const sphereA = await initWallet(a, MNEMONIC_A); + + await Sphere.clear({ storage: a.storage }); + + // A owned that KV, so leaving it running over emptied storage is not an option. + expect(sphereA.isReady).toBe(false); + expect(sphereA.identity).toBeNull(); + expect(() => sphereA.payments).toThrow(); + expect(a.transport.disconnect).toHaveBeenCalled(); + + // ...and B, which was NOT cleared, is still alive. + expect(b.sphere!.isReady).toBe(true); + expect(() => b.sphere!.payments).not.toThrow(); + }); +}); diff --git a/tests/integration/sphere-payments-v2-wiring.test.ts b/tests/integration/sphere-payments-v2-wiring.test.ts index d859657e..b37e07a1 100644 --- a/tests/integration/sphere-payments-v2-wiring.test.ts +++ b/tests/integration/sphere-payments-v2-wiring.test.ts @@ -347,7 +347,7 @@ describe('Sphere payments wiring — the token registry is OWNED, not the proces // The guard now covers everything up to publication, not a named subset of steps. // Twice I widened it one step and the next step along was still unguarded; this pins // the far end — a failure at 'finalizing', after providers AND modules are up, still - // happens before Sphere.instance is assigned, so the caller gets nothing to destroy. + // happens before the Sphere is published, so the caller gets nothing to destroy. const create = vi.spyOn(TokenRegistry, 'create'); const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'pv2-registry-late2-')); const storage = new FileStorageProvider({ dataDir }); @@ -376,10 +376,11 @@ describe('Sphere payments wiring — the token registry is OWNED, not the proces it('disposes the registry when the LAST init step rejects, after publication would have been', async () => { // Publication used to happen mid-init, and the guard ended there on the premise that a - // published Sphere is recoverable via Sphere.getInstance(). That premise fails under - // concurrent inits — a second one overwrites the static — so the guard now runs to the - // end and publication is the last thing before the return. 'complete' is the final - // progress step in create(), so a throw here is past every other fallible operation. + // published Sphere is still recoverable by the caller. It is not: publication only + // records the Sphere in the private per-storage registry clear()/import() tear down + // (#766) — there is no lookup API at all — so the guard runs to the end and publication + // is the last thing before the return. 'complete' is the final progress step in + // create(), so a throw here is past every other fallible operation. const create = vi.spyOn(TokenRegistry, 'create'); const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'pv2-registry-last-')); const storage = new FileStorageProvider({ dataDir }); @@ -401,11 +402,10 @@ describe('Sphere payments wiring — the token registry is OWNED, not the proces expect(create).toHaveBeenCalledTimes(1); expect((create.mock.results[0]!.value as TokenRegistry).isDisposed).toBe(true); - // And the failed init published nothing, so no half-built Sphere is reachable. - expect(Sphere.getInstance()).toBeNull(); + // The failed init published nothing, so no half-built Sphere is reachable by anyone. // Precisely because nothing is reachable, the failure path must tear the whole - // Sphere down — providers are connected and the vertical is running by now, and - // disposing only the registry would strand all of it with no owner. + // Sphere down itself — providers are connected and the vertical is running by now, + // and disposing only the registry would strand all of it with no owner. expect(transport.disconnect).toHaveBeenCalled(); expect(storage.isConnected()).toBe(false); } finally { diff --git a/tests/integration/wallet-clear.test.ts b/tests/integration/wallet-clear.test.ts index 485332bd..e7f5ea53 100644 --- a/tests/integration/wallet-clear.test.ts +++ b/tests/integration/wallet-clear.test.ts @@ -116,14 +116,10 @@ describe('Sphere.clear() integration', () => { beforeEach(() => { cleanTestDir(); - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } storage = new FileStorageProvider({ dataDir: DATA_DIR }); }); afterEach(() => { - (Sphere as unknown as { instance: null }).instance = null; cleanTestDir(); clearNostrRelay(); }); diff --git a/tests/unit/core/Sphere.clear.test.ts b/tests/unit/core/Sphere.clear.test.ts index 1c730e12..6fce5012 100644 --- a/tests/unit/core/Sphere.clear.test.ts +++ b/tests/unit/core/Sphere.clear.test.ts @@ -5,7 +5,7 @@ * cursors, journals all live in the plain StorageProvider). */ -import { describe, it, expect, vi, beforeEach } from 'vitest'; +import { describe, it, expect, vi } from 'vitest'; import { Sphere } from '../../../core/Sphere'; import type { StorageProvider } from '../../../storage'; import type { ProviderStatus } from '../../../types'; @@ -43,14 +43,6 @@ function createMockStorage(): StorageProvider & { _data: Map } { // ============================================================================= describe('Sphere.clear()', () => { - beforeEach(() => { - // Reset Sphere singleton - if (Sphere.getInstance()) { - // Force reset without calling destroy (which needs providers) - (Sphere as unknown as { instance: null }).instance = null; - } - }); - it('should call storage.clear() to remove all data', async () => { const storage = createMockStorage(); @@ -83,18 +75,28 @@ describe('Sphere.clear()', () => { it('should destroy existing Sphere instance before clearing', async () => { const storage = createMockStorage(); - // Simulate an existing instance whose destroy() resets the singleton + // A live Sphere registered against THIS storage. clear() must tear it down before + // wiping the KV out from under it. Seeded straight into the private per-storage + // registry that replaced the process-global singleton (#766); the mock's destroy() + // deregisters itself the way the real Sphere.destroy() does. + const liveByStorage = (Sphere as unknown as { + _liveByStorage: WeakMap>; + })._liveByStorage; + const registered = new Set(); const mockInstance = { destroy: vi.fn(async () => { - (Sphere as unknown as { instance: null }).instance = null; + registered.delete(mockInstance); }), }; - (Sphere as unknown as { instance: typeof mockInstance }).instance = mockInstance; + registered.add(mockInstance); + liveByStorage.set(storage, registered); await Sphere.clear({ storage }); expect(mockInstance.destroy).toHaveBeenCalled(); - expect(Sphere.getInstance()).toBeNull(); + // ...and it is gone from the registry afterwards — clear() leaves no live Sphere + // holding storage it just emptied. + expect(registered.size).toBe(0); }); it('should connect storage if disconnected before clearing', async () => { diff --git a/tests/unit/core/Sphere.destroy-secrets.test.ts b/tests/unit/core/Sphere.destroy-secrets.test.ts index f3fb0231..5b40dc53 100644 --- a/tests/unit/core/Sphere.destroy-secrets.test.ts +++ b/tests/unit/core/Sphere.destroy-secrets.test.ts @@ -35,6 +35,12 @@ function secrets(sphere: Sphere): SphereSecrets { return sphere as unknown as SphereSecrets; } +/** + * The Sphere the current test built, so afterEach can tear it down. There is no + * process-global instance to look it up from (#766) — hold the reference. + */ +let live: Sphere | null = null; + async function initWallet(password?: string): Promise { const { storage, transport, oracle, walletApi } = makeMockProviders({ walletExists: false }); const { sphere } = await Sphere.init({ @@ -46,26 +52,22 @@ async function initWallet(password?: string): Promise { mnemonic: TEST_MNEMONIC, ...(password ? { password } : {}), }); + live = sphere; return sphere; } -function resetSingleton(): void { - (Sphere as unknown as { instance: Sphere | null }).instance = null; -} - describe('Sphere.destroy() secret hygiene', () => { beforeEach(() => { TokenRegistry.resetInstance(); stubFetch(); - resetSingleton(); + live = null; }); afterEach(async () => { - const live = Sphere.getInstance(); if (live) { try { await live.destroy(); } catch { /* ignore */ } } - resetSingleton(); + live = null; TokenRegistry.destroy(); vi.unstubAllGlobals(); }); @@ -105,15 +107,14 @@ describe('Sphere.encrypt() fails closed', () => { beforeEach(() => { TokenRegistry.resetInstance(); stubFetch(); - resetSingleton(); + live = null; }); afterEach(async () => { - const live = Sphere.getInstance(); if (live) { try { await live.destroy(); } catch { /* ignore */ } } - resetSingleton(); + live = null; TokenRegistry.destroy(); vi.unstubAllGlobals(); }); diff --git a/tests/unit/core/Sphere.network-delegation.test.ts b/tests/unit/core/Sphere.network-delegation.test.ts index 19852ec7..6583be6c 100644 --- a/tests/unit/core/Sphere.network-delegation.test.ts +++ b/tests/unit/core/Sphere.network-delegation.test.ts @@ -48,19 +48,21 @@ function stubFetchRecording(): string[] { } describe('Sphere.init network → TokenRegistry delegation (regression guard)', () => { + // The Sphere the current test built, so afterEach can tear it down. There is no + // process-global instance to look it up from (#766) — hold the reference. + let live: Sphere | null = null; + beforeEach(() => { // Fresh registry singleton per test so a prior test's remoteUrl can't leak. TokenRegistry.resetInstance(); - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } + live = null; }); afterEach(async () => { - if (Sphere.getInstance()) { - try { await Sphere.getInstance()!.destroy(); } catch { /* ignore */ } + if (live) { + try { await live.destroy(); } catch { /* ignore */ } } - (Sphere as unknown as { instance: null }).instance = null; + live = null; TokenRegistry.destroy(); vi.unstubAllGlobals(); }); @@ -77,6 +79,7 @@ describe('Sphere.init network → TokenRegistry delegation (regression guard)', network: TEST_NETWORK, autoGenerate: true, }); + live = sphere; // Sanity: this went through the create branch. expect(created).toBe(true); @@ -104,6 +107,7 @@ describe('Sphere.init network → TokenRegistry delegation (regression guard)', walletApi, network: TEST_NETWORK, }); + live = sphere; // Sanity: this went through the load branch. expect(created).toBe(false); diff --git a/tests/unit/core/Sphere.registerNametag.test.ts b/tests/unit/core/Sphere.registerNametag.test.ts index 56bc712d..c16f7f6f 100644 --- a/tests/unit/core/Sphere.registerNametag.test.ts +++ b/tests/unit/core/Sphere.registerNametag.test.ts @@ -88,14 +88,10 @@ describe('Sphere.registerNametag() — Nostr-binding only (D5, no on-chain mint) beforeEach(() => { cleanTestDir(); - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } storage = new FileStorageProvider({ dataDir: DATA_DIR }); }); afterEach(() => { - (Sphere as unknown as { instance: null }).instance = null; cleanTestDir(); }); diff --git a/tests/unit/core/Sphere.status.test.ts b/tests/unit/core/Sphere.status.test.ts index 88b155dc..f55fe6a7 100644 --- a/tests/unit/core/Sphere.status.test.ts +++ b/tests/unit/core/Sphere.status.test.ts @@ -16,19 +16,20 @@ const TEST_NETWORK = 'testnet2' as const; describe('Sphere Status & Provider Management', () => { let providers: MockProviders; + // The Sphere the current test built, so afterEach can tear it down. There is no + // process-global instance to look it up from (#766) — hold the reference. + let live: Sphere | null = null; beforeEach(() => { - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } + live = null; providers = makeMockProviders(); }); afterEach(async () => { - if (Sphere.getInstance()) { - try { await Sphere.getInstance()!.destroy(); } catch { /* ignore */ } + if (live) { + try { await live.destroy(); } catch { /* ignore */ } } - (Sphere as unknown as { instance: null }).instance = null; + live = null; }); async function initSphere(options?: { price?: { platform: PricePlatform } }) { @@ -49,6 +50,7 @@ describe('Sphere Status & Provider Management', () => { }; } const { sphere } = await Sphere.init(initOpts); + live = sphere; return sphere; } From 654a58e112ad942ef9b5bfd64626bba6d953d15b Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Wed, 2 Sep 2026 22:18:00 +0200 Subject: [PATCH 02/43] fix(storage): merge tracked addresses on write instead of clobbering them (#766) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Each Sphere loads its own snapshot of the tracked-address registry and every persist wrote that snapshot WHOLESALE. Two Spheres over one storage: A switchToAddress(1) leaves disk [0,1]; B switchToAddress(2) leaves disk [0,2] — A's address erased from the record while A's in-memory getActiveAddresses() still reports it. The user's other funded addresses vanish from the UI until rediscovered. Deliberately NOT network-scoped, which is what #766 assumed. The payload is {index, hidden, createdAt, updatedAt} and deriveDirectAddress takes no network, so index n is byte-identical on every network — the content belongs on the shared side of "shared seed, isolated operational state". And this is a lost update, not a scoping problem: it happens with both Spheres on testnet2. Renaming the key would orphan every deployed wallet's address list for no benefit; the pv2g2 precedent does not transfer, because there the old data was genuinely unreadable and a survivor was actively harmful. saveTrackedAddresses is now read-merge-write, serialized per provider instance. Union by index; the greater updatedAt wins `hidden`; createdAt keeps the earlier value. A union is only safe because there is no delete path — verified, `_trackedAddresses.delete` has zero hits; removals go through Sphere.clear(), which drops the key rather than persisting a short list. That reasoning is recorded on the port docstring, since it is the next implementer who would otherwise reintroduce this. setAddressHidden and trackScannedAddresses now stamp updatedAt alongside hidden. Without a real clock the merge degenerates to arbitrary-wins. One correction to the design during implementation: a strict parse that dropped unusable rows broke an existing round-trip pin and would have silently deleted a real address if any stored row were ever odd — worse than the bug being fixed. The parse now repairs instead: only a row with no finite index is dropped, missing timestamps read as 0 so a timeless observation loses every conflict rather than winning on a fabricated one, and unknown fields survive both parse and merge. Test written BEFORE the fix and confirmed red against unfixed code (disk [0,2] where [0,1,2] was expected), then red again twice more when saveTrackedAddresses was reverted to the wholesale write. Mutation probe added and hand-verified KILLED. Known residual, now tracked as #771: two SEPARATE FileStorageProvider objects over one dataDir still clobber — that provider caches the whole KV in memory and rewrites the entire file on every set(), so this affects every key, not just this one. The browser providers read through to the shared store and are fixed cross-instance. --- .../storage/IndexedDBStorageProvider.ts | 39 ++- impl/browser/storage/LocalStorageProvider.ts | 39 ++- impl/nodejs/storage/FileStorageProvider.ts | 39 ++- storage/storage-provider.ts | 27 +- storage/tracked-addresses.ts | 77 +++++ .../tracked-addresses-concurrent.test.ts | 315 ++++++++++++++++++ tests/integration/tracked-addresses.test.ts | 7 - tests/mutation/probes.json | 10 + 8 files changed, 517 insertions(+), 36 deletions(-) create mode 100644 storage/tracked-addresses.ts create mode 100644 tests/integration/tracked-addresses-concurrent.test.ts diff --git a/impl/browser/storage/IndexedDBStorageProvider.ts b/impl/browser/storage/IndexedDBStorageProvider.ts index 2c6643d7..f67bae59 100644 --- a/impl/browser/storage/IndexedDBStorageProvider.ts +++ b/impl/browser/storage/IndexedDBStorageProvider.ts @@ -8,6 +8,11 @@ import { SphereError } from '../../../core/errors'; import type { ProviderStatus, FullIdentity, TrackedAddressEntry } from '../../../types'; import type { StorageProvider } from '../../../storage'; import { STORAGE_KEYS_ADDRESS, STORAGE_KEYS_GLOBAL, isNetworkScopedAddressKey, type NetworkType } from '../../../constants'; +import { + mergeTrackedAddresses, + parseTrackedAddresses, + type TrackedAddressesFile, +} from '../../../storage/tracked-addresses'; // ============================================================================= // Configuration @@ -222,19 +227,35 @@ export class IndexedDBStorageProvider implements StorageProvider { } } + /** Serializes the read-merge-write below, per provider instance. */ + private trackedWrites: Promise = Promise.resolve(); + + /** + * Persist the tracked-address registry by MERGING, never replacing. + * + * Every Sphere over this storage holds its own snapshot and writes it in + * full, so a wholesale write drops the addresses this writer never saw + * (#766 item 5 — a lost update, reproducible on one network). Concurrent + * calls are serialized on `trackedWrites` so a read can never interleave + * with another call's write. + */ async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { - await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify({ version: 1, addresses: entries })); + const run = this.trackedWrites.then(async () => { + const onDisk = parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); + const file: TrackedAddressesFile = { + version: 1, + addresses: mergeTrackedAddresses(onDisk, entries), + }; + await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify(file)); + }); + // The chain tail swallows the rejection so one failed write cannot brick + // every later one; the caller still sees the error by awaiting `run`. + this.trackedWrites = run.then(() => undefined, () => undefined); + await run; } async loadTrackedAddresses(): Promise { - const data = await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES); - if (!data) return []; - try { - const parsed = JSON.parse(data); - return parsed.addresses ?? []; - } catch { - return []; - } + return parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); } // =========================================================================== diff --git a/impl/browser/storage/LocalStorageProvider.ts b/impl/browser/storage/LocalStorageProvider.ts index 243f8790..16fc9df4 100644 --- a/impl/browser/storage/LocalStorageProvider.ts +++ b/impl/browser/storage/LocalStorageProvider.ts @@ -8,6 +8,11 @@ import { SphereError } from '../../../core/errors'; import type { ProviderStatus, FullIdentity, TrackedAddressEntry } from '../../../types'; import type { StorageProvider } from '../../../storage'; import { STORAGE_KEYS_ADDRESS, STORAGE_KEYS_GLOBAL, isNetworkScopedAddressKey, type NetworkType } from '../../../constants'; +import { + mergeTrackedAddresses, + parseTrackedAddresses, + type TrackedAddressesFile, +} from '../../../storage/tracked-addresses'; // ============================================================================= // Configuration @@ -150,19 +155,35 @@ export class LocalStorageProvider implements StorageProvider { } } + /** Serializes the read-merge-write below, per provider instance. */ + private trackedWrites: Promise = Promise.resolve(); + + /** + * Persist the tracked-address registry by MERGING, never replacing. + * + * Every Sphere over this storage holds its own snapshot and writes it in + * full, so a wholesale write drops the addresses this writer never saw + * (#766 item 5 — a lost update, reproducible on one network). Concurrent + * calls are serialized on `trackedWrites` so a read can never interleave + * with another call's write. + */ async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { - await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify({ version: 1, addresses: entries })); + const run = this.trackedWrites.then(async () => { + const onDisk = parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); + const file: TrackedAddressesFile = { + version: 1, + addresses: mergeTrackedAddresses(onDisk, entries), + }; + await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify(file)); + }); + // The chain tail swallows the rejection so one failed write cannot brick + // every later one; the caller still sees the error by awaiting `run`. + this.trackedWrites = run.then(() => undefined, () => undefined); + await run; } async loadTrackedAddresses(): Promise { - const data = await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES); - if (!data) return []; - try { - const parsed = JSON.parse(data); - return parsed.addresses ?? []; - } catch { - return []; - } + return parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); } // =========================================================================== diff --git a/impl/nodejs/storage/FileStorageProvider.ts b/impl/nodejs/storage/FileStorageProvider.ts index 335dabae..e069fec7 100644 --- a/impl/nodejs/storage/FileStorageProvider.ts +++ b/impl/nodejs/storage/FileStorageProvider.ts @@ -8,6 +8,11 @@ import * as path from 'path'; import type { StorageProvider } from '../../../storage'; import type { FullIdentity, ProviderStatus, TrackedAddressEntry } from '../../../types'; import { STORAGE_KEYS_ADDRESS, STORAGE_KEYS_GLOBAL, isNetworkScopedAddressKey, type NetworkType } from '../../../constants'; +import { + mergeTrackedAddresses, + parseTrackedAddresses, + type TrackedAddressesFile, +} from '../../../storage/tracked-addresses'; export interface FileStorageProviderConfig { /** Directory to store wallet data */ @@ -169,19 +174,35 @@ export class FileStorageProvider implements StorageProvider { await this.save(); } + /** Serializes the read-merge-write below, per provider instance. */ + private trackedWrites: Promise = Promise.resolve(); + + /** + * Persist the tracked-address registry by MERGING, never replacing. + * + * Every Sphere over this storage holds its own snapshot and writes it in + * full, so a wholesale write drops the addresses this writer never saw + * (#766 item 5 — a lost update, reproducible on one network). Concurrent + * calls are serialized on `trackedWrites` so a read can never interleave + * with another call's write. + */ async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { - await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify({ version: 1, addresses: entries })); + const run = this.trackedWrites.then(async () => { + const onDisk = parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); + const file: TrackedAddressesFile = { + version: 1, + addresses: mergeTrackedAddresses(onDisk, entries), + }; + await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify(file)); + }); + // The chain tail swallows the rejection so one failed write cannot brick + // every later one; the caller still sees the error by awaiting `run`. + this.trackedWrites = run.then(() => undefined, () => undefined); + await run; } async loadTrackedAddresses(): Promise { - const data = await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES); - if (!data) return []; - try { - const parsed = JSON.parse(data); - return parsed.addresses ?? []; - } catch { - return []; - } + return parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); } /** diff --git a/storage/storage-provider.ts b/storage/storage-provider.ts index 015014f3..a82ce624 100644 --- a/storage/storage-provider.ts +++ b/storage/storage-provider.ts @@ -50,12 +50,35 @@ export interface StorageProvider extends BaseProvider { clear(prefix?: string): Promise; /** - * Save tracked addresses (only user state: index, hidden, timestamps) + * Save tracked addresses (only user state: index, hidden, timestamps). + * + * MUST MERGE, NEVER REPLACE (#766 item 5). `entries` is ONE writer's snapshot, + * not the whole truth: every Sphere sharing this storage keeps its own copy of + * the registry and persists all of it, so writing the argument verbatim is a + * lost update — A activates index 1, B (whose snapshot predates that) activates + * index 2, and B's write erases index 1 while A still reports it. This happens + * on a single network with a single provider; do NOT "fix" it by renaming or + * network-scoping the key. + * + * The contract, implemented by `storage/tracked-addresses.ts` — reuse those + * helpers rather than re-deriving this: + * - read the stored registry, union it with `entries` BY `index`; + * - on a conflicting index, the entry with the greater `updatedAt` supplies + * `hidden`, and `createdAt` keeps the earlier value; + * - serialize concurrent calls on the provider instance, so one call's read + * cannot interleave with another's write; + * - a failed write must not brick later writes, and must still reject to its + * own caller. + * + * A union is safe because there is no delete path: entries are only ever added, + * and wiping the wallet removes the key itself (`Sphere.clear()`). Adding a + * per-entry delete would require revisiting this contract. */ saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise; /** - * Load tracked addresses + * Load tracked addresses. Tolerant: unusable/corrupt storage reads as `[]` + * (see `parseTrackedAddresses` in `storage/tracked-addresses.ts`). */ loadTrackedAddresses(): Promise; } diff --git a/storage/tracked-addresses.ts b/storage/tracked-addresses.ts new file mode 100644 index 00000000..230ac8ac --- /dev/null +++ b/storage/tracked-addresses.ts @@ -0,0 +1,77 @@ +import type { TrackedAddressEntry } from '../types'; + +/** On-disk shape of the global `tracked_addresses` key. */ +export interface TrackedAddressesFile { + version: 1; + addresses: TrackedAddressEntry[]; +} + +function num(value: unknown, fallback: number): number { + return typeof value === 'number' && Number.isFinite(value) ? value : fallback; +} + +/** Repair rather than drop — dropping an odd row deletes one of the user's addresses. + * A missing timestamp reads as 0, so a timeless observation loses every conflict. */ +function toEntry(value: unknown): TrackedAddressEntry | null { + if (typeof value !== 'object' || value === null) return null; + const e = value as Record; + if (typeof e.index !== 'number' || !Number.isFinite(e.index)) return null; + return { + ...e, + index: e.index, + hidden: e.hidden === true, + createdAt: num(e.createdAt, 0), + updatedAt: num(e.updatedAt, 0), + } as TrackedAddressEntry; +} + +/** Tolerant read: unusable JSON and a wrong top-level shape both read as absent. */ +export function parseTrackedAddresses(raw: string | null): TrackedAddressEntry[] { + if (!raw) return []; + let parsed: unknown; + try { + parsed = JSON.parse(raw); + } catch { + return []; + } + if (typeof parsed !== 'object' || parsed === null) return []; + const addresses = (parsed as { addresses?: unknown }).addresses; + if (!Array.isArray(addresses)) return []; + const entries: TrackedAddressEntry[] = []; + for (const row of addresses) { + const entry = toEntry(row); + if (entry) entries.push(entry); + } + return entries; +} + +/** + * Union `incoming` (one writer's snapshot) into `onDisk` by `index`, sorted by index. + * On a conflict the greater `updatedAt` supplies `hidden` (ties keep `incoming`) and the + * earlier `createdAt` survives — safe because nothing removes a single entry; see + * `StorageProvider.saveTrackedAddresses` (#766 item 5). + */ +export function mergeTrackedAddresses( + onDisk: readonly TrackedAddressEntry[], + incoming: readonly TrackedAddressEntry[], +): TrackedAddressEntry[] { + const merged = new Map(); + for (const entry of onDisk) merged.set(entry.index, entry); + + for (const entry of incoming) { + const existing = merged.get(entry.index); + if (!existing) { + merged.set(entry.index, entry); + continue; + } + const winner = entry.updatedAt >= existing.updatedAt ? entry : existing; + merged.set(entry.index, { + ...existing, + ...winner, + index: entry.index, + createdAt: Math.min(existing.createdAt, entry.createdAt), + }); + } + + return Array.from(merged.values()).sort((a, b) => a.index - b.index); +} diff --git a/tests/integration/tracked-addresses-concurrent.test.ts b/tests/integration/tracked-addresses-concurrent.test.ts new file mode 100644 index 00000000..c1aa1368 --- /dev/null +++ b/tests/integration/tracked-addresses-concurrent.test.ts @@ -0,0 +1,315 @@ +/** + * Integration tests for the `tracked_addresses` LOST UPDATE (#766 item 5). + * + * Each Sphere loads its own snapshot of the tracked-address registry into + * `_trackedAddresses`, and every persist used to write that snapshot WHOLESALE. + * Two Spheres over one storage therefore clobber each other: + * + * A.switchToAddress(1) -> disk [0,1] + * B.switchToAddress(2) -> disk [0,2] <- A's entry erased, while A's + * getActiveAddresses() still reports it + * + * This is a lost update, not a network-scoping problem: it reproduces on ONE + * network with ONE storage provider. The fix is read-merge-write inside + * `saveTrackedAddresses`, serialized per provider instance. + */ + +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import * as fs from 'fs'; +import * as path from 'path'; +import { Sphere } from '../../core/Sphere'; +import { STORAGE_KEYS_GLOBAL } from '../../constants'; +import { FileStorageProvider } from '../../impl/nodejs/storage/FileStorageProvider'; +import type { TransportProvider, OracleProvider } from '../../index'; +import type { ProviderStatus, TrackedAddressEntry } from '../../types'; +import { TEST_NETWORK } from '../test-network'; +import { makePv2World } from '../support/pv2-world'; +import { TRUSTBASE_TESTNET2 } from '../../assets/trustbase'; +import { mergeTrackedAddresses, parseTrackedAddresses } from '../../storage/tracked-addresses'; + +// ============================================================================= +// Test directories +// ============================================================================= + +const TEST_DIR = path.join(__dirname, '.test-tracked-addresses-concurrent'); +const DATA_DIR = path.join(TEST_DIR, 'data'); + +// ============================================================================= +// Mock providers (same shape as tests/integration/tracked-addresses.test.ts) +// ============================================================================= + +const nostrRelayNametags = new Map(); + +function createMockTransport(): TransportProvider { + return { + id: 'mock-transport', + name: 'Mock Transport', + type: 'p2p' as const, + description: 'Mock transport', + setIdentity: vi.fn(), + connect: vi.fn().mockResolvedValue(undefined), + disconnect: vi.fn().mockResolvedValue(undefined), + isConnected: vi.fn().mockReturnValue(true), + getStatus: vi.fn().mockReturnValue('connected' as ProviderStatus), + sendMessage: vi.fn().mockResolvedValue('event-id'), + onMessage: vi.fn().mockReturnValue(() => {}), + subscribeToBroadcast: vi.fn().mockReturnValue(() => {}), + publishBroadcast: vi.fn().mockResolvedValue('broadcast-id'), + onEvent: vi.fn().mockReturnValue(() => {}), + resolveNametag: vi.fn((nametag: string) => { + return Promise.resolve(nostrRelayNametags.get(nametag) ?? null); + }), + publishIdentityBinding: vi.fn((chainPubkey: string, _directAddress: string, nametag?: string) => { + if (nametag) { + const existing = nostrRelayNametags.get(nametag); + if (existing && existing !== chainPubkey) return Promise.resolve(false); + nostrRelayNametags.set(nametag, chainPubkey); + } + return Promise.resolve(true); + }), + recoverNametag: vi.fn().mockResolvedValue(null), + } as TransportProvider; +} + +function createMockOracle(): OracleProvider { + return { + id: 'mock-oracle', + name: 'Mock Oracle', + type: 'aggregator' as const, + connect: vi.fn().mockResolvedValue(undefined), + disconnect: vi.fn().mockResolvedValue(undefined), + isConnected: vi.fn().mockReturnValue(true), + getStatus: vi.fn().mockReturnValue('connected' as ProviderStatus), + initialize: vi.fn().mockResolvedValue(undefined), + getTrustBaseJson: () => TRUSTBASE_TESTNET2, + getAggregatorUrl: () => 'https://gateway.testnet2.unicity.network', + getApiKey: () => 'test-key', + } as unknown as OracleProvider; +} + +function cleanTestDir(): void { + if (fs.existsSync(TEST_DIR)) { + fs.rmSync(TEST_DIR, { recursive: true, force: true }); + } +} + +async function readPersistedIndices(storage: FileStorageProvider): Promise { + const raw = await storage.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES); + expect(raw).not.toBeNull(); + const parsed = JSON.parse(raw!) as { version: number; addresses: TrackedAddressEntry[] }; + return parsed.addresses.map((a) => a.index); +} + +// ============================================================================= +// Tests +// ============================================================================= + +describe('Tracked addresses — concurrent Spheres (#766 item 5)', () => { + let storage: FileStorageProvider; + + beforeEach(() => { + cleanTestDir(); + nostrRelayNametags.clear(); + storage = new FileStorageProvider({ dataDir: DATA_DIR }); + }); + + afterEach(() => { + cleanTestDir(); + nostrRelayNametags.clear(); + }); + + it('does not lose A\'s address when B persists its own stale snapshot', async () => { + // --- A creates the wallet on this storage --- + const { sphere: a } = await Sphere.init({ + storage, + transport: createMockTransport(), + oracle: createMockOracle(), + network: TEST_NETWORK, + walletApi: makePv2World().walletApi, + autoGenerate: true, + }); + + // --- B loads the SAME wallet over the SAME storage provider --- + const { sphere: b, created } = await Sphere.init({ + storage, + transport: createMockTransport(), + oracle: createMockOracle(), + network: TEST_NETWORK, + walletApi: makePv2World().walletApi, + }); + expect(created).toBeFalsy(); + + // Both hold the same snapshot: { 0 }. + expect(a.getAllTrackedAddresses().map((x) => x.index)).toEqual([0]); + expect(b.getAllTrackedAddresses().map((x) => x.index)).toEqual([0]); + + // A activates address 1 -> disk should hold [0, 1] + await a.switchToAddress(1); + expect(await readPersistedIndices(storage)).toEqual([0, 1]); + + // B — whose snapshot never saw address 1 — activates address 2. + // A wholesale write erases index 1 here; a merge keeps it. + await b.switchToAddress(2); + + expect(await readPersistedIndices(storage)).toEqual([0, 1, 2]); + + // A's in-memory view is still truthful about index 1. + expect(a.getAllTrackedAddresses().map((x) => x.index)).toEqual([0, 1]); + + await a.destroy(); + await b.destroy(); + + // --- A fresh load sees all three --- + const storage2 = new FileStorageProvider({ dataDir: DATA_DIR }); + const { sphere: reloaded } = await Sphere.init({ + storage: storage2, + transport: createMockTransport(), + oracle: createMockOracle(), + network: TEST_NETWORK, + walletApi: makePv2World().walletApi, + }); + + expect(reloaded.getAllTrackedAddresses().map((x) => x.index)).toEqual([0, 1, 2]); + for (const addr of reloaded.getAllTrackedAddresses()) { + expect(addr.directAddress.startsWith('DIRECT://')).toBe(true); + } + + await reloaded.destroy(); + }); + + it('keeps a hidden flag set by the other Sphere (greater updatedAt wins)', async () => { + const { sphere: a } = await Sphere.init({ + storage, + transport: createMockTransport(), + oracle: createMockOracle(), + network: TEST_NETWORK, + walletApi: makePv2World().walletApi, + autoGenerate: true, + }); + await a.switchToAddress(1); + + const { sphere: b } = await Sphere.init({ + storage, + transport: createMockTransport(), + oracle: createMockOracle(), + network: TEST_NETWORK, + walletApi: makePv2World().walletApi, + }); + expect(b.getAllTrackedAddresses().map((x) => x.index)).toEqual([0, 1]); + + // B hides address 1 (bumping its updatedAt); A then writes its own snapshot, + // which still believes index 1 is visible but carries an OLDER updatedAt. + // Step off A's millisecond first, so the case is about merge policy and not + // about clock granularity producing a tie. + const aUpdatedAt = a.getTrackedAddress(1)!.updatedAt; + while (Date.now() <= aUpdatedAt) { + await new Promise((resolve) => setTimeout(resolve, 1)); + } + await b.setAddressHidden(1, true); + await a.switchToAddress(2); + + const raw = await storage.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES); + const entries = (JSON.parse(raw!) as { addresses: TrackedAddressEntry[] }).addresses; + expect(entries.map((e) => e.index)).toEqual([0, 1, 2]); + expect(entries.find((e) => e.index === 1)!.hidden).toBe(true); + + await a.destroy(); + await b.destroy(); + }); + + it('serializes concurrent saveTrackedAddresses calls on one provider', async () => { + const now = Date.now(); + const mk = (index: number): TrackedAddressEntry => ({ + index, + hidden: false, + createdAt: now, + updatedAt: now, + }); + + // Two writers that each only know about their own index, issued without + // awaiting each other: the per-instance write chain must not interleave a + // read of one with the write of the other. + await Promise.all([ + storage.saveTrackedAddresses([mk(0), mk(1)]), + storage.saveTrackedAddresses([mk(0), mk(2)]), + ]); + + expect((await storage.loadTrackedAddresses()).map((e) => e.index)).toEqual([0, 1, 2]); + }); + + describe('merge semantics (deterministic, no clock)', () => { + it('unions by index, greater updatedAt wins hidden, earlier createdAt is kept', () => { + const merged = mergeTrackedAddresses( + [ + { index: 0, hidden: false, createdAt: 100, updatedAt: 100 }, + { index: 2, hidden: true, createdAt: 300, updatedAt: 900 }, + { index: 5, hidden: false, createdAt: 500, updatedAt: 500 }, + ], + [ + { index: 2, hidden: false, createdAt: 250, updatedAt: 400 }, // stale: hidden loses + { index: 1, hidden: true, createdAt: 200, updatedAt: 200 }, // only this writer knows it + ], + ); + + expect(merged.map((e) => e.index)).toEqual([0, 1, 2, 5]); + // Greater updatedAt supplies hidden... + expect(merged.find((e) => e.index === 2)).toEqual({ + index: 2, + hidden: true, + createdAt: 250, // ...while createdAt keeps the EARLIER value + updatedAt: 900, + }); + // The stale writer's own new entry survives, as does the entry it never saw. + expect(merged.find((e) => e.index === 1)!.hidden).toBe(true); + expect(merged.find((e) => e.index === 5)).toBeDefined(); + }); + + it('lets a fresher incoming entry overwrite hidden', () => { + const merged = mergeTrackedAddresses( + [{ index: 1, hidden: false, createdAt: 10, updatedAt: 10 }], + [{ index: 1, hidden: true, createdAt: 10, updatedAt: 11 }], + ); + expect(merged).toEqual([{ index: 1, hidden: true, createdAt: 10, updatedAt: 11 }]); + }); + + it('parses tolerantly: junk and a wrong top-level shape read as empty', () => { + expect(parseTrackedAddresses(null)).toEqual([]); + expect(parseTrackedAddresses('')).toEqual([]); + expect(parseTrackedAddresses('not json')).toEqual([]); + expect(parseTrackedAddresses('{"version":1}')).toEqual([]); + expect(parseTrackedAddresses('[1,2,3]')).toEqual([]); + }); + + it('repairs an odd row instead of dropping the address it names', () => { + // Dropping a row would delete one of the user's addresses; only a row with no + // usable index is unrecoverable. A repaired timestamp is 0, so such an entry + // loses every conflict rather than winning one on a fabricated time. + const parsed = parseTrackedAddresses( + JSON.stringify({ + version: 1, + addresses: [ + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 1, hidden: 'nope' }, + { hidden: true, createdAt: 1, updatedAt: 1 }, + null, + ], + }), + ); + + expect(parsed).toEqual([ + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 1, hidden: false, createdAt: 0, updatedAt: 0 }, + ]); + + const merged = mergeTrackedAddresses(parsed, [ + { index: 1, hidden: true, createdAt: 5, updatedAt: 5 }, + ]); + expect(merged.find((e) => e.index === 1)).toEqual({ + index: 1, + hidden: true, + createdAt: 0, + updatedAt: 5, + }); + }); + }); +}); diff --git a/tests/integration/tracked-addresses.test.ts b/tests/integration/tracked-addresses.test.ts index ed72a5dd..162a41e7 100644 --- a/tests/integration/tracked-addresses.test.ts +++ b/tests/integration/tracked-addresses.test.ts @@ -109,14 +109,10 @@ describe('Tracked addresses integration', () => { beforeEach(() => { cleanTestDir(); clearNostrRelay(); - if (Sphere.getInstance()) { - (Sphere as unknown as { instance: null }).instance = null; - } storage = new FileStorageProvider({ dataDir: DATA_DIR }); }); afterEach(() => { - (Sphere as unknown as { instance: null }).instance = null; cleanTestDir(); clearNostrRelay(); }); @@ -346,7 +342,6 @@ describe('Tracked addresses integration', () => { await sphere.destroy(); // --- Reload wallet from same storage --- - (Sphere as unknown as { instance: null }).instance = null; const storage2 = new FileStorageProvider({ dataDir: DATA_DIR }); const transport2 = createMockTransport(); const oracle2 = createMockOracle(); @@ -503,7 +498,6 @@ describe('Tracked addresses integration', () => { expect(await storage.get(STORAGE_KEYS_GLOBAL.ADDRESS_NAMETAGS)).not.toBeNull(); await sphere.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // Clear wallet await Sphere.clear({ storage }); @@ -539,7 +533,6 @@ describe('Tracked addresses integration', () => { const firstAddresses = first.getAllTrackedAddresses(); await first.destroy(); - (Sphere as unknown as { instance: null }).instance = null; // Clear await Sphere.clear({ storage }); diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 9366b860..cec1ab10 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -600,5 +600,15 @@ "tests": [ "tests/integration/sphere-payments-v2-wiring.test.ts" ] + }, + { + "name": "tracked-addresses-merge-drop", + "note": "#766 item 5: saveTrackedAddresses MERGES the on-disk registry with the writer's snapshot; writing the snapshot verbatim is the lost update that erases another Sphere's address (tracked-addresses-concurrent.test.ts)", + "file": "impl/nodejs/storage/FileStorageProvider.ts", + "find": " addresses: mergeTrackedAddresses(onDisk, entries),", + "replace": " addresses: (onDisk, entries),", + "tests": [ + "tests/integration/tracked-addresses-concurrent.test.ts" + ] } ] From c1b05e0f8592f9a5888cef303bb328446243cf5a Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Wed, 2 Sep 2026 23:20:28 +0200 Subject: [PATCH 03/43] fix(storage): reject fractional/negative tracked-address indices; close the coverage gaps Copilot found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot reviewed #772 and found one correctness bug, one stale doc, and five places where the guard could be deleted with the suite still green. All valid. CORRECTNESS — an unusable index aliased a real address parseTrackedAddresses accepted any FINITE index, so a stored row with index 1.5 passed. deriveKeyAtPath parseInt()s the path segment (core/crypto.ts:232), so that row derives index 1's keys — a malformed row silently impersonating a real address. Negatives reached derivation too. Now a non-negative integer is required. This inverts the repair-over-drop rule the merge otherwise follows, deliberately: dropping a row normally deletes one of the user's addresses, but an unusable index is not a recoverable address, and keeping it aliases one that is. The rationale lives on the port docstring, where the next implementer looks. DOCS — docs/MIGRATION-TOKEN-REGISTRY.md still told consumers Sphere.getInstance() and isInitialized() exist and that clear()/import() are storage-blind. I wrote that in #767 and never revisited it when this PR removed them. Now points at #771 for what genuinely remains. COVERAGE — five guards that could be deleted with the suite green This is the class that has repeatedly bitten this branch, and Copilot caught it by auditing whether the TESTS constrain the code rather than whether the code is correct: - Either `verification: options.verification` forwarding line could be deleted; the existing worker tests build the engine directly and never go through Sphere.init. - _liveByStorage holds a Set per storage, but every test had one Sphere per storage, so an implementation tracking only one instance passed. - importFromLegacyFile/importFromJSON had no test at all — a refactor could discard result.sphere again, reinstating the wrong-object bug this PR fixes. - resetInstance()'s new teardown had no static-path regression. - The merge contract was tested against FileStorageProvider only; the two browser providers could satisfy the interface while still replacing snapshots. Closed with 26 new tests including a shared provider contract suite run against all three implementations, following the tests/unit/payments-v2/contracts/ idiom. Probes 61 -> 71. Every new test was falsified individually, and the falsifications discriminate: deleting the load-branch forwarding reds only the load test, the create-branch only the create test; each provider's merge mutation reds only that provider's four cases. For resetInstance the interval assertion alone does NOT red — only the in-flight-abort assertion does, which is exactly the point, since the old code cleared the timer and did nothing else. One vacuity caught in passing: a stored NaN cannot round-trip (JSON.stringify writes null), so a NaN-only assertion would have passed either way. Folded into a rejection-set case carrying 1.5, -1, Infinity, a string and a missing index, so every test in that file reds under the revert. --- core/Sphere.ts | 3 +- docs/MIGRATION-TOKEN-REGISTRY.md | 15 +- storage/storage-provider.ts | 4 + storage/tracked-addresses.ts | 6 +- .../sphere-instance-scoping.test.ts | 157 ++++++++++++++++++ tests/mutation/probes.json | 100 +++++++++++ .../core/Sphere.init-verification.test.ts | 117 +++++++++++++ .../registry/TokenRegistry.instances.test.ts | 71 ++++++++ .../contracts/tracked-addresses.contract.ts | 99 +++++++++++ .../tracked-addresses-providers.test.ts | 56 +++++++ tests/unit/storage/tracked-addresses.test.ts | 93 +++++++++++ 11 files changed, 711 insertions(+), 10 deletions(-) create mode 100644 tests/unit/core/Sphere.init-verification.test.ts create mode 100644 tests/unit/storage/contracts/tracked-addresses.contract.ts create mode 100644 tests/unit/storage/tracked-addresses-providers.test.ts create mode 100644 tests/unit/storage/tracked-addresses.test.ts diff --git a/core/Sphere.ts b/core/Sphere.ts index 3c26b943..75457f8f 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -1981,7 +1981,8 @@ export class Sphere { /** * Import wallet from JSON backup * - * @returns Object with success status and optionally recovered mnemonic + * @returns `{ success, sphere?, mnemonic?, error? }`. `sphere` is the instance built + * on the SUPPLIED storage — hold it, there is no global to look it up from (#766). * * @example * ```ts diff --git a/docs/MIGRATION-TOKEN-REGISTRY.md b/docs/MIGRATION-TOKEN-REGISTRY.md index 19eefdf1..ab6334ea 100644 --- a/docs/MIGRATION-TOKEN-REGISTRY.md +++ b/docs/MIGRATION-TOKEN-REGISTRY.md @@ -92,9 +92,12 @@ rather than a large one. ## Not fixed by this release -`Sphere` still holds a process-global `static instance`, so `Sphere.getInstance()` and -`isInitialized()` describe whichever Sphere was created last, and `Sphere.clear()` / -`Sphere.import()` destroy whichever instance holds that static **regardless of which storage -they were given**. Two `FileStorageProvider`s pointed at one `dataDir` also clobber each -other's wallet file. Those are tracked in [#766](https://github.com/unicity-sphere/sphere-sdk/issues/766) -and are the remainder of "create new ones at the same time". +`Sphere.getInstance()`, `Sphere.isInitialized()` and the `getSphere` export are **removed** +(see [#766](https://github.com/unicity-sphere/sphere-sdk/issues/766)); hold the instance the +entry point returns, and use `sphere.isReady`. `Sphere.clear()` / `Sphere.import()` now +destroy only Spheres built on the storage they are given. + +What remains: two `FileStorageProvider` objects pointed at one `dataDir` still clobber each +other's wallet file — that provider caches the whole store in memory and rewrites the entire +file on every `set()`, so it affects every key, not just one. Tracked as +[#771](https://github.com/unicity-sphere/sphere-sdk/issues/771). diff --git a/storage/storage-provider.ts b/storage/storage-provider.ts index a82ce624..d093f5db 100644 --- a/storage/storage-provider.ts +++ b/storage/storage-provider.ts @@ -70,6 +70,10 @@ export interface StorageProvider extends BaseProvider { * - a failed write must not brick later writes, and must still reject to its * own caller. * + * A stored `index` must be a NON-NEGATIVE INTEGER. `deriveKeyAtPath` parseInt()s + * that path segment, so `1.5` derives index 1's keys and the row aliases a real + * address; such rows are dropped on read rather than repaired. + * * A union is safe because there is no delete path: entries are only ever added, * and wiping the wallet removes the key itself (`Sphere.clear()`). Adding a * per-entry delete would require revisiting this contract. diff --git a/storage/tracked-addresses.ts b/storage/tracked-addresses.ts index 230ac8ac..e20af5bf 100644 --- a/storage/tracked-addresses.ts +++ b/storage/tracked-addresses.ts @@ -10,12 +10,12 @@ function num(value: unknown, fallback: number): number { return typeof value === 'number' && Number.isFinite(value) ? value : fallback; } -/** Repair rather than drop — dropping an odd row deletes one of the user's addresses. - * A missing timestamp reads as 0, so a timeless observation loses every conflict. */ +/** Repair, don't drop — except the index: deriveKeyAtPath parseInt()s it, so 1.5 + * would alias index 1's real address (see the port docstring). */ function toEntry(value: unknown): TrackedAddressEntry | null { if (typeof value !== 'object' || value === null) return null; const e = value as Record; - if (typeof e.index !== 'number' || !Number.isFinite(e.index)) return null; + if (typeof e.index !== 'number' || !Number.isInteger(e.index) || e.index < 0) return null; return { ...e, index: e.index, diff --git a/tests/integration/sphere-instance-scoping.test.ts b/tests/integration/sphere-instance-scoping.test.ts index 64effeec..1e11c292 100644 --- a/tests/integration/sphere-instance-scoping.test.ts +++ b/tests/integration/sphere-instance-scoping.test.ts @@ -69,6 +69,8 @@ interface Wallet { } const wallets: Wallet[] = []; +/** Spheres a test built outside `makeWallet`'s one-per-wallet slot. */ +const extraSpheres: Sphere[] = []; function makeWallet(label: string): Wallet { const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), `sphere-scope-${label}-`)); @@ -112,6 +114,9 @@ describe('Sphere lifecycle statics are scoped to the storage they are handed (#7 }); afterEach(async () => { + for (const sphere of extraSpheres.splice(0)) { + try { await sphere.destroy(); } catch { /* already torn down by the test */ } + } for (const wallet of wallets.splice(0)) { try { await wallet.sphere?.destroy(); @@ -210,4 +215,156 @@ describe('Sphere lifecycle statics are scoped to the storage they are handed (#7 expect(b.sphere!.isReady).toBe(true); expect(() => b.sphere!.payments).not.toThrow(); }); + + it('clear() destroys EVERY Sphere on that storage, not merely one of them', async () => { + // The registry maps a provider to a SET, because more than one Sphere can be built + // over one provider — the second one LOADS the wallet the first created. Tracking a + // single instance per storage would leave the other running over an emptied KV: the + // exact silent-death the scoping fix exists to prevent, just one level in. + const a = makeWallet('multi'); + const first = await initWallet(a, MNEMONIC_A); + + const { sphere: second, created } = await Sphere.init({ + storage: a.storage, + transport: createMockTransport(), + oracle: a.oracle, + walletApi: makePv2World(NET).walletApi, + network: NET, + }); + extraSpheres.push(second); + expect(created, 'the second init must LOAD the same wallet, not create another').toBe(false); + expect(second).not.toBe(first); + expect(first.isReady).toBe(true); + expect(second.isReady).toBe(true); + + await Sphere.clear({ storage: a.storage }); + + expect(first.isReady).toBe(false); + expect(second.isReady).toBe(false); + expect(() => first.payments).toThrow(); + expect(() => second.payments).toThrow(); + }); +}); + +/** + * #766: `importFromLegacyFile` / `importFromJSON` return the Sphere they built. + * + * importFromJSON used to DISCARD it, so importFromLegacyFile reached for the + * process-global instead — which held whichever Sphere was constructed last, not the + * one imported into the storage the caller supplied. Threading the instance out is + * what removed the global's last reader, and nothing failed when it was dropped: both + * call sites returned a `success: true` result either way. + */ +describe('the legacy-import entry points return the Sphere on the SUPPLIED storage (#766)', () => { + beforeEach(() => { + vi.stubGlobal( + 'fetch', + vi.fn(async () => ({ + ok: true, + status: 200, + statusText: 'OK', + json: async () => [], + text: async () => '[]', + } as unknown as Response)), + ); + }); + + afterEach(async () => { + for (const sphere of extraSpheres.splice(0)) { + try { await sphere.destroy(); } catch { /* already torn down by the test */ } + } + for (const wallet of wallets.splice(0)) { + try { await wallet.sphere?.destroy(); } catch { /* already torn down by the test */ } + fs.rmSync(wallet.dataDir, { recursive: true, force: true }); + } + vi.unstubAllGlobals(); + }); + + /** A real `sphere-wallet` backup of MNEMONIC_C, plus the identity it must restore. */ + async function backupOfWalletC(): Promise<{ chainPubkey: string; withMnemonic: string; masterKeyOnly: string }> { + const c = makeWallet('c'); + const sphereC = await initWallet(c, MNEMONIC_C); + const chainPubkey = sphereC.identity!.chainPubkey; + const withMnemonic = JSON.stringify(sphereC.exportToJSON()); + const masterKeyOnly = JSON.stringify(sphereC.exportToJSON({ includeMnemonic: false })); + await sphereC.destroy(); + c.sphere = undefined; + return { chainPubkey, withMnemonic, masterKeyOnly }; + } + + it('importFromLegacyFile threads the imported Sphere out, past a live one elsewhere', async () => { + const backup = await backupOfWalletC(); + + const b = makeWallet('b'); + // A is built LAST, so it is exactly the instance the deleted process-global held. + const a = makeWallet('a'); + const sphereA = await initWallet(a, MNEMONIC_A); + + const result = await Sphere.importFromLegacyFile({ + fileContent: backup.withMnemonic, + fileName: 'sphere-wallet-backup.json', + storage: b.storage, + transport: b.transport, + oracle: b.oracle, + walletApi: b.world.walletApi, + network: NET, + }); + + expect(result.success).toBe(true); + expect(result.sphere, 'the imported Sphere must reach the caller').toBeDefined(); + b.sphere = result.sphere; + + // It is the wallet that was imported, not the other live one. + expect(result.sphere).not.toBe(sphereA); + expect(result.sphere!.identity!.chainPubkey).toBe(backup.chainPubkey); + expect(result.sphere!.identity!.chainPubkey).not.toBe(sphereA.identity!.chainPubkey); + + // ...and it is bound to the storage the CALLER supplied: clearing B tears it down + // (which an instance belonging to A's storage would survive), and A is untouched. + await Sphere.clear({ storage: b.storage }); + expect(result.sphere!.isReady).toBe(false); + expect(sphereA.isReady).toBe(true); + }); + + it('importFromJSON returns the Sphere for the mnemonic branch', async () => { + const backup = await backupOfWalletC(); + const b = makeWallet('b'); + + const result = await Sphere.importFromJSON({ + jsonContent: backup.withMnemonic, + storage: b.storage, + transport: b.transport, + oracle: b.oracle, + walletApi: b.world.walletApi, + network: NET, + }); + + expect(result.success).toBe(true); + expect(result.mnemonic).toBe(MNEMONIC_C); + expect(result.sphere).toBeDefined(); + b.sphere = result.sphere; + expect(result.sphere!.identity!.chainPubkey).toBe(backup.chainPubkey); + }); + + it('importFromJSON returns the Sphere for the master-key branch too', async () => { + // A backup with no mnemonic takes the OTHER return site — a second place the + // instance can be dropped, with the mnemonic branch still green. + const backup = await backupOfWalletC(); + expect(JSON.parse(backup.masterKeyOnly).mnemonic).toBeUndefined(); + const b = makeWallet('b'); + + const result = await Sphere.importFromJSON({ + jsonContent: backup.masterKeyOnly, + storage: b.storage, + transport: b.transport, + oracle: b.oracle, + walletApi: b.world.walletApi, + network: NET, + }); + + expect(result.success).toBe(true); + expect(result.sphere).toBeDefined(); + b.sphere = result.sphere; + expect(result.sphere!.identity!.chainPubkey).toBe(backup.chainPubkey); + }); }); diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index cec1ab10..9d8ac112 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -610,5 +610,105 @@ "tests": [ "tests/integration/tracked-addresses-concurrent.test.ts" ] + }, + { + "name": "init-verification-not-forwarded-to-load", + "note": "#769.2: Sphere.init() forwards `verification` to load() - dropped, a consumer opting into the worker pool at the documented entry point silently gets the sequential verifier", + "file": "core/Sphere.ts", + "find": "\n verification: options.verification,", + "replace": "\n // mutant: verification dropped on the load branch", + "tests": [ + "tests/unit/core/Sphere.init-verification.test.ts" + ] + }, + { + "name": "init-verification-not-forwarded-to-create", + "note": "#769.2: the SAME forwarding on the create branch - each call site lists its options by hand, so one can be dropped with the other intact", + "file": "core/Sphere.ts", + "find": "\n verification: options.verification,", + "replace": "\n // mutant: verification dropped on the create branch", + "tests": [ + "tests/unit/core/Sphere.init-verification.test.ts" + ] + }, + { + "name": "live-registry-tracks-one-sphere-per-storage", + "note": "#766: _liveByStorage maps a provider to a SET - two Spheres can share one provider (the second LOADS the wallet the first created), and clear() must destroy both or one keeps running over an emptied KV", + "file": "core/Sphere.ts", + "find": " let live = Sphere._liveByStorage.get(sphere._storage);\n if (!live) {\n live = new Set();\n Sphere._liveByStorage.set(sphere._storage, live);\n }\n live.add(sphere);", + "replace": " Sphere._liveByStorage.set(sphere._storage, new Set([sphere]));", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "importfromjson-discards-imported-sphere", + "note": "#766: importFromJSON must RETURN the Sphere it built - discarding it is what made importFromLegacyFile reach for the process-global and hand back the wrong instance", + "file": "core/Sphere.ts", + "find": " // Import using mnemonic if available (preferred)\n if (mnemonic) {\n const sphere = await Sphere.import({ ...baseOptions, mnemonic, basePath });\n return { success: true, sphere, mnemonic };", + "replace": " // Import using mnemonic if available (preferred)\n if (mnemonic) {\n await Sphere.import({ ...baseOptions, mnemonic, basePath });\n return { success: true, mnemonic };", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "importfromlegacyfile-drops-threaded-sphere", + "note": "#766: the sphere-wallet JSON branch threads importFromJSON's instance out to the caller - dropping it returns success with no Sphere at all", + "file": "core/Sphere.ts", + "find": " return { success: true, sphere: result.sphere, mnemonic: result.mnemonic };", + "replace": " return { success: true, mnemonic: result.mnemonic };", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "registry-resetinstance-skips-dispose", + "note": "#770.5: resetInstance() must DISPOSE the outgoing singleton, not merely stop its timer - stopAutoRefresh leaves `disposed` unset, the generation unchanged and the in-flight fetch (plus its 10s abort timer) running on an instance getInstance() can no longer return", + "file": "registry/TokenRegistry.ts", + "find": " TokenRegistry.instance?.dispose();", + "replace": " TokenRegistry.instance?.stopAutoRefresh();", + "tests": [ + "tests/unit/registry/TokenRegistry.instances.test.ts" + ] + }, + { + "name": "tracked-addresses-merge-drop-idb", + "note": "#766 item 5: the browser IDB provider keeps its OWN copy of the read-merge-write; a wholesale write here is the same lost update the Node provider's probe guards", + "file": "impl/browser/storage/IndexedDBStorageProvider.ts", + "find": " addresses: mergeTrackedAddresses(onDisk, entries),", + "replace": " addresses: entries,", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, + { + "name": "tracked-addresses-merge-drop-localstorage", + "note": "#766 item 5: and again in LocalStorageProvider - three implementations, three chances to regress to a wholesale write", + "file": "impl/browser/storage/LocalStorageProvider.ts", + "find": " addresses: mergeTrackedAddresses(onDisk, entries),", + "replace": " addresses: entries,", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, + { + "name": "tracked-addresses-failed-write-bricks-chain", + "note": "#766 item 5: the serializing chain must swallow a rejection - carried forward, one transient write error freezes the registry for the life of the provider while every later caller still sees the ORIGINAL error", + "file": "impl/nodejs/storage/FileStorageProvider.ts", + "find": " this.trackedWrites = run.then(() => undefined, () => undefined);", + "replace": " this.trackedWrites = run;", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, + { + "name": "tracked-address-index-aliases-real-address", + "note": "#766 review: the stored index must be a non-negative INTEGER - deriveKeyAtPath parseInt()s the path segment, so a 1.5 row derives index 1's keys and aliases a real address; Number.isFinite let 1.5 and negatives through", + "file": "storage/tracked-addresses.ts", + "find": " if (typeof e.index !== 'number' || !Number.isInteger(e.index) || e.index < 0) return null;", + "replace": " if (typeof e.index !== 'number' || !Number.isFinite(e.index)) return null;", + "tests": [ + "tests/unit/storage/tracked-addresses.test.ts" + ] } ] diff --git a/tests/unit/core/Sphere.init-verification.test.ts b/tests/unit/core/Sphere.init-verification.test.ts new file mode 100644 index 00000000..f7315d72 --- /dev/null +++ b/tests/unit/core/Sphere.init-verification.test.ts @@ -0,0 +1,117 @@ +/** + * `Sphere.init({ verification })` reaches the Sphere-OWNED engine (#769.2). + * + * init() does not build the engine itself — it dispatches to load() or create() + * depending on whether a wallet already exists, and each call site lists the + * options it forwards by hand. The option was silently dropped there, so a + * consumer opting into the worker pool at the documented entry point got the + * sequential verifier with no error and no log. + * + * The existing worker-pool suite (tests/unit/token-engine/worker-verification.test.ts) + * builds the engine DIRECTLY, so it cannot see a forwarding hole: both branches + * are pinned here instead, because forgetting one is exactly the shape of the bug. + * + * The discriminator is which verifier the engine ends up holding. A probe token is + * not a real SDK token, so the pool verifier rejects while `Token.verify` (the + * sequential path, which the probe stubs) resolves — the control cases show the + * probe really does answer differently either way, so a rejection here means the + * pool, not an unrelated failure. + */ + +import { afterEach, describe, expect, it, vi } from 'vitest'; + +import { Sphere, type SphereInitOptions } from '../../../core/Sphere'; +import type { OracleProvider } from '../../../oracle'; +import type { TransportProvider } from '../../../transport'; +import type { ITokenEngine, SphereToken, VerificationWorker } from '../../../token-engine'; +import { VerificationStatus } from '../../../token-engine/sdk'; +import { TEST_NETWORK } from '../../test-network'; +import { makeMockProviders } from './support/mock-providers'; + +/** Never actually spawned: the pool rejects a probe token before it acquires a worker. */ +class NoopWorker implements VerificationWorker { + onerror: ((event: { message: string }) => void) | null = null; + onmessage: ((event: { data: unknown }) => void) | null = null; + postMessage(): void {} + terminate(): void {} +} + +/** The engine Sphere built for the active address — the thing money operations use. */ +function engineOf(sphere: Sphere): ITokenEngine { + const engine = (sphere as unknown as { _tokenEngine?: ITokenEngine })._tokenEngine; + expect(engine, 'Sphere built no token engine — the test would be vacuous').toBeDefined(); + return engine as ITokenEngine; +} + +/** A token whose only job is to record whether the SEQUENTIAL path was taken. */ +function probeToken(): { token: SphereToken; verify: ReturnType } { + const verify = vi.fn().mockResolvedValue({ status: VerificationStatus.OK }); + return { token: { sdkToken: { verify } } as unknown as SphereToken, verify }; +} + +describe('Sphere.init forwards `verification` to the engine it builds (#769.2)', () => { + let live: Sphere | null = null; + + afterEach(async () => { + if (live) { + try { await live.destroy(); } catch { /* already torn down */ } + } + live = null; + }); + + async function initWith( + walletExists: boolean, + verification?: SphereInitOptions['verification'], + ): Promise<{ sphere: Sphere; created: boolean }> { + const providers = makeMockProviders({ walletExists }); + const { sphere, created } = await Sphere.init({ + storage: providers.storage, + transport: providers.transport as unknown as TransportProvider, + oracle: providers.oracle as unknown as OracleProvider, + walletApi: providers.walletApi, + network: TEST_NETWORK, + autoGenerate: true, + ...(verification ? { verification } : {}), + }); + live = sphere; + return { sphere, created }; + } + + it('create branch (no wallet yet): the configured pool verifier owns verify()', async () => { + const createWorker = vi.fn(() => new NoopWorker()); + const { sphere, created } = await initWith(false, { createWorker }); + expect(created, 'this test must exercise create(), not load()').toBe(true); + + const { token, verify } = probeToken(); + await expect(engineOf(sphere).verify(token)).rejects.toBeDefined(); + expect(verify).not.toHaveBeenCalled(); + }); + + it('load branch (wallet already exists): the configured pool verifier owns verify()', async () => { + const createWorker = vi.fn(() => new NoopWorker()); + const { sphere, created } = await initWith(true, { createWorker }); + expect(created, 'this test must exercise load(), not create()').toBe(false); + + const { token, verify } = probeToken(); + await expect(engineOf(sphere).verify(token)).rejects.toBeDefined(); + expect(verify).not.toHaveBeenCalled(); + }); + + it('control — create branch without the option keeps the sequential verifier', async () => { + const { sphere, created } = await initWith(false); + expect(created).toBe(true); + + const { token, verify } = probeToken(); + await expect(engineOf(sphere).verify(token)).resolves.toEqual({ ok: true }); + expect(verify).toHaveBeenCalledTimes(1); + }); + + it('control — load branch without the option keeps the sequential verifier', async () => { + const { sphere, created } = await initWith(true); + expect(created).toBe(false); + + const { token, verify } = probeToken(); + await expect(engineOf(sphere).verify(token)).resolves.toEqual({ ok: true }); + expect(verify).toHaveBeenCalledTimes(1); + }); +}); diff --git a/tests/unit/registry/TokenRegistry.instances.test.ts b/tests/unit/registry/TokenRegistry.instances.test.ts index a28a3e23..5c43f9f9 100644 --- a/tests/unit/registry/TokenRegistry.instances.test.ts +++ b/tests/unit/registry/TokenRegistry.instances.test.ts @@ -286,3 +286,74 @@ describe('TokenRegistry#dispose', () => { } }); }); + +/** + * The STATIC teardown path. `resetInstance()` used to call only `stopAutoRefresh()`, + * which clears the interval but leaves `disposed` unset, the generation unchanged and + * the request already in the air still running — so a load past its entry guard re-armed + * the interval on an instance `getInstance()` could no longer return, and the fetch plus + * its 10s abort timer went on holding Node's event loop open. Clearing the timer is the + * half the old code got right, so the timer assertion alone cannot see the defect: the + * abort, the disposed flag and the refusal to re-arm are what separate the two. + */ +describe('TokenRegistry.resetInstance', () => { + it('disposes the outgoing singleton: interval cleared, in-flight fetch aborted, no re-arm', async () => { + vi.useFakeTimers(); + const { storage } = makeStorage(); + // The first fetch answers at once (so the interval really gets armed); every later + // one hangs until aborted, so a request is genuinely in the air at teardown. + const calls: string[] = []; + let aborted = 0; + const original = globalThis.fetch; + globalThis.fetch = ((input: unknown, init?: { signal?: AbortSignal }) => { + calls.push(String(input)); + if (calls.length === 1) { + return Promise.resolve(new Response(JSON.stringify(defsA), { status: 200 })); + } + return new Promise((_resolve, reject) => { + init?.signal?.addEventListener('abort', () => { + aborted++; + reject(new Error('aborted')); + }); + }); + }) as typeof globalThis.fetch; + + try { + TokenRegistry.configure({ remoteUrl: URL_A, storage, autoRefresh: true, refreshIntervalMs: 1000 }); + const registry = TokenRegistry.getInstance(); + await vi.advanceTimersByTimeAsync(0); + expect(hasLiveInterval(registry)).toBe(true); + + // One interval tick later a second fetch is in the air and never settles. + await vi.advanceTimersByTimeAsync(1000); + expect(calls.length).toBe(2); + const timersBefore = vi.getTimerCount(); + expect(timersBefore).toBeGreaterThan(0); + + TokenRegistry.resetInstance(); + + expect(registry.isDisposed).toBe(true); + expect(hasLiveInterval(registry)).toBe(false); + expect(aborted).toBe(1); + expect(vi.getTimerCount()).toBeLessThan(timersBefore); + + // Several intervals on, the discarded instance issues nothing. + await vi.advanceTimersByTimeAsync(5000); + expect(calls.length).toBe(2); + + // ...and nothing can re-arm it — the caller that still holds this reference is the + // one getInstance() can no longer hand back, so its timer would be unstoppable. + registry.startAutoRefresh(1000); + expect(hasLiveInterval(registry)).toBe(false); + expect(await registry.refreshFromRemote()).toBe(false); + expect(calls.length).toBe(2); + + // Sanity: the singleton really was replaced, so this is not "reset did nothing". + const next = TokenRegistry.getInstance(); + expect(next).not.toBe(registry); + expect(next.isDisposed).toBe(false); + } finally { + globalThis.fetch = original; + } + }); +}); diff --git a/tests/unit/storage/contracts/tracked-addresses.contract.ts b/tests/unit/storage/contracts/tracked-addresses.contract.ts new file mode 100644 index 00000000..0e7ebd89 --- /dev/null +++ b/tests/unit/storage/contracts/tracked-addresses.contract.ts @@ -0,0 +1,99 @@ +/** + * The `StorageProvider.saveTrackedAddresses` contract (#766 item 5). + * + * A wholesale write is a LOST UPDATE: every Sphere over one storage holds its own + * snapshot of the tracked-address registry and persists all of it, so a writer whose + * snapshot predates another's activation erases that address while the other Sphere + * still reports it. The port docstring states the rule; every implementation must obey + * it, and each keeps its own copy of the read-merge-write, so proving it for one + * provider proves nothing about the other two. + * + * Run this against a provider with `describeTrackedAddressesContract` — see + * tests/unit/storage/tracked-addresses-providers.test.ts. + */ +import { describe, expect, it, vi } from 'vitest'; + +import type { StorageProvider } from '../../../../storage'; +import type { TrackedAddressEntry } from '../../../../types'; + +export interface TrackedAddressesHarness { + provider: StorageProvider; + /** Release the backing store (temp dir, IDB connection, …). */ + cleanup?: () => Promise | void; +} + +function entry(index: number, over: Partial = {}): TrackedAddressEntry { + return { index, hidden: false, createdAt: 1_000, updatedAt: 1_000, ...over }; +} + +const indices = (entries: readonly TrackedAddressEntry[]): number[] => entries.map((e) => e.index); + +export function describeTrackedAddressesContract( + name: string, + makeHarness: () => TrackedAddressesHarness | Promise +): void { + describe(`saveTrackedAddresses contract: ${name}`, () => { + async function withProvider( + body: (provider: StorageProvider) => Promise + ): Promise { + const harness = await makeHarness(); + try { + await body(harness.provider); + } finally { + await harness.cleanup?.(); + } + } + + it('merges the writer snapshot into the stored registry instead of replacing it', async () => { + await withProvider(async (provider) => { + await provider.saveTrackedAddresses([entry(0), entry(1)]); + // A second Sphere's snapshot, which never saw index 1. Replacing loses it. + await provider.saveTrackedAddresses([entry(0), entry(2)]); + + expect(indices(await provider.loadTrackedAddresses())).toEqual([0, 1, 2]); + }); + }); + + it('resolves a conflicting index by the greater updatedAt, keeping the earlier createdAt', async () => { + await withProvider(async (provider) => { + await provider.saveTrackedAddresses([entry(1, { hidden: true, createdAt: 300, updatedAt: 900 })]); + // Stale writer: it still believes index 1 is visible, on an older timestamp. + await provider.saveTrackedAddresses([entry(1, { hidden: false, createdAt: 250, updatedAt: 400 })]); + + expect(await provider.loadTrackedAddresses()).toEqual([ + { index: 1, hidden: true, createdAt: 250, updatedAt: 900 }, + ]); + }); + }); + + it('serializes concurrent calls, so one call read cannot interleave with another write', async () => { + await withProvider(async (provider) => { + await Promise.all([ + provider.saveTrackedAddresses([entry(0), entry(1)]), + provider.saveTrackedAddresses([entry(0), entry(2)]), + provider.saveTrackedAddresses([entry(0), entry(3)]), + ]); + + expect(indices(await provider.loadTrackedAddresses())).toEqual([0, 1, 2, 3]); + }); + }); + + it('a failed write rejects to its own caller and does not brick later writes', async () => { + await withProvider(async (provider) => { + await provider.saveTrackedAddresses([entry(0), entry(1)]); + + // The serializing chain must not carry the rejection forward: every later write + // would then reject without ever running, so one transient disk/IDB error would + // freeze the registry for the life of the provider. + const set = vi.spyOn(provider, 'set').mockRejectedValueOnce(new Error('backing store is full')); + await expect(provider.saveTrackedAddresses([entry(2)])).rejects.toThrow('backing store is full'); + + await provider.saveTrackedAddresses([entry(3)]); + set.mockRestore(); + + // The failed write stored nothing; the one after it merged as usual. + expect(indices(await provider.loadTrackedAddresses())).toEqual([0, 1, 3]); + }); + }); + }); +} diff --git a/tests/unit/storage/tracked-addresses-providers.test.ts b/tests/unit/storage/tracked-addresses-providers.test.ts new file mode 100644 index 00000000..0346e709 --- /dev/null +++ b/tests/unit/storage/tracked-addresses-providers.test.ts @@ -0,0 +1,56 @@ +/** + * The saveTrackedAddresses merge contract, run against ALL THREE StorageProvider + * implementations. Each carries its own copy of the read-merge-write, so a suite + * bound to one of them leaves the other two free to regress to a wholesale write — + * the lost update that erases a live Sphere's address (#766 item 5). + */ +import 'fake-indexeddb/auto'; + +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; + +import { IndexedDBStorageProvider } from '../../../impl/browser/storage/IndexedDBStorageProvider'; +import { LocalStorageProvider } from '../../../impl/browser/storage/LocalStorageProvider'; +import { FileStorageProvider } from '../../../impl/nodejs/storage/FileStorageProvider'; +import { describeTrackedAddressesContract } from './contracts/tracked-addresses.contract'; + +let seq = 0; + +/** Enough of the Web Storage surface for LocalStorageProvider (get/set/remove + keys()). */ +function memoryWebStorage(): Storage { + const map = new Map(); + return { + get length(): number { return map.size; }, + clear: (): void => { map.clear(); }, + getItem: (key: string): string | null => map.get(key) ?? null, + key: (i: number): string | null => Array.from(map.keys())[i] ?? null, + removeItem: (key: string): void => { map.delete(key); }, + setItem: (key: string, value: string): void => { map.set(key, value); }, + } as Storage; +} + +describeTrackedAddressesContract('FileStorageProvider', async () => { + const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'sphere-tracked-')); + const provider = new FileStorageProvider({ dataDir }); + await provider.connect(); + return { + provider, + cleanup: async () => { + await provider.disconnect(); + fs.rmSync(dataDir, { recursive: true, force: true }); + }, + }; +}); + +describeTrackedAddressesContract('IndexedDBStorageProvider', async () => { + const provider = new IndexedDBStorageProvider({ prefix: 'test_', dbName: `tracked-db-${seq++}` }); + await provider.connect(); + return { provider, cleanup: () => provider.disconnect() }; +}); + +describeTrackedAddressesContract('LocalStorageProvider', async () => { + const provider = new LocalStorageProvider({ prefix: 'test_', storage: memoryWebStorage() }); + await provider.connect(); + return { provider, cleanup: () => provider.disconnect() }; +}); diff --git a/tests/unit/storage/tracked-addresses.test.ts b/tests/unit/storage/tracked-addresses.test.ts new file mode 100644 index 00000000..c0d3914d --- /dev/null +++ b/tests/unit/storage/tracked-addresses.test.ts @@ -0,0 +1,93 @@ +/** + * `parseTrackedAddresses` is deliberately REPAIRING — an odd `hidden` or a missing + * timestamp must not delete one of the user's addresses. The index is the single + * exception, and it is a money-safety one: the address path is rebuilt as + * `.../${index}` and `deriveKeyAtPath` parseInt()s that segment (core/crypto.ts), so a + * stored `1.5` derives index 1's keys and the row ALIASES a real address — a second + * registry entry, with its own hidden flag and timestamps, silently steering funds at + * an address the user already has. Negatives and NaN have no valid derivation at all. + * + * `Number.isFinite` accepted every one of those. + */ +import { describe, expect, it } from 'vitest'; + +import { parseTrackedAddresses } from '../../../storage/tracked-addresses'; + +function stored(...addresses: unknown[]): string { + return JSON.stringify({ version: 1, addresses }); +} + +describe('parseTrackedAddresses — the index must be a non-negative integer', () => { + it('drops a fractional index rather than letting it alias a real address', () => { + // parseInt('1.5') === 1: this row would derive index 1's keys. + const parsed = parseTrackedAddresses( + stored( + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 1, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 1.5, hidden: true, createdAt: 2, updatedAt: 2 }, + ), + ); + + expect(parsed.map((e) => e.index)).toEqual([0, 1]); + }); + + it('drops a negative index', () => { + const parsed = parseTrackedAddresses( + stored( + { index: -1, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + ), + ); + + expect(parsed.map((e) => e.index)).toEqual([0]); + }); + + it('drops the whole rejection set at once, keeping only the valid rows', () => { + // The fractional and negative cases again, now beside every other shape an index + // can arrive in. A NaN index cannot round-trip — JSON.stringify writes it as null — + // but an overflowing literal can: JSON.parse('1e999') is Infinity. + expect(JSON.parse(stored({ index: NaN })).addresses[0].index).toBeNull(); + const row = (index: string): string => + `{"index":${index},"hidden":false,"createdAt":1,"updatedAt":1}`; + const parsed = parseTrackedAddresses( + `{"version":1,"addresses":[${[ + row('1.5'), + row('-1'), + row('null'), // what a NaN write leaves behind + row('1e999'), // Infinity + row('"2"'), // a number-shaped string + '{"hidden":false,"createdAt":1,"updatedAt":1}', // no index at all + row('0'), + row('2'), + ].join(',')}]}`, + ); + + expect(parsed.map((e) => e.index)).toEqual([0, 2]); + }); + + it('keeps every integer index, including 0', () => { + const parsed = parseTrackedAddresses( + stored( + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 7, hidden: true, createdAt: 2, updatedAt: 3 }, + ), + ); + + expect(parsed).toEqual([ + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 7, hidden: true, createdAt: 2, updatedAt: 3 }, + ]); + }); + + it('keeps unknown extra fields on a valid row — a newer writer own columns survive', () => { + // A row written by a future version must round-trip through an older reader, or the + // merge-on-write turns every save by this Sphere into a downgrade of the other one. + const parsed = parseTrackedAddresses( + stored({ index: 3, hidden: false, createdAt: 1, updatedAt: 1, label: 'savings', pinned: true }), + ); + + expect(parsed).toEqual([ + { index: 3, hidden: false, createdAt: 1, updatedAt: 1, label: 'savings', pinned: true }, + ]); + }); +}); From 15e1e3e7bff05922a1a2cf9a29a8f67d683d02e8 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Wed, 2 Sep 2026 23:54:25 +0200 Subject: [PATCH 04/43] fix(storage): bound tracked-address indices to uint32; test the logger fix; reconcile the release notes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's second pass, three findings, all valid — and one shows the previous round fixed only half a bug. 1. The index guard stopped short of the derivation's real range Last round rejected fractional and negative indices, because deriveKeyAtPath parseInt()s the path segment so a row with index 1.5 derives index 1's keys. But deriveChildKey serializes the child number as `index.toString(16).padStart(8, '0')` (core/crypto.ts:166-177), and above 0xffffffff toString(16) yields MORE than 8 hex digits — padStart does not truncate, so the derivation silently emits extra bytes. Same class of defect, the other end of the range. Now a uint32. Boundary tests added, and each clause proven independently load-bearing: removing `<= 0xffffffff` reds only the over-range case, `>= 0` only the negatives, `Number.isInteger` only the fractionals. 0xffffffff, 0x80000000 (hardened threshold) and 0 stay green under every mutant, so the ceiling never swallows a legal index. 2. The logger fix had no regression The one-way flag could have come back with the suite green — the same coverage gap Copilot flagged last round, in code written to satisfy that round. Eleven tests, discriminating per entry point: reverting `init` reds three, `create`/`load`/`import` one each, no cross-talk. That work turned up a subtlety worth recording: `Sphere.init` does NOT forward `debug` to the create/load it dispatches to. It works today only because init configures the global first AND `!== undefined` makes the later calls no-ops. Under a `?? false` rewrite, `init({ debug: true })` would end up OFF, because create would then run `configure({ debug: undefined ?? false })` after init had set it true. So the guard is load-bearing for a second reason beyond the one-way bug, and a test pins exactly that sequence. My earlier claim that "#769's debug finding was wrong" was too strong — the forwarding really is missing; it is currently harmless rather than absent. 3. The release notes contradicted the code The migration guide still listed the now-removed statics under "Not fixed by this release", and CHANGELOG [Unreleased] described only #767's registry work — so a consumer reading either was told the Sphere lifecycle globals still exist. Both now give one account, with only the #771 FileStorageProvider limitation left under "Not fixed". Probes 71 -> 77, all hand-verified KILLED. One had gone STALE when the uint32 fix moved the check into isDerivableIndex(); repointed rather than deleted, and its replace still reproduces the original Number.isFinite bug. --- CHANGELOG.md | 55 ++++++ docs/MIGRATION-TOKEN-REGISTRY.md | 26 ++- storage/tracked-addresses.ts | 10 +- tests/mutation/probes.json | 66 ++++++- tests/unit/core/Sphere.debug-logging.test.ts | 194 +++++++++++++++++++ tests/unit/storage/tracked-addresses.test.ts | 55 ++++++ 6 files changed, 393 insertions(+), 13 deletions(-) create mode 100644 tests/unit/core/Sphere.debug-logging.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 0aa747ac..04c4171b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Removed (BREAKING) — the Sphere lifecycle globals (#766) + +`Sphere.getInstance()`, `Sphere.isInitialized()` and the root export `getSphere` are gone +(root exports 126 → 125). Hold the instance the entry point returns; `sphere.isReady` +answers the instance-level question. The consumer gate found zero users across every +sibling repo. + +Not deprecated, because the defect survives deprecation: after a second Sphere was created +*and destroyed*, `getInstance()` returned `null` while the first was alive and serving money. + +### Fixed — `clear()`/`import()` no longer destroy an unrelated wallet (#766) + +Both keyed off the removed static rather than the storage they were handed, so +`Sphere.import({ storage: B })` destroyed a live wallet on storage **A**: identity nulled, +`payments` throwing `NOT_INITIALIZED`, providers disconnected, and every `sphere.on()` +handler dropped with no event and no error. They are now scoped by provider object identity +(`StorageProvider.id` is a class constant, so comparing it would still have hit the wrong +instance). The `exists(storage)` disjunct is preserved — that is the storage-wipe contract +consumers actually depend on. + +`importFromLegacyFile` returned the wrong Sphere under an interleaved init, because +`importFromJSON` discarded the one it built; it is now threaded out (additive). + +### Fixed — tracked addresses are merged, not clobbered (#766) + +Every persist wrote the instance's whole snapshot, so a second Sphere's address switch +erased the first's entry from disk while its in-memory view still showed it. Now +read-merge-write, serialized per provider: union by index, greater `updatedAt` wins +`hidden`. Safe because there is no delete path. Deliberately **not** network-scoped — the +payload is network-agnostic (`deriveDirectAddress` takes no network) and the bug reproduces +with both Spheres on one network. + +A stored `index` must now be a uint32. `deriveKeyAtPath` `parseInt()`s the path segment, so +a row with index `1.5` derived index `1`'s keys and impersonated a real address; and +`deriveChildKey` pads the child number to 8 hex digits, so anything above `0xffffffff` +emitted extra bytes. Such rows are dropped on read. + +### Fixed — the `debug` flag was one-way, and `verification` was dropped (#766, #769) + +`if (options.debug)` meant no later init could turn debug off; all four entry points now +honour an explicit `false`. `createNodeProviders` had the mirror-image bug — `?? false` +silently disabled a flag the consumer had set — and now only overrides when told to. + +`Sphere.init` also never forwarded `verification` to `create`/`load`, so opting into the +worker pool at the documented entry point silently gave you the sequential verifier. + +The logger stays process-global by design: most of its 370 call sites are in providers +constructed before any Sphere exists and shared between them. + +### Fixed — `TokenRegistry.resetInstance()` now disposes (#770) + +It called only `stopAutoRefresh()`, leaving `disposed` unset, the generation unchanged and +the in-flight fetch running — so a load past its entry guard re-armed the interval on an +instance `getInstance()` could no longer return. + ### Changed — a Sphere owns its token registry, and destroy() disposes it (#766) `TokenRegistry` is a process-global singleton whose `configure()` repoints whatever instance diff --git a/docs/MIGRATION-TOKEN-REGISTRY.md b/docs/MIGRATION-TOKEN-REGISTRY.md index ab6334ea..56921ebb 100644 --- a/docs/MIGRATION-TOKEN-REGISTRY.md +++ b/docs/MIGRATION-TOKEN-REGISTRY.md @@ -90,14 +90,26 @@ You can get ahead of it now: Nothing above is required in this release. It is what will make the removal a small change rather than a large one. -## Not fixed by this release +## Also removed: the Sphere lifecycle globals `Sphere.getInstance()`, `Sphere.isInitialized()` and the `getSphere` export are **removed** -(see [#766](https://github.com/unicity-sphere/sphere-sdk/issues/766)); hold the instance the -entry point returns, and use `sphere.isReady`. `Sphere.clear()` / `Sphere.import()` now -destroy only Spheres built on the storage they are given. +([#766](https://github.com/unicity-sphere/sphere-sdk/issues/766)). Hold the instance the entry +point returns, and use `sphere.isReady`. Nothing in the fleet used them — the consumer gate +found zero call sites across every sibling repo. + +They could not be safely deprecated: after a second Sphere is created *and destroyed*, +`getInstance()` returned `null` while the first was alive and serving money, and a deprecation +note does not stop a wrong answer being consumed. + +`Sphere.clear()` and `Sphere.import()` now destroy only Spheres built on the storage they are +given, compared by object identity. Previously they destroyed whichever Sphere was constructed +last — so `Sphere.import({ storage: B })` killed a live wallet on storage A, dropping every +`sphere.on()` handler with no event and no error. The `exists(storage)` behaviour that callers +actually depend on is unchanged. + +## Not fixed by this release -What remains: two `FileStorageProvider` objects pointed at one `dataDir` still clobber each -other's wallet file — that provider caches the whole store in memory and rewrites the entire -file on every `set()`, so it affects every key, not just one. Tracked as +Two `FileStorageProvider` objects pointed at one `dataDir` still clobber each other's wallet +file — that provider caches the whole store in memory and rewrites the entire file on every +`set()`, so it affects every key, money journals included, not just one. Tracked as [#771](https://github.com/unicity-sphere/sphere-sdk/issues/771). diff --git a/storage/tracked-addresses.ts b/storage/tracked-addresses.ts index e20af5bf..43fd75d8 100644 --- a/storage/tracked-addresses.ts +++ b/storage/tracked-addresses.ts @@ -6,16 +6,20 @@ export interface TrackedAddressesFile { addresses: TrackedAddressEntry[]; } +/** A BIP32 child number is a uint32 — see the port docstring for why. */ +function isDerivableIndex(value: unknown): value is number { + return typeof value === 'number' && Number.isInteger(value) && value >= 0 && value <= 0xffffffff; +} + function num(value: unknown, fallback: number): number { return typeof value === 'number' && Number.isFinite(value) ? value : fallback; } -/** Repair, don't drop — except the index: deriveKeyAtPath parseInt()s it, so 1.5 - * would alias index 1's real address (see the port docstring). */ +/** Repair, don't drop — except an underivable index, which would alias a real address. */ function toEntry(value: unknown): TrackedAddressEntry | null { if (typeof value !== 'object' || value === null) return null; const e = value as Record; - if (typeof e.index !== 'number' || !Number.isInteger(e.index) || e.index < 0) return null; + if (!isDerivableIndex(e.index)) return null; return { ...e, index: e.index, diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 9d8ac112..a5dd535c 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -703,10 +703,70 @@ }, { "name": "tracked-address-index-aliases-real-address", - "note": "#766 review: the stored index must be a non-negative INTEGER - deriveKeyAtPath parseInt()s the path segment, so a 1.5 row derives index 1's keys and aliases a real address; Number.isFinite let 1.5 and negatives through", + "note": "#766 review: the stored index must be a non-negative INTEGER - deriveKeyAtPath parseInt()s the path segment, so a 1.5 row derives index 1's keys and aliases a real address; Number.isFinite let 1.5 and negatives through. Re-pointed at isDerivableIndex(), where the check now lives.", "file": "storage/tracked-addresses.ts", - "find": " if (typeof e.index !== 'number' || !Number.isInteger(e.index) || e.index < 0) return null;", - "replace": " if (typeof e.index !== 'number' || !Number.isFinite(e.index)) return null;", + "find": " return typeof value === 'number' && Number.isInteger(value) && value >= 0 && value <= 0xffffffff;", + "replace": " return typeof value === 'number' && Number.isFinite(value);", + "tests": [ + "tests/unit/storage/tracked-addresses.test.ts" + ] + }, + { + "name": "sphere-debug-flag-one-way-init", + "note": "#766: the logger's state lives on globalThis, so `debug` is a PROCESS-global flag. Truthy-only made it ONE-WAY \u2014 once anything turned debug on, no later Sphere.init({ debug: false }) could quieten it. init() does NOT forward `debug` to the create/load it dispatches to, so this site is the only one the documented entry point uses.", + "file": "core/Sphere.ts", + "find": " // Configure debug logging (also needed in main bundle context, same as TokenRegistry)\n // `undefined` leaves whatever the provider factory or consumer set; an explicit\n // `false` MUST turn debug off. A truthy-only check made this process-global flag\n // one-way \u2014 no second init could ever quieten it (#766).\n if (options.debug !== undefined) logger.configure({ debug: options.debug });", + "replace": " // Configure debug logging (also needed in main bundle context, same as TokenRegistry)\n // `undefined` leaves whatever the provider factory or consumer set; an explicit\n // `false` MUST turn debug off. A truthy-only check made this process-global flag\n // one-way \u2014 no second init could ever quieten it (#766).\n if (options.debug) logger.configure({ debug: true }); // mutant: truthy-only, the flag goes one-way again", + "tests": [ + "tests/unit/core/Sphere.debug-logging.test.ts" + ] + }, + { + "name": "sphere-debug-flag-one-way-create", + "note": "#766: the SAME guard on the direct create() entry point \u2014 all four sites are written out by hand, so one can regress with the other three intact.", + "file": "core/Sphere.ts", + "find": " if (options.debug !== undefined) logger.configure({ debug: options.debug });\n\n // Fail-closed BEFORE any storage write:", + "replace": " if (options.debug) logger.configure({ debug: true }); // mutant: truthy-only, the flag goes one-way again\n\n // Fail-closed BEFORE any storage write:", + "tests": [ + "tests/unit/core/Sphere.debug-logging.test.ts" + ] + }, + { + "name": "sphere-debug-flag-one-way-load", + "note": "#766: the SAME guard on the direct load() entry point.", + "file": "core/Sphere.ts", + "find": " if (options.debug !== undefined) logger.configure({ debug: options.debug });\n\n // Fail-closed first: retired module options", + "replace": " if (options.debug) logger.configure({ debug: true }); // mutant: truthy-only, the flag goes one-way again\n\n // Fail-closed first: retired module options", + "tests": [ + "tests/unit/core/Sphere.debug-logging.test.ts" + ] + }, + { + "name": "sphere-debug-flag-one-way-import", + "note": "#766: the SAME guard on the direct import() entry point.", + "file": "core/Sphere.ts", + "find": " if (options.debug !== undefined) logger.configure({ debug: options.debug });\n\n // Fail-closed BEFORE the destructive clear below:", + "replace": " if (options.debug) logger.configure({ debug: true }); // mutant: truthy-only, the flag goes one-way again\n\n // Fail-closed BEFORE the destructive clear below:", + "tests": [ + "tests/unit/core/Sphere.debug-logging.test.ts" + ] + }, + { + "name": "tracked-address-index-above-uint32", + "note": "#766 review: a BIP32 child number is a uint32. deriveChildKey does index.toString(16).padStart(8,'0') and padStart only ADDS characters, so an index above 0xffffffff emits a 9th hex digit and pushes an extra byte into the HMAC input \u2014 off-standard derivation, silently. 0xffffffff itself and the hardened threshold 0x80000000 must SURVIVE.", + "file": "storage/tracked-addresses.ts", + "find": " return typeof value === 'number' && Number.isInteger(value) && value >= 0 && value <= 0xffffffff;", + "replace": " return typeof value === 'number' && Number.isInteger(value) && value >= 0;", + "tests": [ + "tests/unit/storage/tracked-addresses.test.ts" + ] + }, + { + "name": "tracked-address-index-negative-allowed", + "note": "#766 review: the floor clause on its own \u2014 a negative index has no BIP32 derivation at all. Probed apart from the integer and ceiling clauses so a regression names which one went.", + "file": "storage/tracked-addresses.ts", + "find": " return typeof value === 'number' && Number.isInteger(value) && value >= 0 && value <= 0xffffffff;", + "replace": " return typeof value === 'number' && Number.isInteger(value) && value <= 0xffffffff;", "tests": [ "tests/unit/storage/tracked-addresses.test.ts" ] diff --git a/tests/unit/core/Sphere.debug-logging.test.ts b/tests/unit/core/Sphere.debug-logging.test.ts new file mode 100644 index 00000000..4c5f8107 --- /dev/null +++ b/tests/unit/core/Sphere.debug-logging.test.ts @@ -0,0 +1,194 @@ +/** + * `debug: false` at a Sphere entry point must turn the process-global logger OFF (#766). + * + * The logger's state lives on `globalThis` so it is shared across tsup bundles — which + * makes it a PROCESS-global flag, not a per-Sphere one. All four entry points used to + * write it truthy-only (`if (options.debug) logger.configure({ debug: true })`), so the + * flag was ONE-WAY: once anything switched debug on — a provider factory, an earlier + * Sphere, a consumer calling `logger.configure` directly — no later `Sphere.init({ debug: + * false })` could ever quieten it again. A wallet that logs every operation forever is a + * privacy leak the consumer has no documented way to stop. + * + * `tests/unit/core/logger.test.ts` proves `logger.configure({ debug: false })` works; it + * cannot see whether Sphere ever CALLS it. That is the hole this file closes. + * + * The suite discriminates WHICH of the four sites broke, because each one is written out + * by hand and any one can be reverted with the other three intact: + * - `init` is its own site: it does NOT forward `debug` to the create/load call it + * dispatches to (see the option lists in `Sphere.init`), so both init tests fail + * together and only when line ~640 regresses. + * - the direct `create` / `load` / `import` tests each fail alone. + * + * The omission cases guard the other half of the contract — `!== undefined` rather than + * `?? false`. `debug` left out must leave whatever the provider factory or the consumer + * set, so a Sphere built without an opinion never silences someone else's logging. + */ + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { Sphere } from '../../../core/Sphere'; +import { logger } from '../../../core/logger'; +import type { OracleProvider } from '../../../oracle'; +import type { TransportProvider } from '../../../transport'; +import { TEST_NETWORK } from '../../test-network'; +import { makeMockProviders, TEST_MNEMONIC, type MockProviders } from './support/mock-providers'; + +/** A second valid BIP39 vector, so import() writes a different wallet than it reads. */ +const OTHER_MNEMONIC = + 'legal winner thank year wave sausage worth useful legal winner thank yellow'; + +describe('Sphere entry points honour `debug: false` (#766)', () => { + let providers: MockProviders; + let live: Sphere | null = null; + + beforeEach(() => { + live = null; + // The state every test starts from: SOMETHING already turned debug on. The handler + // keeps the debug lines the Sphere entry points emit out of the test output; it does + // not affect `isDebugEnabled`, which is the flag under test. + logger.reset(); + logger.configure({ debug: true, handler: () => {} }); + expect(logger.isDebugEnabled(), 'the fixture must start with debug ON').toBe(true); + }); + + afterEach(async () => { + if (live) { + try { await live.destroy(); } catch { /* already torn down */ } + } + live = null; + logger.reset(); + }); + + function base(walletExists: boolean): MockProviders { + providers = makeMockProviders({ walletExists }); + return providers; + } + + function common(p: MockProviders) { + return { + storage: p.storage, + transport: p.transport as unknown as TransportProvider, + oracle: p.oracle as unknown as OracleProvider, + walletApi: p.walletApi, + network: TEST_NETWORK, + }; + } + + // =========================================================================== + // debug: false must turn the global flag OFF + // =========================================================================== + + it('init() — create branch (no wallet yet)', async () => { + const p = base(false); + const { sphere, created } = await Sphere.init({ + ...common(p), + autoGenerate: true, + debug: false, + }); + live = sphere; + + expect(created, 'this test must exercise init()’s create branch').toBe(true); + expect(logger.isDebugEnabled()).toBe(false); + }); + + it('init() — load branch (wallet already exists)', async () => { + const p = base(true); + const { sphere, created } = await Sphere.init({ + ...common(p), + autoGenerate: true, + debug: false, + }); + live = sphere; + + expect(created, 'this test must exercise init()’s load branch').toBe(false); + expect(logger.isDebugEnabled()).toBe(false); + }); + + it('create() called directly', async () => { + const p = base(false); + live = await Sphere.create({ ...common(p), mnemonic: TEST_MNEMONIC, debug: false }); + + expect(logger.isDebugEnabled()).toBe(false); + }); + + it('load() called directly', async () => { + const p = base(true); + live = await Sphere.load({ ...common(p), debug: false }); + + expect(logger.isDebugEnabled()).toBe(false); + }); + + it('import() called directly', async () => { + const p = base(false); + live = await Sphere.import({ ...common(p), mnemonic: OTHER_MNEMONIC, debug: false }); + + expect(logger.isDebugEnabled()).toBe(false); + }); + + // =========================================================================== + // debug OMITTED must leave the current value alone (`!== undefined`, not `?? false`) + // =========================================================================== + + describe('`debug` omitted leaves the flag where it was', () => { + it('init() — create branch', async () => { + const p = base(false); + const { sphere, created } = await Sphere.init({ ...common(p), autoGenerate: true }); + live = sphere; + + expect(created).toBe(true); + expect(logger.isDebugEnabled()).toBe(true); + }); + + it('init() — load branch', async () => { + const p = base(true); + const { sphere, created } = await Sphere.init({ ...common(p), autoGenerate: true }); + live = sphere; + + expect(created).toBe(false); + expect(logger.isDebugEnabled()).toBe(true); + }); + + it('create() called directly', async () => { + const p = base(false); + live = await Sphere.create({ ...common(p), mnemonic: TEST_MNEMONIC }); + + expect(logger.isDebugEnabled()).toBe(true); + }); + + it('load() called directly', async () => { + const p = base(true); + live = await Sphere.load({ ...common(p) }); + + expect(logger.isDebugEnabled()).toBe(true); + }); + + it('import() called directly', async () => { + const p = base(false); + live = await Sphere.import({ ...common(p), mnemonic: OTHER_MNEMONIC }); + + expect(logger.isDebugEnabled()).toBe(true); + }); + }); + + // =========================================================================== + // The one-way trap itself: an explicit `true` still works, and a later `false` undoes it + // =========================================================================== + + it('a debug:true init followed by a debug:false init ends up OFF', async () => { + // The exact sequence the old code could not express. Two Spheres, one storage each, + // because that is how a consumer hits it: enable debug while diagnosing, then build + // the next wallet with it off. + logger.configure({ debug: false }); + + const first = base(false); + const a = await Sphere.init({ ...common(first), autoGenerate: true, debug: true }); + expect(logger.isDebugEnabled()).toBe(true); + await a.sphere.destroy(); + + const second = base(false); + const b = await Sphere.init({ ...common(second), autoGenerate: true, debug: false }); + live = b.sphere; + + expect(logger.isDebugEnabled()).toBe(false); + }); +}); diff --git a/tests/unit/storage/tracked-addresses.test.ts b/tests/unit/storage/tracked-addresses.test.ts index c0d3914d..ec0bf17b 100644 --- a/tests/unit/storage/tracked-addresses.test.ts +++ b/tests/unit/storage/tracked-addresses.test.ts @@ -91,3 +91,58 @@ describe('parseTrackedAddresses — the index must be a non-negative integer', ( ]); }); }); + +/** + * The ceiling half of the same guard. `deriveChildKey` (core/crypto.ts) serializes the + * child number as `index.toString(16).padStart(8, '0')` — and `padStart` only ever ADDS + * characters. An index above 0xffffffff therefore emits MORE than eight hex digits and + * pushes extra bytes into the HMAC input: the derivation silently stops being BIP32, with + * no error and no log, producing a key no other wallet implementation can reproduce. + * (2**53-1 survives a JSON round-trip intact, so this is reachable from stored data.) + * + * Each clause is pinned on its own so a regression names itself: dropping `<= 0xffffffff` + * must red only the over-range row, dropping `>= 0` only the negative one. + */ +describe('parseTrackedAddresses — the index must fit a BIP32 child number (uint32)', () => { + it('keeps 0xffffffff — the largest child number BIP32 can express', () => { + const parsed = parseTrackedAddresses( + stored({ index: 0xffffffff, hidden: false, createdAt: 1, updatedAt: 1 }), + ); + + expect(parsed.map((e) => e.index)).toEqual([4294967295]); + }); + + it('keeps 0x80000000 — a hardened index is legal, not out of range', () => { + // deriveChildKey treats >= 0x80000000 as hardened derivation; the whole hardened + // half of the range is valid, so a ceiling set at the threshold would delete + // addresses the wallet can perfectly well derive. + const parsed = parseTrackedAddresses( + stored({ index: 0x80000000, hidden: false, createdAt: 1, updatedAt: 1 }), + ); + + expect(parsed.map((e) => e.index)).toEqual([2147483648]); + }); + + it('drops 0x100000000 — one past the ceiling, where the child number grows a 9th digit', () => { + // (0x100000000).toString(16) === '100000000': nine hex digits, one whole byte more + // than BIP32's serialization allows. + expect((0x100000000).toString(16).padStart(8, '0')).toHaveLength(9); + + const parsed = parseTrackedAddresses( + stored( + { index: 0, hidden: false, createdAt: 1, updatedAt: 1 }, + { index: 0x100000000, hidden: false, createdAt: 2, updatedAt: 2 }, + ), + ); + + expect(parsed.map((e) => e.index)).toEqual([0]); + }); + + it('keeps 0 — the floor is inclusive, and the ceiling check must not swallow it', () => { + const parsed = parseTrackedAddresses( + stored({ index: 0, hidden: false, createdAt: 1, updatedAt: 1 }), + ); + + expect(parsed.map((e) => e.index)).toEqual([0]); + }); +}); From 3a4a0d54ddfdd595a7a8db858f578a87925d7846 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 00:34:50 +0200 Subject: [PATCH 05/43] fix(sphere): scope liveness by BACKING STORE, not provider object (#766) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot round 3, and it caught a regression I introduced in this PR. _liveByStorage was keyed by provider OBJECT identity, which does not match what clear() actually erases. Two provider objects can address the same store: two IndexedDBStorageProviders sharing dbName+prefix, two LocalStorageProviders over one Storage with one prefix, two FileStorageProviders resolving to one filePath. Different WeakMap keys, identical data. So Sphere.clear({storage: A}) emptied the KV while a Sphere registered under its twin B stayed isReady — mnemonic and money journals deleted underneath a wallet that believed it was fine. Strictly worse than before this PR: the old last-constructed-wins static at least destroyed SOME Sphere in that case. Object keying destroyed none. I had been next to this and mis-scoped it. #771 records two FileStorageProviders clobbering each other's WRITES, and the migration guide listed "two providers over one dataDir" as a known residual — but I never saw that the same aliasing breaks the teardown scoping I was adding, in the browser providers too, and that this half is a regression rather than an inherited limit. The port gains an optional readonly backingStoreId identifying the STORE — not the object, and not the class (`id` is a class constant like 'file-storage', exactly the wrong granularity). Two providers returning the same value share erasure; omitting it falls back to object identity, so custom providers keep today's behaviour. Values are scheme-namespaced so a file store and an IndexedDB store can never collide: file: indexeddb:: localstorage:: The localStorage object tag is not belt-and-braces: getStorageSafe() mints a fresh in-memory Storage per provider under SSR, so prefix-only keying would have merged unrelated stores into one — the same class of bug in the other direction. _liveByStorage becomes Map> and unregisterLive deletes the entry when its Set empties, so the map stays bounded now that keys are strings rather than weak refs. Eleven falsifications, each with its own failure message, including one against REAL pre-fix code rather than a mutant: an accidental checkout ran the new tests against HEAD's WeakMap version and the twin Sphere survived exactly as reported. Notably `backingStoreId = 'file:'` (a class constant, the mistake the docstring warns against) reds four integration tests as well as the unit ones — cross-wallet kills come straight back. Probes 77 -> 80, all hand-verified KILLED; one refreshed rather than deleted when registerLive moved onto the key. Suppressions file untouched. Docs: CHANGELOG and the migration guide both said "object identity"; INTEGRATION.md publishes the StorageProvider interface custom implementers code against and now carries the member. The migration guide's "Not fixed" section now separates the halves — teardown aliasing is fixed here, the whole-file write clobber (#771) is not. --- CHANGELOG.md | 14 +- core/Sphere.ts | 47 ++++-- docs/INTEGRATION.md | 8 + docs/MIGRATION-TOKEN-REGISTRY.md | 23 ++- .../storage/IndexedDBStorageProvider.ts | 4 + impl/browser/storage/LocalStorageProvider.ts | 22 +++ impl/nodejs/storage/FileStorageProvider.ts | 3 + storage/storage-provider.ts | 20 +++ .../sphere-instance-scoping.test.ts | 142 +++++++++++++++++- tests/mutation/probes.json | 37 ++++- tests/unit/core/Sphere.clear.test.ts | 17 ++- tests/unit/impl/backing-store-id.test.ts | 127 ++++++++++++++++ 12 files changed, 433 insertions(+), 31 deletions(-) create mode 100644 tests/unit/impl/backing-store-id.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 04c4171b..fd84fe42 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,10 +22,16 @@ Not deprecated, because the defect survives deprecation: after a second Sphere w Both keyed off the removed static rather than the storage they were handed, so `Sphere.import({ storage: B })` destroyed a live wallet on storage **A**: identity nulled, `payments` throwing `NOT_INITIALIZED`, providers disconnected, and every `sphere.on()` -handler dropped with no event and no error. They are now scoped by provider object identity -(`StorageProvider.id` is a class constant, so comparing it would still have hit the wrong -instance). The `exists(storage)` disjunct is preserved — that is the storage-wipe contract -consumers actually depend on. +handler dropped with no event and no error. They are now scoped by the **backing store** the +provider addresses, reported by a new optional `StorageProvider.backingStoreId` — the resolved +wallet path for `FileStorageProvider`, `dbName` + key prefix for `IndexedDBStorageProvider`, +the `Storage` object + prefix for `LocalStorageProvider`. Neither of the two obvious keys +works: `StorageProvider.id` is a class constant, so it collides every wallet in the process; +provider *object* identity is too narrow the other way — two providers over one `dataDir` are +distinct objects addressing one wallet.json, so it would have destroyed NEITHER of their +Spheres, leaving a live wallet over a KV that was just emptied. A custom provider that declares +no `backingStoreId` keeps per-object scoping, unchanged. The `exists(storage)` disjunct is +preserved — that is the storage-wipe contract consumers actually depend on. `importFromLegacyFile` returned the wrong Sphere under an interleaved init, because `importFromJSON` discarded the one it built; it is now threaded out (additive). diff --git a/core/Sphere.ts b/core/Sphere.ts index 75457f8f..c0630ced 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -458,12 +458,16 @@ export interface AddressModuleSet { // ============================================================================= export class Sphere { - // Live Spheres, keyed by the StorageProvider whose data they own. NOT a liveness API: - // it exists only so clear()/import() tear down the instances that actually use the - // storage they were handed, instead of whichever Sphere was constructed last. Keyed by - // object identity, so Spheres on different providers never see each other. Two - // providers over one dataDir/DB are still distinct keys — see #766. - private static readonly _liveByStorage = new WeakMap>(); + // Live Spheres, keyed by the BACKING STORE their provider addresses. NOT a liveness + // API: it exists only so clear()/import() tear down the instances whose data they + // really erase. Object identity was too narrow — two providers over one dataDir/DB + // are different objects but the same file, so clearing through one left the other's + // Sphere live over an emptied KV (#766). + private static readonly _liveByStorage = new Map>(); + + /** Fallback keys for providers that declare no `backingStoreId` — one per object. */ + private static readonly _objectStoreKeys = new WeakMap(); + private static _objectStoreSeq = 0; // One-time best-effort cleanup of the orphaned vesting cache (prior versions). private static _orphanCacheCleaned = false; @@ -1307,23 +1311,46 @@ export class Sphere { } catch { /* ignore — cleanup is best-effort */ } } + /** + * The key a provider registers under: the store it addresses, so every provider + * over one dataDir/DB shares an entry. A provider that declares no store keeps a + * private key, leaving custom implementations scoped by object identity. + */ + private static storeKeyOf(storage: StorageProvider): string { + const declared = storage.backingStoreId; + if (declared) return `store:${declared}`; + let key = Sphere._objectStoreKeys.get(storage); + if (!key) { + key = `object:${++Sphere._objectStoreSeq}`; + Sphere._objectStoreKeys.set(storage, key); + } + return key; + } + /** Record a fully-built Sphere against the storage it owns. See `_liveByStorage`. */ private static registerLive(sphere: Sphere): void { - let live = Sphere._liveByStorage.get(sphere._storage); + const key = Sphere.storeKeyOf(sphere._storage); + let live = Sphere._liveByStorage.get(key); if (!live) { live = new Set(); - Sphere._liveByStorage.set(sphere._storage, live); + Sphere._liveByStorage.set(key, live); } live.add(sphere); } private static unregisterLive(sphere: Sphere): void { - Sphere._liveByStorage.get(sphere._storage)?.delete(sphere); + const key = Sphere.storeKeyOf(sphere._storage); + const live = Sphere._liveByStorage.get(key); + if (!live) return; + live.delete(sphere); + // String keys mean the map holds them strongly: an emptied Set must go, or every + // store ever opened is retained for the life of the process. + if (live.size === 0) Sphere._liveByStorage.delete(key); } /** Snapshot — callers iterate this while destroy() mutates the underlying Set. */ private static liveOn(storage: StorageProvider): Sphere[] { - return Array.from(Sphere._liveByStorage.get(storage) ?? []); + return Array.from(Sphere._liveByStorage.get(Sphere.storeKeyOf(storage)) ?? []); } /** diff --git a/docs/INTEGRATION.md b/docs/INTEGRATION.md index b9eebfc6..19cc74d6 100644 --- a/docs/INTEGRATION.md +++ b/docs/INTEGRATION.md @@ -798,6 +798,14 @@ interface StorageProvider { isConnected(): boolean; getStatus(): ProviderStatus; + /** + * Optional, but supply it if two provider objects can address one store: two + * instances returning the same value share erasure, so `Sphere.clear()` tears + * down the live Spheres of both. Identify the STORE (path / database + prefix), + * never the class. Omitted, liveness falls back to per-object identity. + */ + readonly backingStoreId?: string; + setIdentity(identity: FullIdentity): void; get(key: string): Promise; set(key: string, value: string): Promise; diff --git a/docs/MIGRATION-TOKEN-REGISTRY.md b/docs/MIGRATION-TOKEN-REGISTRY.md index 56921ebb..6f08d55c 100644 --- a/docs/MIGRATION-TOKEN-REGISTRY.md +++ b/docs/MIGRATION-TOKEN-REGISTRY.md @@ -102,14 +102,25 @@ They could not be safely deprecated: after a second Sphere is created *and destr note does not stop a wrong answer being consumed. `Sphere.clear()` and `Sphere.import()` now destroy only Spheres built on the storage they are -given, compared by object identity. Previously they destroyed whichever Sphere was constructed -last — so `Sphere.import({ storage: B })` killed a live wallet on storage A, dropping every -`sphere.on()` handler with no event and no error. The `exists(storage)` behaviour that callers -actually depend on is unchanged. +given — scoped by the **backing store** that storage addresses, not by the provider object. +Previously they destroyed whichever Sphere was constructed last, so `Sphere.import({ storage: B })` +killed a live wallet on storage A, dropping every `sphere.on()` handler with no event and no +error. The `exists(storage)` behaviour that callers actually depend on is unchanged. + +The store is reported by a new optional `StorageProvider.backingStoreId`: the resolved wallet +path for `FileStorageProvider`, `dbName` + key prefix for `IndexedDBStorageProvider`, the +`Storage` object + prefix for `LocalStorageProvider`. Custom providers need not implement it — +without it, each object is scoped to itself, as before. ## Not fixed by this release Two `FileStorageProvider` objects pointed at one `dataDir` still clobber each other's wallet -file — that provider caches the whole store in memory and rewrites the entire file on every -`set()`, so it affects every key, money journals included, not just one. Tracked as +file while both are live: that provider caches the whole store in memory and rewrites the +entire file on every `set()`, so a stale in-memory copy overwrites every key, money journals +included, not just the one being written. Tracked as [#771](https://github.com/unicity-sphere/sphere-sdk/issues/771). + +Only the **concurrent-write** half is still open. The **teardown** half is fixed above: the two +objects report the same `backingStoreId`, so `Sphere.clear()` through either one destroys every +live Sphere on that file instead of leaving one running over a wallet that was just emptied +underneath it. diff --git a/impl/browser/storage/IndexedDBStorageProvider.ts b/impl/browser/storage/IndexedDBStorageProvider.ts index f67bae59..161b9589 100644 --- a/impl/browser/storage/IndexedDBStorageProvider.ts +++ b/impl/browser/storage/IndexedDBStorageProvider.ts @@ -48,6 +48,8 @@ export class IndexedDBStorageProvider implements StorageProvider { readonly name = 'IndexedDB Storage'; readonly type = 'local' as const; readonly description = 'Browser IndexedDB for large-capacity persistence'; + /** The database + prefix pair — two providers over one pair share erasure (#766). */ + readonly backingStoreId: string; private prefix: string; private dbName: string; @@ -64,6 +66,8 @@ export class IndexedDBStorageProvider implements StorageProvider { this.dbName = config?.dbName ?? DB_NAME; this.network = config?.network; this.debug = config?.debug ?? false; + this.backingStoreId = + `indexeddb:${encodeURIComponent(this.dbName)}:${encodeURIComponent(this.prefix)}`; } // =========================================================================== diff --git a/impl/browser/storage/LocalStorageProvider.ts b/impl/browser/storage/LocalStorageProvider.ts index 16fc9df4..9cc4c253 100644 --- a/impl/browser/storage/LocalStorageProvider.ts +++ b/impl/browser/storage/LocalStorageProvider.ts @@ -36,11 +36,31 @@ export interface LocalStorageProviderConfig { // Implementation // ============================================================================= +/** + * Per-`Storage` tags for `backingStoreId`. The prefix alone does not identify the + * store: an SSR fallback mints a private in-memory `Storage` per provider, so two + * providers with the same prefix over different objects hold unrelated data. Weak, + * lazily assigned, and process-local — it is only ever compared with itself. + */ +const storageObjectTags = new WeakMap(); +let storageObjectSeq = 0; + +function storageObjectTag(storage: Storage): string { + let tag = storageObjectTags.get(storage); + if (!tag) { + tag = String(++storageObjectSeq); + storageObjectTags.set(storage, tag); + } + return tag; +} + export class LocalStorageProvider implements StorageProvider { readonly id = 'localStorage'; readonly name = 'Local Storage'; readonly type = 'local' as const; readonly description = 'Browser localStorage for single-device persistence'; + /** The `Storage` object + prefix — two providers over one pair share erasure (#766). */ + readonly backingStoreId: string; private config: Required> & { storage: Storage; @@ -59,6 +79,8 @@ export class LocalStorageProvider implements StorageProvider { debug: config?.debug ?? false, }; this.network = config?.network; + this.backingStoreId = + `localstorage:${storageObjectTag(storage)}:${encodeURIComponent(this.config.prefix)}`; } // =========================================================================== diff --git a/impl/nodejs/storage/FileStorageProvider.ts b/impl/nodejs/storage/FileStorageProvider.ts index e069fec7..489bb22f 100644 --- a/impl/nodejs/storage/FileStorageProvider.ts +++ b/impl/nodejs/storage/FileStorageProvider.ts @@ -30,6 +30,8 @@ export class FileStorageProvider implements StorageProvider { readonly id = 'file-storage'; readonly name = 'File Storage'; readonly type = 'local' as const; + /** The resolved wallet file — two providers over one path share erasure (#766). */ + readonly backingStoreId: string; private dataDir: string; private filePath: string; @@ -49,6 +51,7 @@ export class FileStorageProvider implements StorageProvider { this.network = config.network; } this.isTxtMode = this.filePath.endsWith('.txt'); + this.backingStoreId = `file:${path.resolve(this.filePath)}`; } setIdentity(identity: FullIdentity): void { diff --git a/storage/storage-provider.ts b/storage/storage-provider.ts index d093f5db..22195586 100644 --- a/storage/storage-provider.ts +++ b/storage/storage-provider.ts @@ -14,6 +14,26 @@ import type { BaseProvider, FullIdentity, TrackedAddressEntry } from '../types'; * All operations are async for platform flexibility */ export interface StorageProvider extends BaseProvider { + /** + * Stable identity of the BACKING STORE this provider addresses — not of this + * object, and not of the class (`id` is a class constant like `'file-storage'`, + * which is exactly the wrong granularity). + * + * Two providers that return the SAME value address the same data, so erasing + * through one erases through the other: `Sphere.clear({ storage })` tears down + * the live Spheres of every provider sharing this value, not merely those built + * on this object. Compose it from everything that selects the store (file path, + * database name, key prefix) behind a scheme prefix, so two kinds of store can + * never collide on one string. + * + * It must not change over the provider's lifetime — it is read again on teardown, + * and a value that moved would strand the entry it was registered under. + * + * Optional: omit it and liveness falls back to per-object identity, i.e. a + * second provider over the same data is treated as unrelated. + */ + readonly backingStoreId?: string; + /** * Set identity for scoped storage */ diff --git a/tests/integration/sphere-instance-scoping.test.ts b/tests/integration/sphere-instance-scoping.test.ts index 1e11c292..a86609be 100644 --- a/tests/integration/sphere-instance-scoping.test.ts +++ b/tests/integration/sphere-instance-scoping.test.ts @@ -13,6 +13,12 @@ * pinned — clearing an unrelated storage must NOT destroy A (tests 1 and 2), and * clearing A's OWN storage still MUST (test 3). Scoping that forgot the second half * would leave a Sphere alive on a KV that was just emptied under it. + * + * The scope is the BACKING STORE, not the provider object: two FileStorageProviders over + * one `dataDir` are distinct objects addressing one wallet.json, and object-identity + * keying made `clear()` through either of them destroy NEITHER of their Spheres — worse + * than the process-global it replaced, which at least destroyed one. `backingStoreId` is + * what they share; a provider that declares none keeps per-object scoping. */ import * as fs from 'fs'; @@ -24,7 +30,8 @@ import { Sphere } from '../../core/Sphere'; import { FileStorageProvider } from '../../impl/nodejs/storage/FileStorageProvider'; import type { TransportProvider } from '../../transport'; import type { OracleProvider } from '../../oracle'; -import type { ProviderStatus } from '../../types'; +import type { StorageProvider } from '../../storage'; +import type { ProviderStatus, TrackedAddressEntry } from '../../types'; import { makePv2World, createEngineOracle, type Pv2World } from '../support/pv2-world'; const NET = 'testnet2' as const; @@ -58,6 +65,50 @@ function createMockTransport(): TransportProvider { } as unknown as TransportProvider; } +/** + * A StorageProvider with NO `backingStoreId`, so liveness falls back to object identity. + * Two of these over one `shared` Map are the same store as far as the DATA is concerned — + * exactly the case the port member exists to declare, and this one declines to. + */ +class SharedMemoryStorage implements StorageProvider { + readonly id = 'shared-memory'; + readonly name = 'Shared Memory Storage'; + readonly type = 'local' as const; + private connected = false; + + constructor(private readonly shared: Map) {} + + async connect(): Promise { this.connected = true; } + async disconnect(): Promise { this.connected = false; } + isConnected(): boolean { return this.connected; } + getStatus(): ProviderStatus { return this.connected ? 'connected' : 'disconnected'; } + setIdentity(): void {} + async get(key: string): Promise { return this.shared.get(key) ?? null; } + async set(key: string, value: string): Promise { this.shared.set(key, value); } + async remove(key: string): Promise { this.shared.delete(key); } + async has(key: string): Promise { return this.shared.has(key); } + async keys(prefix?: string): Promise { + const all = Array.from(this.shared.keys()); + return prefix ? all.filter((k) => k.startsWith(prefix)) : all; + } + async clear(prefix?: string): Promise { + for (const k of await this.keys(prefix)) this.shared.delete(k); + } + async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { + this.shared.set('__tracked_addresses', JSON.stringify(entries)); + } + async loadTrackedAddresses(): Promise { + const raw = this.shared.get('__tracked_addresses'); + return raw ? (JSON.parse(raw) as TrackedAddressEntry[]) : []; + } +} + +/** The private live registry — keys are stores, so two providers over one share an entry. */ +function liveStoreKeys(): string[] { + const registry = (Sphere as unknown as { _liveByStorage: Map> })._liveByStorage; + return Array.from(registry.keys()); +} + /** One wallet's worth of independent providers — its own dataDir, storage, transport. */ interface Wallet { dataDir: string; @@ -244,6 +295,95 @@ describe('Sphere lifecycle statics are scoped to the storage they are handed (#7 expect(() => first.payments).toThrow(); expect(() => second.payments).toThrow(); }); + + it('clear() through one provider destroys the Spheres of every provider on that STORE', async () => { + // Two provider OBJECTS over one dataDir address one wallet.json. Keyed by object + // identity they are unrelated, so clear() through `a.storage` destroyed neither the + // twin's Sphere nor anything else — it just emptied the file under a live wallet. + const a = makeWallet('twin'); + const first = await initWallet(a, MNEMONIC_A); + + const twin = new FileStorageProvider({ dataDir: a.dataDir }); + expect(twin).not.toBe(a.storage); + expect(twin.backingStoreId).toBe(a.storage.backingStoreId); + + const { sphere: second, created } = await Sphere.init({ + storage: twin, + transport: createMockTransport(), + oracle: a.oracle, + walletApi: makePv2World(NET).walletApi, + network: NET, + }); + extraSpheres.push(second); + expect(created, 'the twin provider must LOAD the wallet the first created').toBe(false); + expect(second.identity!.chainPubkey).toBe(first.identity!.chainPubkey); + + await Sphere.clear({ storage: a.storage }); + + expect(first.isReady).toBe(false); + expect(second.isReady, 'the twin addresses the KV clear() just emptied').toBe(false); + expect(() => second.payments).toThrow(); + }); + + it('the live registry drops a store entry once its last Sphere is destroyed', async () => { + // The map is keyed by STRING now, so nothing collects an emptied Set for us: every + // dataDir a process ever opened would be retained, with a dead Sphere inside it. + const before = liveStoreKeys().length; + + const a = makeWallet('bounded'); + const first = await initWallet(a, MNEMONIC_A); + const { sphere: second } = await Sphere.init({ + storage: new FileStorageProvider({ dataDir: a.dataDir }), + transport: createMockTransport(), + oracle: a.oracle, + walletApi: makePv2World(NET).walletApi, + network: NET, + }); + extraSpheres.push(second); + + expect(liveStoreKeys().length, 'both providers share ONE entry').toBe(before + 1); + + await first.destroy(); + expect(liveStoreKeys().length, 'the second Sphere still holds the entry').toBe(before + 1); + + await second.destroy(); + expect(liveStoreKeys().length, 'the emptied Set must be removed, not left behind').toBe(before); + }); + + it('a provider that declares no backing store keeps object-identity scoping', async () => { + // The documented fallback for custom implementations: without `backingStoreId` the + // SDK cannot know two objects share data, so it scopes each one to itself — the + // pre-existing behaviour, and it must not degrade into one shared bucket for all. + const shared = new Map(); + const memA: StorageProvider = new SharedMemoryStorage(shared); + const memB: StorageProvider = new SharedMemoryStorage(shared); + expect(memA.backingStoreId).toBeUndefined(); + + const { sphere: sphereA } = await Sphere.init({ + storage: memA, + transport: createMockTransport(), + oracle: createEngineOracle(), + walletApi: makePv2World(NET).walletApi, + network: NET, + mnemonic: MNEMONIC_A, + }); + extraSpheres.push(sphereA); + + const { sphere: sphereB, created } = await Sphere.init({ + storage: memB, + transport: createMockTransport(), + oracle: createEngineOracle(), + walletApi: makePv2World(NET).walletApi, + network: NET, + }); + extraSpheres.push(sphereB); + expect(created, 'memB reads the same Map, so it LOADS').toBe(false); + + await Sphere.clear({ storage: memA }); + + expect(sphereA.isReady).toBe(false); + expect(sphereB.isReady, 'undeclared stores stay scoped per object').toBe(true); + }); }); /** diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index a5dd535c..6b8ff859 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -633,10 +633,10 @@ }, { "name": "live-registry-tracks-one-sphere-per-storage", - "note": "#766: _liveByStorage maps a provider to a SET - two Spheres can share one provider (the second LOADS the wallet the first created), and clear() must destroy both or one keeps running over an emptied KV", + "note": "#766: _liveByStorage maps a STORE to a SET - more than one Sphere can run over one store (a second provider LOADS the wallet the first created), and clear() must destroy all of them or one keeps running over an emptied KV", "file": "core/Sphere.ts", - "find": " let live = Sphere._liveByStorage.get(sphere._storage);\n if (!live) {\n live = new Set();\n Sphere._liveByStorage.set(sphere._storage, live);\n }\n live.add(sphere);", - "replace": " Sphere._liveByStorage.set(sphere._storage, new Set([sphere]));", + "find": " const key = Sphere.storeKeyOf(sphere._storage);\n let live = Sphere._liveByStorage.get(key);\n if (!live) {\n live = new Set();\n Sphere._liveByStorage.set(key, live);\n }\n live.add(sphere);", + "replace": " Sphere._liveByStorage.set(Sphere.storeKeyOf(sphere._storage), new Set([sphere]));", "tests": [ "tests/integration/sphere-instance-scoping.test.ts" ] @@ -770,5 +770,36 @@ "tests": [ "tests/unit/storage/tracked-addresses.test.ts" ] + }, + { + "name": "live-registry-keyed-by-object-not-store", + "note": "#766 review: liveness must key on the BACKING STORE, not the provider object. Two FileStorageProviders over one dataDir are distinct objects addressing one wallet.json, so object keying makes clear() through either destroy NEITHER Sphere - worse than the process-global it replaced, which destroyed one.", + "file": "core/Sphere.ts", + "find": " const declared = storage.backingStoreId;", + "replace": " const declared: string | undefined = undefined; // mutant: object identity again", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "live-registry-entry-never-released", + "note": "#766 review: the map is keyed by STRING now, so nothing collects an emptied Set - every store the process ever opened would be retained, each holding a destroyed Sphere", + "file": "core/Sphere.ts", + "find": " if (live.size === 0) Sphere._liveByStorage.delete(key);", + "replace": " void live.size; // mutant: the emptied Set stays and the map grows forever", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "file-backing-store-id-is-a-class-constant", + "note": "#766 review: backingStoreId identifies the STORE. A constant (what `id` is) makes every FileStorageProvider in the process one store, so clear() on one wallet tears down every live Sphere - the exact cross-wallet kill #766 removed.", + "file": "impl/nodejs/storage/FileStorageProvider.ts", + "find": " this.backingStoreId = `file:${path.resolve(this.filePath)}`;", + "replace": " this.backingStoreId = 'file:'; // mutant: a class constant, every wallet collides", + "tests": [ + "tests/unit/impl/backing-store-id.test.ts", + "tests/integration/sphere-instance-scoping.test.ts" + ] } ] diff --git a/tests/unit/core/Sphere.clear.test.ts b/tests/unit/core/Sphere.clear.test.ts index 6fce5012..29ad22b2 100644 --- a/tests/unit/core/Sphere.clear.test.ts +++ b/tests/unit/core/Sphere.clear.test.ts @@ -76,12 +76,15 @@ describe('Sphere.clear()', () => { const storage = createMockStorage(); // A live Sphere registered against THIS storage. clear() must tear it down before - // wiping the KV out from under it. Seeded straight into the private per-storage - // registry that replaced the process-global singleton (#766); the mock's destroy() - // deregisters itself the way the real Sphere.destroy() does. - const liveByStorage = (Sphere as unknown as { - _liveByStorage: WeakMap>; - })._liveByStorage; + // wiping the KV out from under it. Seeded straight into the private registry that + // replaced the process-global singleton (#766) — under the key the provider itself + // resolves to, since that registry is keyed by BACKING STORE, not by object. The + // mock's destroy() deregisters itself the way the real Sphere.destroy() does. + const sphereStatics = Sphere as unknown as { + _liveByStorage: Map>; + storeKeyOf(storage: StorageProvider): string; + }; + const liveByStorage = sphereStatics._liveByStorage; const registered = new Set(); const mockInstance = { destroy: vi.fn(async () => { @@ -89,7 +92,7 @@ describe('Sphere.clear()', () => { }), }; registered.add(mockInstance); - liveByStorage.set(storage, registered); + liveByStorage.set(sphereStatics.storeKeyOf(storage), registered); await Sphere.clear({ storage }); diff --git a/tests/unit/impl/backing-store-id.test.ts b/tests/unit/impl/backing-store-id.test.ts new file mode 100644 index 00000000..10b41716 --- /dev/null +++ b/tests/unit/impl/backing-store-id.test.ts @@ -0,0 +1,127 @@ +/** + * #766 — `StorageProvider.backingStoreId` identifies the STORE, not the object and not + * the class. + * + * `Sphere.clear()` tears down the live Spheres of every provider that reports the same + * value, so the value has to be exactly as coarse as the data: equal whenever two + * providers would read and erase each other's keys, different whenever they would not. + * A class constant (`id`) collides every wallet in the process; the object's own identity + * (the default fallback) never collides at all and misses the case this exists for. + */ + +import { describe, expect, it } from 'vitest'; +import * as os from 'os'; +import * as path from 'path'; + +import { FileStorageProvider } from '../../../impl/nodejs/storage/FileStorageProvider'; +import { IndexedDBStorageProvider } from '../../../impl/browser/storage/IndexedDBStorageProvider'; +import { LocalStorageProvider } from '../../../impl/browser/storage/LocalStorageProvider'; + +function fakeStorage(): Storage { + const data = new Map(); + return { + get length() { return data.size; }, + clear: () => data.clear(), + getItem: (k: string) => data.get(k) ?? null, + key: (i: number) => Array.from(data.keys())[i] ?? null, + removeItem: (k: string) => { data.delete(k); }, + setItem: (k: string, v: string) => { data.set(k, v); }, + } as Storage; +} + +describe('FileStorageProvider.backingStoreId', () => { + const dataDir = path.join(os.tmpdir(), 'sphere-backing-store-id'); + + it('is equal for two providers over one wallet file, however the path was written', () => { + const a = new FileStorageProvider({ dataDir }); + // Same file, spelled relative to the cwd: unresolved, the two strings differ and the + // providers look unrelated while writing the same wallet.json. + const b = new FileStorageProvider({ dataDir: path.relative(process.cwd(), dataDir) }); + const c = new FileStorageProvider(dataDir); + + expect(a).not.toBe(b); + expect(a.backingStoreId).toBe(b.backingStoreId); + expect(a.backingStoreId, 'the string-config constructor addresses the same file').toBe( + c.backingStoreId, + ); + }); + + it('differs for a different directory or a different file in one directory', () => { + const a = new FileStorageProvider({ dataDir }); + const elsewhere = new FileStorageProvider({ dataDir: `${dataDir}-other` }); + const otherFile = new FileStorageProvider({ dataDir, fileName: 'second.json' }); + + expect(a.backingStoreId).not.toBe(elsewhere.backingStoreId); + expect(a.backingStoreId).not.toBe(otherFile.backingStoreId); + }); + + it('is not the class constant `id`', () => { + const a = new FileStorageProvider({ dataDir }); + expect(a.backingStoreId).not.toBe(a.id); + }); +}); + +describe('IndexedDBStorageProvider.backingStoreId', () => { + it('is equal for two providers over one database + prefix', () => { + const a = new IndexedDBStorageProvider(); + const b = new IndexedDBStorageProvider({ dbName: 'sphere-storage', prefix: 'sphere_' }); + + expect(a).not.toBe(b); + expect(a.backingStoreId, 'those ARE the defaults').toBe(b.backingStoreId); + }); + + it('differs on the database name and on the key prefix independently', () => { + const base = new IndexedDBStorageProvider({ dbName: 'db-a', prefix: 'p_' }); + const otherDb = new IndexedDBStorageProvider({ dbName: 'db-b', prefix: 'p_' }); + const otherPrefix = new IndexedDBStorageProvider({ dbName: 'db-a', prefix: 'q_' }); + + expect(base.backingStoreId).not.toBe(otherDb.backingStoreId); + expect(base.backingStoreId).not.toBe(otherPrefix.backingStoreId); + }); + + it('cannot be forged across the dbName/prefix boundary', () => { + // Unencoded concatenation would make ('a:b','c') and ('a','b:c') one store. + const left = new IndexedDBStorageProvider({ dbName: 'a:b', prefix: 'c' }); + const right = new IndexedDBStorageProvider({ dbName: 'a', prefix: 'b:c' }); + expect(left.backingStoreId).not.toBe(right.backingStoreId); + }); +}); + +describe('LocalStorageProvider.backingStoreId', () => { + it('is equal for two providers over one Storage object and prefix', () => { + const storage = fakeStorage(); + const a = new LocalStorageProvider({ storage }); + const b = new LocalStorageProvider({ storage, prefix: 'sphere_' }); + + expect(a).not.toBe(b); + expect(a.backingStoreId).toBe(b.backingStoreId); + }); + + it('differs when the Storage object differs, prefix held equal', () => { + // The SSR fallback mints a private in-memory Storage per provider, so the prefix + // alone would call two unrelated stores one — and erasure would follow. + const a = new LocalStorageProvider({ storage: fakeStorage(), prefix: 'sphere_' }); + const b = new LocalStorageProvider({ storage: fakeStorage(), prefix: 'sphere_' }); + + expect(a.backingStoreId).not.toBe(b.backingStoreId); + }); + + it('differs when the prefix differs, Storage object held equal', () => { + const storage = fakeStorage(); + const a = new LocalStorageProvider({ storage, prefix: 'one_' }); + const b = new LocalStorageProvider({ storage, prefix: 'two_' }); + + expect(a.backingStoreId).not.toBe(b.backingStoreId); + }); +}); + +describe('the three provider kinds never collide', () => { + it('gives every implementation its own namespace', () => { + const ids = [ + new FileStorageProvider({ dataDir: 'sphere_' }).backingStoreId, + new IndexedDBStorageProvider({ dbName: 'sphere_', prefix: 'sphere_' }).backingStoreId, + new LocalStorageProvider({ storage: fakeStorage(), prefix: 'sphere_' }).backingStoreId, + ]; + expect(new Set(ids).size, 'unrelated stores must not share one id').toBe(ids.length); + }); +}); From 4210c8ec651a3fc4f0cb7876e26102f6faef2d05 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:18:35 +0200 Subject: [PATCH 06/43] fix(storage): serialize tracked-address writes per BACKING STORE, not per provider object MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot round 4. Two of its five findings are consequences of the contract the previous commit added: backingStoreId explicitly permits multiple provider objects per store, which by my own definition makes a per-instance write chain insufficient. Two objects both read the old registry, the last put wins — the lost update this PR set out to fix, one level up. IndexedDB: the per-instance chain is gone; read, merge and put now happen inside ONE readwrite transaction. IndexedDB serializes overlapping readwrite transactions across every connection to a database, so this covers separate provider objects AND separate tabs, which no in-process coordination could. LocalStorage: no transaction exists, so the chain moves to a module-level Map. That covers separate provider objects in ONE JS realm and nothing more — two tabs still lose updates, and the comment says so rather than implying a guarantee that is not there. Also fixed from the same round: - FileStorageProvider resolved only its backingStoreId, leaving dataDir/filePath relative. A later process.chdir() would have it read and write a DIFFERENT file while reporting the id computed from the old cwd — round 3's failure mode by another route. Both are now resolved at construction, and the id derives from the absolute filePath so the identifier and the file it names cannot diverge. - The port docstring said "non-negative integer" while the code requires a uint32, so a custom provider following the text would keep values deriveChildKey serializes with extra bytes. It now states both ends of the range and why. FileStorageProvider's cross-object case is deliberately NOT claimed, and the agent proved why empirically rather than arguing it: after a sibling correctly persists [0,1,2], a single unrelated `a.set('other_key','x')` leaves the on-disk registry EMPTY — that provider caches the whole KV and rewrites the file from its own stale copy. Serializing just this path would turn the contract case green while any unrelated write still destroys the registry. A green test that a neighbouring write invalidates is worse than an honestly absent one, so the cross-object cases are omitted for File with the reason recorded at the call site. #771. One piece of defensive code was written and then removed: an explicit try/catch + tx.abort() in the success handler proved UNFALSIFIABLE — the spec's abort path already rejects, and the test passed with it deleted. Dead code that looks like a safeguard is worse than none. Probes 80 -> 84, all hand-verified KILLED; one re-pointed when the merge moved inside the transaction. No probe was added for the chain-map eviction: removing it leaks memory without changing observable behaviour, so the probe would SURVIVE, and a hollow probe is worse than an acknowledged gap. --- .../storage/IndexedDBStorageProvider.ts | 52 +++++---- impl/browser/storage/LocalStorageProvider.ts | 38 +++++-- impl/nodejs/storage/FileStorageProvider.ts | 14 ++- storage/storage-provider.ts | 6 +- tests/mutation/probes.json | 46 +++++++- .../contracts/tracked-addresses.contract.ts | 102 ++++++++++++++++-- .../tracked-addresses-providers.test.ts | 76 +++++++++++-- 7 files changed, 278 insertions(+), 56 deletions(-) diff --git a/impl/browser/storage/IndexedDBStorageProvider.ts b/impl/browser/storage/IndexedDBStorageProvider.ts index 161b9589..75bee2aa 100644 --- a/impl/browser/storage/IndexedDBStorageProvider.ts +++ b/impl/browser/storage/IndexedDBStorageProvider.ts @@ -231,31 +231,24 @@ export class IndexedDBStorageProvider implements StorageProvider { } } - /** Serializes the read-merge-write below, per provider instance. */ - private trackedWrites: Promise = Promise.resolve(); - /** * Persist the tracked-address registry by MERGING, never replacing. * * Every Sphere over this storage holds its own snapshot and writes it in * full, so a wholesale write drops the addresses this writer never saw - * (#766 item 5 — a lost update, reproducible on one network). Concurrent - * calls are serialized on `trackedWrites` so a read can never interleave - * with another call's write. + * (#766 item 5 — a lost update, reproducible on one network). The read, the + * merge and the write share ONE transaction: a lock on this object would + * order only this object's calls, and `backingStoreId` exists precisely + * because two provider objects — or two TABS — can address one database. + * IndexedDB serializes overlapping readwrite transactions across every + * connection, so both are covered. */ async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { - const run = this.trackedWrites.then(async () => { - const onDisk = parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); - const file: TrackedAddressesFile = { - version: 1, - addresses: mergeTrackedAddresses(onDisk, entries), - }; - await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify(file)); - }); - // The chain tail swallows the rejection so one failed write cannot brick - // every later one; the caller still sees the error by awaiting `run`. - this.trackedWrites = run.then(() => undefined, () => undefined); - await run; + this.ensureConnected(); + await this.idbMergeTrackedAddresses( + this.getFullKey(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES), + entries, + ); } async loadTrackedAddresses(): Promise { @@ -407,6 +400,29 @@ export class IndexedDBStorageProvider implements StorageProvider { }); } + private idbMergeTrackedAddresses(key: string, entries: TrackedAddressEntry[]): Promise { + return new Promise((resolve, reject) => { + const tx = this.db!.transaction(STORE_NAME, 'readwrite'); + const store = tx.objectStore(STORE_NAME); + const read = store.get(key); + read.onerror = () => reject(read.error); + read.onsuccess = () => { + const onDisk = parseTrackedAddresses((read.result as { v?: string } | undefined)?.v ?? null); + const file: TrackedAddressesFile = { + version: 1, + addresses: mergeTrackedAddresses(onDisk, entries), + }; + store.put({ k: key, v: JSON.stringify(file) }); + }; + // Resolve on the TRANSACTION, not the put: only completion means durable. + tx.oncomplete = () => resolve(); + tx.onerror = () => reject(tx.error); + // A throw inside onsuccess aborts the transaction: this is what stops such a save + // from resolving with nothing written. + tx.onabort = () => reject(tx.error ?? new Error('tracked-address write aborted')); + }); + } + private idbClear(): Promise { return new Promise((resolve, reject) => { const tx = this.db!.transaction(STORE_NAME, 'readwrite'); diff --git a/impl/browser/storage/LocalStorageProvider.ts b/impl/browser/storage/LocalStorageProvider.ts index 9cc4c253..79886013 100644 --- a/impl/browser/storage/LocalStorageProvider.ts +++ b/impl/browser/storage/LocalStorageProvider.ts @@ -54,6 +54,29 @@ function storageObjectTag(storage: Storage): string { return tag; } +/** + * One write chain per BACKING STORE, not per provider object: `backingStoreId` exists + * because two providers may address the same localStorage + prefix, and two per-instance + * chains both read the old registry before either writes — the lost update again, one + * level up. This coordinates a single JS realm only; a SECOND TAB writing the same store + * is genuinely not covered, and no in-process lock can cover it. + */ +const trackedWriteChains = new Map>(); + +function serializeTrackedWrite(storeId: string, task: () => Promise): Promise { + const previous = trackedWriteChains.get(storeId) ?? Promise.resolve(); + const run = previous.then(task); + // Carried forward, one transient error would reject every later write without running it. + const tail = run.then(() => undefined, () => undefined); + trackedWriteChains.set(storeId, tail); + // Drop the entry once nothing is queued behind it, so a per-request SSR storage does + // not leave a chain behind forever. + void tail.then(() => { + if (trackedWriteChains.get(storeId) === tail) trackedWriteChains.delete(storeId); + }); + return run; +} + export class LocalStorageProvider implements StorageProvider { readonly id = 'localStorage'; readonly name = 'Local Storage'; @@ -177,20 +200,17 @@ export class LocalStorageProvider implements StorageProvider { } } - /** Serializes the read-merge-write below, per provider instance. */ - private trackedWrites: Promise = Promise.resolve(); - /** * Persist the tracked-address registry by MERGING, never replacing. * * Every Sphere over this storage holds its own snapshot and writes it in * full, so a wholesale write drops the addresses this writer never saw - * (#766 item 5 — a lost update, reproducible on one network). Concurrent - * calls are serialized on `trackedWrites` so a read can never interleave - * with another call's write. + * (#766 item 5 — a lost update, reproducible on one network). localStorage + * offers no transaction, so the read-merge-write is serialized by BACKING + * STORE — see `serializeTrackedWrite`, and the realm limit stated there. */ async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { - const run = this.trackedWrites.then(async () => { + await serializeTrackedWrite(this.backingStoreId, async () => { const onDisk = parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES)); const file: TrackedAddressesFile = { version: 1, @@ -198,10 +218,6 @@ export class LocalStorageProvider implements StorageProvider { }; await this.set(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES, JSON.stringify(file)); }); - // The chain tail swallows the rejection so one failed write cannot brick - // every later one; the caller still sees the error by awaiting `run`. - this.trackedWrites = run.then(() => undefined, () => undefined); - await run; } async loadTrackedAddresses(): Promise { diff --git a/impl/nodejs/storage/FileStorageProvider.ts b/impl/nodejs/storage/FileStorageProvider.ts index 489bb22f..86319f90 100644 --- a/impl/nodejs/storage/FileStorageProvider.ts +++ b/impl/nodejs/storage/FileStorageProvider.ts @@ -42,16 +42,20 @@ export class FileStorageProvider implements StorageProvider { private _identity: FullIdentity | null = null; constructor(config: FileStorageProviderConfig | string) { + // Resolved once, at construction: a later process.chdir() would otherwise make this + // provider read and write a DIFFERENT file while still reporting the backingStoreId + // computed from the old cwd — so clear() would empty one store and destroy the + // liveness bucket of another. if (typeof config === 'string') { - this.dataDir = config; - this.filePath = path.join(config, 'wallet.json'); + this.dataDir = path.resolve(config); + this.filePath = path.join(this.dataDir, 'wallet.json'); } else { - this.dataDir = config.dataDir; - this.filePath = path.join(config.dataDir, config.fileName ?? 'wallet.json'); + this.dataDir = path.resolve(config.dataDir); + this.filePath = path.join(this.dataDir, config.fileName ?? 'wallet.json'); this.network = config.network; } this.isTxtMode = this.filePath.endsWith('.txt'); - this.backingStoreId = `file:${path.resolve(this.filePath)}`; + this.backingStoreId = `file:${this.filePath}`; } setIdentity(identity: FullIdentity): void { diff --git a/storage/storage-provider.ts b/storage/storage-provider.ts index 22195586..dd9a6d78 100644 --- a/storage/storage-provider.ts +++ b/storage/storage-provider.ts @@ -90,9 +90,11 @@ export interface StorageProvider extends BaseProvider { * - a failed write must not brick later writes, and must still reject to its * own caller. * - * A stored `index` must be a NON-NEGATIVE INTEGER. `deriveKeyAtPath` parseInt()s + * A stored `index` must be a UINT32 — a BIP32 child number. `deriveKeyAtPath` parseInt()s * that path segment, so `1.5` derives index 1's keys and the row aliases a real - * address; such rows are dropped on read rather than repaired. + * address. The ceiling matters too: `deriveChildKey` pads the child number to 8 hex + * digits, so anything above `0xffffffff` emits extra bytes and derives off-standard. + * Such rows are dropped on read rather than repaired. * * A union is safe because there is no delete path: entries are only ever added, * and wiping the wallet removes the key itself (`Sphere.clear()`). Adding a diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 6b8ff859..c9fe0d6b 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -673,10 +673,10 @@ }, { "name": "tracked-addresses-merge-drop-idb", - "note": "#766 item 5: the browser IDB provider keeps its OWN copy of the read-merge-write; a wholesale write here is the same lost update the Node provider's probe guards", + "note": "#766 item 5: the browser IDB provider keeps its OWN copy of the read-merge-write (now inside the single transaction); a wholesale write here is the same lost update the Node provider's probe guards", "file": "impl/browser/storage/IndexedDBStorageProvider.ts", - "find": " addresses: mergeTrackedAddresses(onDisk, entries),", - "replace": " addresses: entries,", + "find": " addresses: mergeTrackedAddresses(onDisk, entries),", + "replace": " addresses: entries,", "tests": [ "tests/unit/storage/tracked-addresses-providers.test.ts" ] @@ -771,6 +771,46 @@ "tests/unit/storage/tracked-addresses.test.ts" ] }, + { + "name": "idb-tracked-write-not-atomic", + "note": "#766 round 4: the IDB read-merge-write must share ONE transaction. Split back into get() then set() and two provider objects over one database - which backingStoreId exists to permit - both read the old registry before either writes, losing an address (tracked-addresses-providers.test.ts cross-object cases)", + "file": "impl/browser/storage/IndexedDBStorageProvider.ts", + "find": " this.ensureConnected();\n await this.idbMergeTrackedAddresses(\n this.getFullKey(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES),\n entries,\n );", + "replace": " const onDisk = parseTrackedAddresses(await this.get(STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES));\n await this.set(\n STORAGE_KEYS_GLOBAL.TRACKED_ADDRESSES,\n JSON.stringify({ version: 1, addresses: mergeTrackedAddresses(onDisk, entries) }),\n );", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, + { + "name": "idb-tracked-write-abort-unswallowed", + "note": "#766 round 4: a throw inside the transaction's success handler ABORTS it, so the abort is the only path that can settle the save. Drop the handler and an unserializable registry hangs saveTrackedAddresses forever instead of rejecting - switchToAddress never returns (tracked-addresses-providers.test.ts 'rejects a registry it cannot serialize')", + "file": "impl/browser/storage/IndexedDBStorageProvider.ts", + "find": " tx.onabort = () => reject(tx.error ?? new Error('tracked-address write aborted'));", + "replace": " tx.onabort = () => undefined;", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, + { + "name": "localstorage-tracked-chain-not-shared", + "note": "#766 round 4: localStorage has no transaction, so the write chain must be keyed by the BACKING STORE. Key it per call (or per object) and two providers over one storage each read the old registry - the lost update, one level up (tracked-addresses-providers.test.ts cross-object cases)", + "file": "impl/browser/storage/LocalStorageProvider.ts", + "find": " await serializeTrackedWrite(this.backingStoreId, async () => {", + "replace": " await serializeTrackedWrite(`${this.backingStoreId}:${String(Math.random())}`, async () => {", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, + { + "name": "localstorage-tracked-chain-carries-rejection", + "note": "#766 round 4: the shared-by-store chain must swallow a rejection - carried forward it is now WORSE than the per-instance version it replaced, freezing the registry for every provider object over that store, not just the one that failed", + "file": "impl/browser/storage/LocalStorageProvider.ts", + "find": " const tail = run.then(() => undefined, () => undefined);", + "replace": " const tail = run;", + "tests": [ + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] + }, { "name": "live-registry-keyed-by-object-not-store", "note": "#766 review: liveness must key on the BACKING STORE, not the provider object. Two FileStorageProviders over one dataDir are distinct objects addressing one wallet.json, so object keying makes clear() through either destroy NEITHER Sphere - worse than the process-global it replaced, which destroyed one.", diff --git a/tests/unit/storage/contracts/tracked-addresses.contract.ts b/tests/unit/storage/contracts/tracked-addresses.contract.ts index 0e7ebd89..bd5d9b2a 100644 --- a/tests/unit/storage/contracts/tracked-addresses.contract.ts +++ b/tests/unit/storage/contracts/tracked-addresses.contract.ts @@ -8,20 +8,44 @@ * it, and each keeps its own copy of the read-merge-write, so proving it for one * provider proves nothing about the other two. * + * The same lost update exists one level up, BETWEEN provider objects: `backingStoreId` + * explicitly permits two providers over one store, and a per-object lock does not order + * them. `crossObject` is where each implementation states whether it closes that. + * * Run this against a provider with `describeTrackedAddressesContract` — see * tests/unit/storage/tracked-addresses-providers.test.ts. */ -import { describe, expect, it, vi } from 'vitest'; +import { describe, expect, it } from 'vitest'; import type { StorageProvider } from '../../../../storage'; import type { TrackedAddressEntry } from '../../../../types'; export interface TrackedAddressesHarness { provider: StorageProvider; - /** Release the backing store (temp dir, IDB connection, …). */ + /** + * Arm a ONE-SHOT failure of the next tracked-address persist, the way this + * implementation actually fails (a rejecting write, a refused transaction). + */ + failNextWrite: (message: string) => void; + /** + * Build a SECOND provider object over the SAME backing store. Required when + * `crossObject` is true; the harness releases it in `cleanup`. + */ + sibling?: () => Promise; + /** Release the backing store (temp dir, IDB connections, …). */ cleanup?: () => Promise | void; } +export interface TrackedAddressesContractOptions { + /** + * `true` when two provider objects over one backing store must not lose each other's + * entries — the harness must then supply `sibling`. Otherwise the reason this + * implementation cannot hold that, so an uncovered provider reads as a stated + * decision rather than an oversight. + */ + crossObject: true | { unsupported: string }; +} + function entry(index: number, over: Partial = {}): TrackedAddressEntry { return { index, hidden: false, createdAt: 1_000, updatedAt: 1_000, ...over }; } @@ -30,20 +54,27 @@ const indices = (entries: readonly TrackedAddressEntry[]): number[] => entries.m export function describeTrackedAddressesContract( name: string, - makeHarness: () => TrackedAddressesHarness | Promise + makeHarness: () => TrackedAddressesHarness | Promise, + options: TrackedAddressesContractOptions ): void { describe(`saveTrackedAddresses contract: ${name}`, () => { - async function withProvider( - body: (provider: StorageProvider) => Promise + async function withHarness( + body: (harness: TrackedAddressesHarness) => Promise ): Promise { const harness = await makeHarness(); try { - await body(harness.provider); + await body(harness); } finally { await harness.cleanup?.(); } } + async function withProvider( + body: (provider: StorageProvider) => Promise + ): Promise { + await withHarness((harness) => body(harness.provider)); + } + it('merges the writer snapshot into the stored registry instead of replacing it', async () => { await withProvider(async (provider) => { await provider.saveTrackedAddresses([entry(0), entry(1)]); @@ -79,21 +110,74 @@ export function describeTrackedAddressesContract( }); it('a failed write rejects to its own caller and does not brick later writes', async () => { - await withProvider(async (provider) => { + await withHarness(async ({ provider, failNextWrite }) => { await provider.saveTrackedAddresses([entry(0), entry(1)]); // The serializing chain must not carry the rejection forward: every later write // would then reject without ever running, so one transient disk/IDB error would // freeze the registry for the life of the provider. - const set = vi.spyOn(provider, 'set').mockRejectedValueOnce(new Error('backing store is full')); + failNextWrite('backing store is full'); await expect(provider.saveTrackedAddresses([entry(2)])).rejects.toThrow('backing store is full'); await provider.saveTrackedAddresses([entry(3)]); - set.mockRestore(); // The failed write stored nothing; the one after it merged as usual. expect(indices(await provider.loadTrackedAddresses())).toEqual([0, 1, 3]); }); }); + + it('rejects a registry it cannot serialize rather than reporting a write it never made', async () => { + await withProvider(async (provider) => { + await provider.saveTrackedAddresses([entry(0)]); + // A BigInt cannot be JSON-serialized. Whatever throws mid-write, the caller must + // hear about it: a resolved save that stored nothing is the failure mode a + // read-merge-write inside a transaction can produce and a plain one cannot. + const unserializable = { ...entry(1), label: 1n } as unknown as TrackedAddressEntry; + await expect(provider.saveTrackedAddresses([unserializable])).rejects.toThrow(); + + expect(indices(await provider.loadTrackedAddresses())).toEqual([0]); + await provider.saveTrackedAddresses([entry(2)]); + expect(indices(await provider.loadTrackedAddresses())).toEqual([0, 2]); + }); + }); + + if (options.crossObject !== true) return; + + it('merges across SEPARATE provider objects over the same backing store', async () => { + await withHarness(async ({ provider, sibling }) => { + if (!sibling) throw new Error('crossObject providers must supply a sibling factory'); + const second = await sibling(); + const third = await sibling(); + + // Three objects, three disjoint snapshots, no awaiting between them. Per-object + // serialization lets all three read the empty registry and the last write wins. + await Promise.all([ + provider.saveTrackedAddresses([entry(0), entry(1)]), + second.saveTrackedAddresses([entry(0), entry(2)]), + third.saveTrackedAddresses([entry(0), entry(3)]), + ]); + + for (const reader of [provider, second, third]) { + expect(indices(await reader.loadTrackedAddresses())).toEqual([0, 1, 2, 3]); + } + }); + }); + + it('keeps serializing separate objects after an earlier round has settled', async () => { + await withHarness(async ({ provider, sibling }) => { + if (!sibling) throw new Error('crossObject providers must supply a sibling factory'); + const second = await sibling(); + + // A settled round may retire the shared coordination slot; the next round must + // still be ordered rather than starting from a fresh, empty chain each time. + await provider.saveTrackedAddresses([entry(0)]); + await Promise.all([ + provider.saveTrackedAddresses([entry(1)]), + second.saveTrackedAddresses([entry(2)]), + ]); + + expect(indices(await second.loadTrackedAddresses())).toEqual([0, 1, 2]); + }); + }); }); } diff --git a/tests/unit/storage/tracked-addresses-providers.test.ts b/tests/unit/storage/tracked-addresses-providers.test.ts index 0346e709..3de59d47 100644 --- a/tests/unit/storage/tracked-addresses-providers.test.ts +++ b/tests/unit/storage/tracked-addresses-providers.test.ts @@ -9,7 +9,9 @@ import 'fake-indexeddb/auto'; import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; +import { vi } from 'vitest'; +import type { StorageProvider } from '../../../storage'; import { IndexedDBStorageProvider } from '../../../impl/browser/storage/IndexedDBStorageProvider'; import { LocalStorageProvider } from '../../../impl/browser/storage/LocalStorageProvider'; import { FileStorageProvider } from '../../../impl/nodejs/storage/FileStorageProvider'; @@ -30,27 +32,85 @@ function memoryWebStorage(): Storage { } as Storage; } +/** The one-shot failure most providers offer: the underlying `set` rejects once. */ +function failNextSet(provider: StorageProvider): (message: string) => void { + return (message) => { + vi.spyOn(provider, 'set').mockRejectedValueOnce(new Error(message)); + }; +} + describeTrackedAddressesContract('FileStorageProvider', async () => { const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'sphere-tracked-')); const provider = new FileStorageProvider({ dataDir }); await provider.connect(); return { provider, + failNextWrite: failNextSet(provider), cleanup: async () => { await provider.disconnect(); fs.rmSync(dataDir, { recursive: true, force: true }); }, }; +}, { + crossObject: { + // Not an oversight and not fixable here: FileStorageProvider caches the WHOLE key-value + // store in memory and rewrites the entire file on every set(), so a sibling object rolls + // the registry back on any unrelated write — a strictly larger lost update than this + // contract, tracked as #771. Serializing the tracked-address write alone would make the + // case pass while leaving the provider unsafe. + unsupported: 'whole-file rewrite from a per-object cache — #771', + }, }); describeTrackedAddressesContract('IndexedDBStorageProvider', async () => { - const provider = new IndexedDBStorageProvider({ prefix: 'test_', dbName: `tracked-db-${seq++}` }); - await provider.connect(); - return { provider, cleanup: () => provider.disconnect() }; -}); + const dbName = `tracked-db-${seq++}`; + const open = async (): Promise => { + const created = new IndexedDBStorageProvider({ prefix: 'test_', dbName }); + await created.connect(); + return created; + }; + const provider = await open(); + const siblings: IndexedDBStorageProvider[] = []; + return { + provider, + sibling: async () => { + const created = await open(); + siblings.push(created); + return created; + }, + // A refused transaction is how IndexedDB fails a write; the read-merge-write takes + // one, so this breaks exactly the next persist and nothing after it. + failNextWrite: (message) => { + const { db } = provider as unknown as { db: IDBDatabase }; + vi.spyOn(db, 'transaction').mockImplementationOnce(() => { + throw new Error(message); + }); + }, + cleanup: async () => { + for (const each of [provider, ...siblings]) await each.disconnect(); + }, + }; +}, { crossObject: true }); describeTrackedAddressesContract('LocalStorageProvider', async () => { - const provider = new LocalStorageProvider({ prefix: 'test_', storage: memoryWebStorage() }); - await provider.connect(); - return { provider, cleanup: () => provider.disconnect() }; -}); + const storage = memoryWebStorage(); + const open = async (): Promise => { + const created = new LocalStorageProvider({ prefix: 'test_', storage }); + await created.connect(); + return created; + }; + const provider = await open(); + const siblings: LocalStorageProvider[] = []; + return { + provider, + sibling: async () => { + const created = await open(); + siblings.push(created); + return created; + }, + failNextWrite: failNextSet(provider), + cleanup: async () => { + for (const each of [provider, ...siblings]) await each.disconnect(); + }, + }; +}, { crossObject: true }); From 90a747711033193e6f53e1baf55c19e54b03e162 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:36:57 +0200 Subject: [PATCH 07/43] test(mutation): re-point the backingStoreId probe after the path-resolve refactor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit filePath is resolved once at construction now, so the probe's find string no longer matched and the runner reported it STALE (which fails the run — the intended behaviour: a probe must never silently stop guarding anything). --- tests/mutation/probes.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index c9fe0d6b..0e7b43f4 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -835,7 +835,7 @@ "name": "file-backing-store-id-is-a-class-constant", "note": "#766 review: backingStoreId identifies the STORE. A constant (what `id` is) makes every FileStorageProvider in the process one store, so clear() on one wallet tears down every live Sphere - the exact cross-wallet kill #766 removed.", "file": "impl/nodejs/storage/FileStorageProvider.ts", - "find": " this.backingStoreId = `file:${path.resolve(this.filePath)}`;", + "find": " this.backingStoreId = `file:${this.filePath}`;", "replace": " this.backingStoreId = 'file:'; // mutant: a class constant, every wallet collides", "tests": [ "tests/unit/impl/backing-store-id.test.ts", From 1ba1c633a257839fb839cf18f46415aca8d0f9da Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:43:53 +0200 Subject: [PATCH 08/43] fix(health): probe the gateway with a call it can actually route (#769.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit checkOracle POSTed get_round_number with empty params and keyed the verdict off response.ok. The gateway is a routing layer: it refuses ANY call carrying neither stateId nor shardId with HTTP 400 before it looks at the method. So the probe reported every HEALTHY gateway as unhealthy — including the one you would reach for to gate a mainnet cutover. Send get_block_height with a 32-byte stateId (all-zero: it routes like any other and reads as a probe) and read the JSON-RPC body. The status code cannot answer this in either direction — a healthy gateway answers a routing mistake with a 400 plus a body, and JSON-RPC puts application errors inside a 200 — so only a numeric result.blockNumber counts as healthy, and the gateway's own error string is surfaced in preference to a bare status. Verified live: testnet2 and mainnet both answer (block heights ~40.9k / ~268k). The e2e case asserted `typeof healthy === 'boolean'`, which passes whether the network is up or down — it is why this survived. It now asserts healthy, with the error text in the expectation so a genuine outage names itself. Reverting the request shape fails it against both live gateways. --- core/network-health.ts | 53 ++++++++++++- tests/e2e/network-health.test.ts | 21 +++-- tests/unit/core/network-health.test.ts | 101 ++++++++++++++++--------- 3 files changed, 130 insertions(+), 45 deletions(-) diff --git a/core/network-health.ts b/core/network-health.ts index cb4417f9..0981e38d 100644 --- a/core/network-health.ts +++ b/core/network-health.ts @@ -241,6 +241,42 @@ async function checkWebSocket(url: string, timeoutMs: number): Promise null); + if (readBlockNumber(body) !== null) { return { healthy: true, url, responseTimeMs }; } + const rpcError = readRpcError(body); return { healthy: false, url, responseTimeMs, - error: `HTTP ${response.status} ${response.statusText}`, + error: + rpcError ?? + (response.ok + ? 'aggregator answered without a block height' + : `HTTP ${String(response.status)} ${response.statusText}`), }; } catch (err) { return { diff --git a/tests/e2e/network-health.test.ts b/tests/e2e/network-health.test.ts index 3ae42118..56a9cbb2 100644 --- a/tests/e2e/network-health.test.ts +++ b/tests/e2e/network-health.test.ts @@ -26,14 +26,15 @@ describe('checkNetworkHealth — live testnet', () => { expect(result.services.oracle).toBeDefined(); expect(result.services.oracle!.url).toContain('gateway.testnet2.unicity.network'); - expect(typeof result.services.oracle!.healthy).toBe('boolean'); - if (result.services.oracle!.healthy) { - expect(result.services.oracle!.responseTimeMs).toBeGreaterThanOrEqual(0); - expect(result.services.oracle!.responseTimeMs).toBeLessThan(15000); - } else { - expect(result.services.oracle!.error).toBeDefined(); - } + // #769.1: assert HEALTHY, not `typeof healthy === 'boolean'`. The old shape passed + // for two years while the probe reported every live gateway unhealthy — a live check + // that accepts both answers checks nothing. The error is in the message so a genuine + // outage says which one. + expect(result.services.oracle!.error ?? 'healthy').toBe('healthy'); + expect(result.services.oracle!.healthy).toBe(true); + expect(result.services.oracle!.responseTimeMs).toBeGreaterThanOrEqual(0); + expect(result.services.oracle!.responseTimeMs).toBeLessThan(15000); expect(result.totalTimeMs).toBeGreaterThanOrEqual(0); }, 20000); @@ -99,6 +100,10 @@ describe('checkNetworkHealth — live testnet', () => { expect(result.services.oracle).toBeDefined(); expect(result.services.oracle!.url).toContain('gateway.mainnet.unicity.network'); - expect(typeof result.services.oracle!.healthy).toBe('boolean'); + + // The mainnet gateway is live (verified 2026-09-03, block height ~268k). This is the + // one check that would catch a mainnet gateway outage, so it asserts the answer. + expect(result.services.oracle!.error ?? 'healthy').toBe('healthy'); + expect(result.services.oracle!.healthy).toBe(true); }, 20000); }); diff --git a/tests/unit/core/network-health.test.ts b/tests/unit/core/network-health.test.ts index db3e4a31..638d962c 100644 --- a/tests/unit/core/network-health.test.ts +++ b/tests/unit/core/network-health.test.ts @@ -15,11 +15,15 @@ describe('checkNetworkHealth', () => { vi.restoreAllMocks(); }); + /** What the live gateway actually answers a well-formed probe (verified 2026-09-03). */ + const blockHeightBody = (n = '40932') => + new Response(JSON.stringify({ jsonrpc: '2.0', result: { blockNumber: n }, id: 1 }), { + status: 200, + }); + describe('oracle check', () => { - it('should report oracle healthy on HTTP 200', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({ jsonrpc: '2.0', result: 42 }), { status: 200 }), - ); + it('should report oracle healthy when the aggregator returns a block height', async () => { + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); @@ -29,6 +33,55 @@ describe('checkNetworkHealth', () => { expect(result.healthy).toBe(true); }); + it('sends a probe the gateway can route: get_block_height carrying a 32-byte stateId', async () => { + // #769.1 — the bug this replaces. The gateway is a routing layer and refuses + // ANY call without a stateId/shardId ("JSON-RPC requests must include either + // stateId or shardId", HTTP 400) before it looks at the method, so the old + // `get_round_number` + `params:{}` probe reported every HEALTHY gateway as + // unhealthy. Asserting only on the response would not catch a regression here: + // the mock answers whatever we send. + fetchSpy.mockResolvedValueOnce(blockHeightBody()); + + await checkNetworkHealth('testnet', { services: ['oracle'] }); + + const [, init] = fetchSpy.mock.calls[0] as [string, RequestInit]; + const sent = JSON.parse(String(init.body)) as { + method: string; + params: { stateId?: string }; + }; + expect(sent.method).toBe('get_block_height'); + expect(sent.params.stateId).toMatch(/^[0-9a-f]{64}$/); + }); + + it('reports unhealthy when a 200 carries a JSON-RPC error instead of a height', async () => { + // The falsification pin for keying off `response.ok`: JSON-RPC puts application + // errors in a 200. Only a numeric result.blockNumber proves the aggregator answered. + fetchSpy.mockResolvedValueOnce( + new Response(JSON.stringify({ jsonrpc: '2.0', error: { code: -32601, message: 'no such method' }, id: 1 }), { + status: 200, + }), + ); + + const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); + + expect(result.services.oracle!.healthy).toBe(false); + expect(result.services.oracle!.error).toContain('no such method'); + }); + + it("surfaces the gateway's own error string from a 400 rather than a bare status", async () => { + fetchSpy.mockResolvedValueOnce( + new Response(JSON.stringify({ error: 'Shard ID not found: 0' }), { + status: 400, + statusText: 'Bad Request', + }), + ); + + const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); + + expect(result.services.oracle!.healthy).toBe(false); + expect(result.services.oracle!.error).toBe('Shard ID not found: 0'); + }); + it('should report oracle unhealthy on HTTP error', async () => { fetchSpy.mockResolvedValueOnce( new Response('Server Error', { status: 500, statusText: 'Internal Server Error' }), @@ -64,9 +117,7 @@ describe('checkNetworkHealth', () => { describe('service filtering', () => { it('should only check specified services', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({ jsonrpc: '2.0', result: 1 }), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody('1')); const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); @@ -77,9 +128,7 @@ describe('checkNetworkHealth', () => { describe('result shape', () => { it('should include totalTimeMs', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); @@ -88,9 +137,7 @@ describe('checkNetworkHealth', () => { }); it('should include url in service results', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); @@ -101,9 +148,7 @@ describe('checkNetworkHealth', () => { describe('network selection', () => { it('should use testnet URLs by default', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); await checkNetworkHealth('testnet', { services: ['oracle'] }); @@ -114,9 +159,7 @@ describe('checkNetworkHealth', () => { }); it('should use mainnet URLs when specified', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); await checkNetworkHealth('mainnet', { services: ['oracle'] }); @@ -253,9 +296,7 @@ describe('checkNetworkHealth', () => { }; // Mock fetch for oracle - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({ jsonrpc: '2.0', result: 1 }), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody('1')); const result = await checkNetworkHealth('testnet', { services: ['relay', 'oracle'], @@ -324,9 +365,7 @@ describe('checkNetworkHealth', () => { }); it('should use custom oracle URL from urls option', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet', { services: ['oracle'], @@ -372,9 +411,7 @@ describe('checkNetworkHealth', () => { close() {} }; - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet', { services: ['relay', 'oracle'], @@ -478,9 +515,7 @@ describe('checkNetworkHealth', () => { }); it('should run custom checks in parallel with built-in', async () => { - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet', { services: ['oracle'], @@ -544,9 +579,7 @@ describe('checkNetworkHealth', () => { close() {} }; - fetchSpy.mockResolvedValueOnce( - new Response(JSON.stringify({}), { status: 200 }), - ); + fetchSpy.mockResolvedValueOnce(blockHeightBody()); const result = await checkNetworkHealth('testnet'); From b51639393de4b6a0fa50e8e77781e9ed4d05fb44 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:44:43 +0200 Subject: [PATCH 09/43] fix(transport): never move nostrClient before the new one is connected (#770.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit setIdentity assigned this.nostrClient = new NostrClient(...) and only disposed the old one on the success tail. If connect rejected or the deadline won, the OLD client kept its sockets, ping intervals and auto-reconnect chain while being unreachable from the provider — disconnect() reaches only the field, and the mux is no backstop (it disposes its own client only when it is not shared). Nothing in setIdentity touches status, so the caller's next retry re-entered the same branch and orphaned another one. The replacement is now built into a local and the field moves only after a successful connect: a failed connect disposes the replacement and leaves the provider on the working old client; a throwing subscribeToEvents disposes the old one from a finally, because by then the swap is committed and nothing else can reach it. Deliberately not 'tear down both' — the mux SHARES this client, so that would kill its socket over a transient relay timeout with nothing to reconnect it. Folds in the same-class timer leak in connect(): Promise.race does not cancel the loser, so the deadline timer pinned Node's event loop for the full timeout after the call returned. Both sites now go through connectWithDeadline, which clears it in a finally. --- ...NostrTransportProvider.setIdentity.test.ts | 217 ++++++++++++++++++ transport/NostrTransportProvider.ts | 78 +++++-- 2 files changed, 273 insertions(+), 22 deletions(-) create mode 100644 tests/unit/transport/NostrTransportProvider.setIdentity.test.ts diff --git a/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts b/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts new file mode 100644 index 00000000..9f7d94ea --- /dev/null +++ b/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts @@ -0,0 +1,217 @@ +/** + * NostrTransportProvider.setIdentity() — client-swap safety (#770 item 1) + * + * setIdentity() replaces the live NostrClient when the identity changes while + * connected. The swap must leave EXACTLY ONE surviving client on every path: + * + * - connect fails → the half-built replacement is disposed, the provider keeps + * the working old client (which the Mux may be sharing — see the comment in + * setIdentity: tearing both down would kill the Mux's socket). + * - connect succeeds, subscribe throws → the field has already moved, so the + * old client must be disposed anyway. + * + * The regression this pins: the field used to be assigned BEFORE connect, with + * `oldClient.disconnect()` only on the success tail — so a failed connect + * orphaned the old client (open socket, unreachable from the provider, so + * `disconnect()` could not reach it either) and every retry leaked another one. + */ + +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import type { WebSocketFactory } from '../../../transport/websocket'; + +// ============================================================================= +// Mock NostrClient — every construction is a DISTINGUISHABLE, recorded instance +// ============================================================================= + +interface MockClient { + readonly id: number; + readonly connect: ReturnType; + readonly disconnect: ReturnType; + readonly isConnected: ReturnType; + readonly getConnectedRelays: ReturnType; + readonly subscribe: ReturnType; + readonly unsubscribe: ReturnType; + readonly publishEvent: ReturnType; + readonly addConnectionListener: ReturnType; + readonly removeConnectionListener: ReturnType; +} + +/** Every NostrClient ever constructed, in construction order. */ +const clients: MockClient[] = []; + +/** Per-client failure injection, keyed by construction index. */ +const rejectConnectFor = new Set(); +const hangConnectFor = new Set(); +const throwSubscribeFor = new Set(); + +vi.mock('@unicitylabs/nostr-js-sdk', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + NostrClient: vi.fn().mockImplementation(() => { + const id = clients.length; + const client: MockClient = { + id, + connect: vi.fn(async () => { + if (rejectConnectFor.has(id)) throw new Error(`relay refused (client ${id})`); + if (hangConnectFor.has(id)) await new Promise(() => { /* never settles */ }); + }), + disconnect: vi.fn(), + isConnected: vi.fn().mockReturnValue(true), + getConnectedRelays: vi.fn().mockReturnValue(new Set(['wss://relay1.test'])), + subscribe: vi.fn(() => { + if (throwSubscribeFor.has(id)) throw new Error(`subscribe failed (client ${id})`); + return `sub-${id}`; + }), + unsubscribe: vi.fn(), + publishEvent: vi.fn().mockResolvedValue('mock-event-id'), + addConnectionListener: vi.fn(), + removeConnectionListener: vi.fn(), + }; + clients.push(client); + return client; + }), + }; +}); + +const { NostrTransportProvider } = await import('../../../transport/NostrTransportProvider'); + +// ============================================================================= +// Helpers +// ============================================================================= + +const TIMEOUT_MS = 50; + +function createProvider() { + return new NostrTransportProvider({ + relays: ['wss://relay1.test'], + // Inert: the (mocked) SDK NostrClient owns its own sockets. + createWebSocket: (() => {}) as unknown as WebSocketFactory, + timeout: TIMEOUT_MS, + autoReconnect: false, + }); +} + +/** Distinct valid secp256k1 private keys — NostrKeyManager is NOT mocked. */ +function identity(n: number) { + return { + privateKey: n.toString(16).padStart(64, '0'), + chainPubkey: `02${n.toString(16).padStart(64, '0')}`, + }; +} + +/** Read the provider's private client field — the identity of the survivor. */ +function currentClient(provider: InstanceType): MockClient | null { + return (provider as unknown as { nostrClient: MockClient | null }).nostrClient; +} + +/** + * A client is leaked when it is neither disposed nor the provider's current + * client: nothing can ever reach it again, and its socket stays open. + */ +function leakedClients(provider: InstanceType): number[] { + const current = currentClient(provider); + return clients + .filter((c) => c.disconnect.mock.calls.length === 0 && c !== current) + .map((c) => c.id); +} + +// ============================================================================= +// Tests +// ============================================================================= + +describe('NostrTransportProvider — setIdentity() client swap', () => { + beforeEach(() => { + vi.clearAllMocks(); + clients.length = 0; + rejectConnectFor.clear(); + hangConnectFor.clear(); + throwSubscribeFor.clear(); + }); + + afterEach(() => { + vi.useRealTimers(); + }); + + it('disposes the replacement and keeps the old client when connect REJECTS', async () => { + const provider = createProvider(); + await provider.connect(); // client 0 (temp key) + await provider.setIdentity(identity(1)); // client 1 — the working client + expect(currentClient(provider)).toBe(clients[1]); + + rejectConnectFor.add(2); + await expect(provider.setIdentity(identity(2))).rejects.toThrow(/relay refused/); + + expect(clients).toHaveLength(3); + expect(clients[2].disconnect).toHaveBeenCalledTimes(1); // replacement disposed + expect(clients[1].disconnect).not.toHaveBeenCalled(); // old client untouched + expect(currentClient(provider)).toBe(clients[1]); // provider still usable + }); + + it('disposes the replacement and keeps the old client when connect TIMES OUT', async () => { + const provider = createProvider(); + await provider.connect(); + await provider.setIdentity(identity(1)); + + hangConnectFor.add(2); + await expect(provider.setIdentity(identity(2))).rejects.toThrow(/timed out/); + + expect(clients).toHaveLength(3); + expect(clients[2].disconnect).toHaveBeenCalledTimes(1); + expect(clients[1].disconnect).not.toHaveBeenCalled(); + expect(currentClient(provider)).toBe(clients[1]); + }); + + it('disposes the OLD client when subscribeToEvents throws after the swap', async () => { + const provider = createProvider(); + await provider.connect(); + await provider.setIdentity(identity(1)); + + throwSubscribeFor.add(2); + await expect(provider.setIdentity(identity(2))).rejects.toThrow(/subscribe failed/); + + expect(clients).toHaveLength(3); + expect(clients[1].disconnect).toHaveBeenCalledTimes(1); // swap committed → old goes + expect(clients[2].disconnect).not.toHaveBeenCalled(); // the new one is live + expect(currentClient(provider)).toBe(clients[2]); + }); + + it('leaks nothing across repeated failures and a later success', async () => { + const provider = createProvider(); + await provider.connect(); + await provider.setIdentity(identity(1)); + + // Two failed retries in a row: setIdentity never touches `status`, so the + // swap branch is re-entered every time — the pre-fix code leaked one client + // per attempt. + rejectConnectFor.add(2); + await expect(provider.setIdentity(identity(2))).rejects.toThrow(); + hangConnectFor.add(3); + await expect(provider.setIdentity(identity(3))).rejects.toThrow(); + // ...then a successful swap, and a failure after the swap point. + await provider.setIdentity(identity(4)); + throwSubscribeFor.add(5); + await expect(provider.setIdentity(identity(5))).rejects.toThrow(); + + expect(clients).toHaveLength(6); + expect(leakedClients(provider)).toEqual([]); + // Exactly one survivor: every other client is disconnected. + const alive = clients.filter((c) => c.disconnect.mock.calls.length === 0); + expect(alive).toEqual([currentClient(provider)]); + // ...and no client is disposed twice. + for (const c of clients) { + expect(c.disconnect.mock.calls.length).toBeLessThanOrEqual(1); + } + }); + + it('leaves no pending connect-deadline timer behind', async () => { + vi.useFakeTimers(); + const provider = createProvider(); + + await provider.connect(); // connect()'s own deadline race + expect(vi.getTimerCount()).toBe(0); + + await provider.setIdentity(identity(1)); // setIdentity()'s deadline race + expect(vi.getTimerCount()).toBe(0); + }); +}); diff --git a/transport/NostrTransportProvider.ts b/transport/NostrTransportProvider.ts index ce9ea84a..263dc7df 100644 --- a/transport/NostrTransportProvider.ts +++ b/transport/NostrTransportProvider.ts @@ -304,14 +304,10 @@ export class NostrTransportProvider implements TransportProvider { }); // Connect to all relays (with timeout to prevent indefinite hang) - await Promise.race([ - this.nostrClient.connect(...this.config.relays), - new Promise((_, reject) => - setTimeout(() => reject(new Error( - `Transport connection timed out after ${this.config.timeout}ms` - )), this.config.timeout) - ), - ]); + await this.connectWithDeadline( + this.nostrClient, + `Transport connection timed out after ${this.config.timeout}ms` + ); // Need at least one successful connection if (!this.nostrClient.isConnected()) { @@ -332,6 +328,28 @@ export class NostrTransportProvider implements TransportProvider { } } + /** + * Race one client's relay connect against the configured timeout. + * + * The timer is cleared in a `finally` on EVERY path. `Promise.race` does not + * cancel the loser: an un-cleared `setTimeout` keeps Node's event loop pinned + * for the whole `config.timeout` after the call has already returned (and + * then rejects a promise nobody is listening to any more). + */ + private async connectWithDeadline(client: NostrClient, timeoutMessage: string): Promise { + let timer: ReturnType | undefined; + try { + await Promise.race([ + client.connect(...this.config.relays), + new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error(timeoutMessage)), this.config.timeout); + }), + ]); + } finally { + if (timer !== undefined) clearTimeout(timer); + } + } + async disconnect(): Promise { if (this.nostrClient) { this.nostrClient.disconnect(); @@ -496,8 +514,12 @@ export class NostrTransportProvider implements TransportProvider { logger.debug('Nostr', 'Identity changed while connected - recreating NostrClient'); const oldClient = this.nostrClient; - // Create new client with real identity - this.nostrClient = new NostrClient(this.keyManager, { + // Build the replacement into a LOCAL: the field must not move until the + // new client is connected. Swapping first and then failing the connect + // orphans `oldClient` — socket open but unreachable from the provider, so + // `disconnect()` cannot reach it either — and setIdentity never touches + // `status`, so the caller's next retry re-enters here and leaks another. + const nextClient = new NostrClient(this.keyManager, { autoReconnect: this.config.autoReconnect, reconnectIntervalMs: this.config.reconnectDelay, maxReconnectIntervalMs: this.config.reconnectDelay * 16, @@ -506,7 +528,7 @@ export class NostrTransportProvider implements TransportProvider { }); // Add connection event listener - this.nostrClient.addConnectionListener({ + nextClient.addConnectionListener({ onConnect: (url) => { logger.debug('Nostr', 'NostrClient connected to relay:', url); }, @@ -521,17 +543,29 @@ export class NostrTransportProvider implements TransportProvider { }, }); - // Connect with new identity, set up subscriptions, then disconnect old client - await Promise.race([ - this.nostrClient.connect(...this.config.relays), - new Promise((_, reject) => - setTimeout(() => reject(new Error( - `Transport reconnection timed out after ${this.config.timeout}ms` - )), this.config.timeout) - ), - ]); - await this.subscribeToEvents(); - oldClient.disconnect(); + try { + await this.connectWithDeadline( + nextClient, + `Transport reconnection timed out after ${this.config.timeout}ms` + ); + } catch (error) { + // Dispose ONLY the replacement; the provider stays on the working + // `oldClient`. NOT both: MultiAddressTransportMux SHARES this client + // (ensureTransportMux suppresses subscriptions and reuses the socket), + // so killing it over a transient relay timeout kills the mux's too. + try { nextClient.disconnect(); } catch { /* best-effort cleanup */ } + throw error; + } + + // The swap is committed. From here on `oldClient` is unreachable from the + // provider, so it must be disposed on EVERY path out — including a + // throwing `subscribeToEvents()`. + this.nostrClient = nextClient; + try { + await this.subscribeToEvents(); + } finally { + try { oldClient.disconnect(); } catch { /* best-effort cleanup */ } + } } else if (this.isConnected()) { // Already connected with right key, just subscribe await this.subscribeToEvents(); From ee440620bce1881ecee421c2dc2c0d6e5a65ce1e Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:47:07 +0200 Subject: [PATCH 10/43] docs: correct four CLAUDE.md claims the code does not honour (#769.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Each verified against source, not inferred: - 'receive seen-set' in the durable pv2g2 KV: there is none. STORE_KEYS lists the complete inventory and has no such entry; the (tokenId, stateHash) dedup runs against heldStates, an in-memory Map built per composition and seeded from the inventory view, backed by the server-side history dedupKey. The KV list was also missing checkpoints, shortfalls, the epoch latch and the suspectedSpent/knownSpends overlays — it now points at STORE_KEYS as the source of truth. - 'short symbols resolved via registry' on the money path: they are not. getCoinIdBySymbol/normalizeCoinId have zero call sites in modules/payments-v2/ or core/Sphere.ts. mint() rejects non-hex outright, send() byte-compares, and requests.create passes coinId straight through — so a reader following the quickstart's coinId: 'UCT' gets a rejection from one and a silent no-match from the others. Three sites carried it. - 'the facade consumes it for short-symbol -> coinId resolution': presentation only. - 'configured both by provider factories and by Sphere itself': #767 removed the factory calls; a Sphere now owns its registry and disposes it. The symbol one is not cosmetic — it is why a registry finding was initially mis-scoped as presentation-only during the #767 review. --- CLAUDE.md | 42 ++++++++++++++++++++++++++++-------------- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1d8573c6..2d7303ae 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -154,13 +154,18 @@ const filtered = sphere.payments.tokens({ coinId: '...' }); const result = await sphere.payments.send({ recipient: '@bob', // @nametag, DIRECT://..., or chain pubkey (02...) amount: '1000000', // in smallest unit (string) - coinId: 'UCT', // coin ID (64-hex canonical; short symbols resolved via registry) + coinId: coinIdHex, // coin ID — 64-hex ONLY on the money path (see the note below) memo: 'Payment for coffee', // optional (recipient-encrypted envelope) }); // result: TransferResult { id, status, tokens, tokenTransfers, error?, // deliveryPending?, deliveryState? } // status: 'pending' | 'submitted' | 'confirmed' | 'delivered' | 'completed' | 'failed' // deliveryPending: certified on-chain but mailbox deposit still owed — NOT a failure. +// NOTE: coinId is NOT symbol-resolved on the money path. `getCoinIdBySymbol` / +// `normalizeCoinId` have zero call sites in modules/payments-v2/ or core/Sphere.ts: +// mint() rejects non-hex outright and send() byte-compares, so passing 'UCT' gets +// a rejection from mint() and a silent no-match from send(). Resolve the symbol +// yourself via the registry first. The registry is presentation only. // 7. Receive: the facade drains the wallet-api mailbox continuously while // started; receive() is an explicit one-shot drain (returns what landed). @@ -190,7 +195,7 @@ const addresses = sphere.getActiveAddresses(); // TrackedAddress[] // 12. Payment requests (wallet-api rail; encrypted memo envelope) const req = await sphere.payments.requests.create('@bob', { - coinId: 'UCT', amount: '1000000', memo: 'Pay for order #1234', + coinId: coinIdHex, amount: '1000000', memo: 'Pay for order #1234', // 64-hex, not a symbol }); sphere.on('payment_request:incoming', (view) => { // PaymentRequestView: { id, requestId, senderPubkey, senderNametag?, amount, @@ -372,7 +377,7 @@ sphere-sdk/ │ │ ├── restore.ts # §5.1 epoch-change reseed (see the KV generation note below) │ │ ├── machine/ # TransferMachine + resume (E.2/E.4) + intent journal │ │ ├── select/ # CoinSelector, Reservations ledger, op queue -│ │ ├── receive/ # Mailbox drain, seen-set, claim, verified-before-balance +│ │ ├── receive/ # Mailbox drain, claim, verified-before-balance │ │ ├── requests/ # Payment requests (streams + settling journal) │ │ ├── history/ # Server read-through paged history │ │ ├── inventory/ # InventoryView + Asset presentation (legacy shape held) @@ -419,7 +424,7 @@ Subpath exports: `.` (root), `./core`, `./token-engine`, `./payments-v2` (the fa - **Server is the record.** Token inventory, blobs, transfer intents, mailbox, history and payment requests live in the wallet-api backend. The client holds keys, a per-address scoped KV (`pv2g2:{network}:{chainPubkey}:*` in the plain `StorageProvider`) with the refresh token, - cursors, seen-set and journals — nothing else. There is no client token store to sync + cursors and journals — nothing else. There is no client token store to sync (`sphere.sync()` is gone). - **Composition:** `Sphere.init({ walletApi })` → `resolvePaymentsV2Composition` → `composePaymentsV2` builds one `PaymentsFacade` per active address over the wallet-api-v2 @@ -436,7 +441,7 @@ Subpath exports: `.` (root), `./core`, `./token-engine`, `./payments-v2` (the fa send); a clean conflict with a demoted source triggers one bounded re-plan (#625 marks the source `suspectedSpent`, excluded from selection, recoverable by resync). - **Receive:** continuous mailbox drain while started + explicit `receive()`. Every incoming - token is engine-verified and ownership-checked BEFORE entering the balance; seen-set dedup by + token is engine-verified and ownership-checked BEFORE entering the balance; dedup by **(tokenId, stateHash)** — genesis id alone would refuse a token legitimately re-acquired at a later state; store-before-ack (a crash between store and claim re-claims, never loses). - **Delivery journal (#621):** a certified-but-undelivered blob is journaled in the scoped KV @@ -540,7 +545,7 @@ interface FullIdentity extends Identity { interface SendRequest { // sphere.payments.send() recipient: string; // @nametag, DIRECT://..., chain pubkey amount: string; // Amount in smallest unit - coinId: string; // Coin ID (64-hex canonical; short symbols resolved via registry) + coinId: string; // Coin ID — even-length lowercase hex; NOT symbol-resolved memo?: string; // Optional message (recipient-encrypted envelope) } @@ -692,8 +697,11 @@ authoritative for build success. while started; `receive()` for an explicit one-shot). - **Verified before entering the balance:** `engine.verify` (full trust-base proof check) + `engine.isOwnedBy(token, own chainPubkey)`; failures are rejected (warn log). Dedup by - **(tokenId, stateHash)** via the durable seen-set — keyed on the genesis id alone, a token sent - away and legitimately received back (A→B→A) would be dropped as a duplicate. Store-before-ack: + **(tokenId, stateHash)** — keyed on the genesis id alone, a token sent away and legitimately + received back (A→B→A) would be dropped as a duplicate. There is **no durable seen-set**: the + comparison is against `heldStates`, an in-memory `Map` built per composition + (`modules/payments-v2/compose.ts`) and seeded from the inventory view, backed by the server-side + history `dedupKey`. Store-before-ack: the token is stored before the mailbox claim is acknowledged, so a crash re-claims instead of losing. @@ -727,16 +735,22 @@ authoritative for build success. icons) by coin ID. No bundled data — remote URL per network (`NETWORKS[network].tokenRegistryUrl`; testnet/testnet2 use `unicity-ids.testnet2.json`) + persistent cache. -- The facade consumes it for Asset presentation and short-symbol → coinId resolution. -- Configured both by provider factories and by `Sphere` itself (tsup bundles - duplicate the singleton per entry point — both bundle contexts need `configure()`). +- The facade consumes it for Asset presentation ONLY. It resolves no symbols on the money + path — `getCoinIdBySymbol`/`normalizeCoinId` have zero call sites in `modules/payments-v2/` + or `core/Sphere.ts`. +- A `Sphere` builds and OWNS its registry (#767), disposed by `sphere.destroy()`. The provider + factories no longer call `TokenRegistry.configure()` — in the published package they are + separate tsup bundles with separate singleton copies, so that call wrote to an object no + consumer could read. ### Durable client state (the complete inventory — design §6) - Everything the client persists for money lives in the per-(network, address) scoped KV: `pv2g2:{network}:{chainPubkey}:*` inside the plain `StorageProvider` — refresh token, sync - cursors, receive seen-set, intent backstop, delivery journal (#621), mint journal, request - settling journal. One writer per store. Being self-prefixed with the network, it never rides - the legacy `isNetworkScopedAddressKey` mechanism (which still guards the remaining + cursors, intent backstop, split-checkpoint cache, delivery journal (#621), mint journal, + #690 shortfalls, request settling journal, the epoch latch and the §5.2 `suspectedSpent` / + `knownSpends` overlays — the complete list is `STORE_KEYS` in `modules/payments-v2/stores.ts`, + and it contains no receive seen-set. One writer per store. Being self-prefixed with the + network, it never rides the legacy `isNetworkScopedAddressKey` mechanism (which still guards the remaining chat/identity keys in the platform storage providers). - **The `pv2:` → `pv2g2:` rename IS the 3.x local migration** (`modules/payments-v2/stores.ts`; `sweepSupersededState()` clears the old prefix once per composition, from From a328052c91af9b22ba7cfc6d9e5cd8c838315d45 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:48:08 +0200 Subject: [PATCH 11/43] fix(token-engine): dispose() must settle the in-flight verification, by REJECTING (#770.4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The SDK 3.0.1 WorkerPool.dispose() only terminate()s its workers. A dispatched task resolves solely from worker.onmessage, which a terminated worker never posts, and the queue is never drained — so WorkerTokenVerifier.verify, which awaits Promise.all(pool.run(...)), hangs forever. pool is private readonly upstream, so the settle lives in our subclass. Reachable: setOracleApiKey -> PaymentsFacade.setEngine disposes the replaced engine while a receive drain may be mid-verify. It REJECTS rather than resolving { ok: false }, and that is the load-bearing part. The only engine.verify caller in the money path is Receive.screen(), where a falsy verdict is a PERMANENT rejectAck(entry, 'invalid') — so resolving would throw away a VALID incoming token because an api-key change landed mid-drain. A rejection takes the drain's catch instead: the entry stays unacked and re-lists. (Confirmed end to end: SphereError carries no .retryable, so isRetryableAckError leaves it on the normal path.) createWorker() also refuses once disposed — the SDK leaves its workers and idle arrays populated, so a task slipping in afterwards could resurrect the pool, or be handed an already-terminated worker that will never answer. Two stale comments claiming in-flight ops finish on the old engine corrected (the core/Sphere.ts one is in the next commit — that file was held by another change). --- modules/payments-v2/PaymentsFacade.ts | 3 +- .../token-engine/worker-verification.test.ts | 211 +++++++++++++++++- token-engine/factory.ts | 68 ++++++ 3 files changed, 280 insertions(+), 2 deletions(-) diff --git a/modules/payments-v2/PaymentsFacade.ts b/modules/payments-v2/PaymentsFacade.ts index fc702d34..c1e1f469 100644 --- a/modules/payments-v2/PaymentsFacade.ts +++ b/modules/payments-v2/PaymentsFacade.ts @@ -204,7 +204,8 @@ export class PaymentsFacade implements PaymentsV2 { } } - /** Swaps what FUTURE operations snapshot; in-flight ops finish on the old engine. */ + /** Swaps what FUTURE operations snapshot. Chain ops finish on the old engine, but it is + * DISPOSED here, so one mid-`verify()` is cancelled — MODULE_DESTROYED, #770(4). */ setEngine(next: ITokenEngine): void { const previous = this.currentEngine ?? this.deps.engineRef(); this.currentEngine = next; diff --git a/tests/unit/token-engine/worker-verification.test.ts b/tests/unit/token-engine/worker-verification.test.ts index b7459e9f..5e6ff488 100644 --- a/tests/unit/token-engine/worker-verification.test.ts +++ b/tests/unit/token-engine/worker-verification.test.ts @@ -12,11 +12,13 @@ * (WorkerTokenVerifierTest there); reproducing it would need a real token with * real inclusion proofs, i.e. the live e2e path. */ -import { describe, expect, it, vi } from 'vitest'; +import { beforeAll, describe, expect, it, vi } from 'vitest'; import { createSphereTokenEngine, createWorkerTokenVerifier } from '../../../token-engine/factory'; import type { SphereToken, VerificationWorker } from '../../../token-engine'; import { SigningService, VerificationStatus, WorkerTokenVerifier } from '../../../token-engine/sdk'; +import { TestAggregatorClient } from './support/TestAggregatorClient'; +import { createTestEngine, freshPubkey } from './test-engine'; /** Parses fine, touches no network (AggregatorClient connects on first request). */ const TRUST_BASE_JSON = { @@ -137,3 +139,210 @@ describe('engine.verify routing', () => { expect(() => engine.dispose?.()).not.toThrow(); }); }); + +/** + * #770 item 4 — `dispose()` must SETTLE the in-flight verification batch. + * + * The SDK's `WorkerPool.dispose()` (3.0.1) calls `worker.terminate()` on every + * worker and nothing else. A dispatched batch resolves ONLY from + * `worker.onmessage`, which a terminated worker never posts, and queued batches + * are never drained — so `WorkerTokenVerifier.verify`, which awaits + * `Promise.all(pool.run(...))`, hangs FOREVER. `pool` is `private readonly` + * upstream, so the settle lives in our subclass (`token-engine/factory.ts`). + * + * Production trigger: `Sphere.setOracleApiKey` → `PaymentsFacade.setEngine` + * disposes the engine it replaced, and a receive drain may be mid-`verify()`. + * + * These tests use a REAL certified token (in-memory aggregator) because the pool + * is only reached after the genesis verifies on the calling thread — a fake token + * never gets that far, so it could not observe the hang at all. + */ +describe('dispose() during an in-flight verification (#770 item 4)', () => { + const COIN = 'a'.repeat(64); + + /** Accepts a batch and NEVER answers — exactly what a terminated worker does. */ + class SilentWorker implements VerificationWorker { + onerror: ((event: { message: string }) => void) | null = null; + onmessage: ((event: { data: unknown }) => void) | null = null; + terminated = false; + posted = 0; + /** Runs inside `WorkerPool.dispatch()`, before it acquires the NEXT worker. */ + onPost: (() => void) | null = null; + + postMessage(): void { + this.posted += 1; + this.onPost?.(); + } + + terminate(): void { + this.terminated = true; + } + } + + type Settled = + | { state: 'fulfilled'; value: unknown } + | { state: 'rejected'; reason: unknown } + | { state: 'pending' }; + + /** Never waits unboundedly: a hang reports as `pending` instead of a suite timeout. */ + async function settleWithin(promise: Promise, ms: number): Promise { + return Promise.race([ + promise.then( + (value): Settled => ({ state: 'fulfilled', value }), + (reason): Settled => ({ state: 'rejected', reason }) + ), + new Promise((resolve) => setTimeout(() => resolve({ state: 'pending' }), ms)), + ]); + } + + async function until(predicate: () => boolean, what: string, ms = 5000): Promise { + const deadline = Date.now() + ms; + while (!predicate()) { + if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`); + await new Promise((resolve) => setTimeout(resolve, 1)); + } + } + + /** One real chain: a token with 1 transfer, one with 2, and the trust base that certified them. */ + let trustBaseJson: unknown; + let mintedOnly: SphereToken; + let oneTransfer: SphereToken; + let twoTransfers: SphereToken; + + beforeAll(async () => { + const aggregator = TestAggregatorClient.create(); + const alice = createTestEngine({ aggregator }); + const bob = createTestEngine({ aggregator }); + mintedOnly = await alice.mint({ + recipientPubkey: alice.getIdentity().chainPubkey, + value: { assets: [{ coinId: COIN, amount: 100n }] }, + }); + oneTransfer = await alice.transfer({ token: mintedOnly, recipientPubkey: bob.getIdentity().chainPubkey }); + twoTransfers = await bob.transfer({ token: oneTransfer, recipientPubkey: freshPubkey() }); + trustBaseJson = aggregator.rootTrustBase.toJSON(); + }, 30000); + + const buildEngine = async ( + createWorker: () => VerificationWorker, + poolSize?: number + ): ReturnType => + createSphereTokenEngine({ + aggregatorUrl: 'http://localhost:3000', + privateKey: SigningService.generatePrivateKey(), + trustBaseJson, + verification: { createWorker, ...(poolSize !== undefined ? { poolSize } : {}) }, + }); + + it('settles the in-flight verification instead of hanging forever', async () => { + const spawned: SilentWorker[] = []; + const engine = await buildEngine(() => { + const worker = new SilentWorker(); + spawned.push(worker); + return worker; + }); + + const verifying = engine.verify(oneTransfer); + await until(() => spawned.some((w) => w.posted > 0), 'the pool to dispatch a batch'); + + engine.dispose?.(); + + // Without the cancellation this never settles: the batch's only resolver is + // the onmessage of a worker that was just terminated. + const outcome = await settleWithin(verifying, 3000); + expect(outcome.state).not.toBe('pending'); + }, 30000); + + it('REJECTS the cancelled verification — never resolves { ok: false }', async () => { + const spawned: SilentWorker[] = []; + const engine = await buildEngine(() => { + const worker = new SilentWorker(); + spawned.push(worker); + return worker; + }); + + const verifying = engine.verify(oneTransfer); + await until(() => spawned.some((w) => w.posted > 0), 'the pool to dispatch a batch'); + engine.dispose?.(); + + const outcome = await settleWithin(verifying, 3000); + // THE money pin. The only engine.verify caller in the vertical is + // modules/payments-v2/receive/Receive.ts `screen()`: + // + // const verdict = await engine.verify(token); + // if (!verdict.ok) return { kind: 'ack', ack: rejectAck(entry, 'invalid') }; + // + // A cancellation that RESOLVED `{ ok: false }` would PERMANENTLY reject a + // valid incoming token at the mailbox, just because an api-key change landed + // mid-drain. Rejecting instead reaches the drain's catch, which leaves the + // entry unacked so it re-lists on the next drain. + expect(outcome.state).toBe('rejected'); + expect(outcome).not.toMatchObject({ state: 'fulfilled' }); + expect(outcome).toMatchObject({ reason: { code: 'MODULE_DESTROYED' } }); + }, 30000); + + it('terminates every spawned worker, and a later verify() rejects without spawning one', async () => { + const spawned: SilentWorker[] = []; + const createWorker = vi.fn(() => { + const worker = new SilentWorker(); + spawned.push(worker); + return worker; + }); + const engine = await buildEngine(createWorker); + + const verifying = engine.verify(oneTransfer); + await until(() => spawned.some((w) => w.posted > 0), 'the pool to dispatch a batch'); + engine.dispose?.(); + await settleWithin(verifying, 3000); + + expect(spawned.length).toBeGreaterThan(0); + expect(spawned.every((w) => w.terminated)).toBe(true); + + const spawnsBeforeSecondVerify = createWorker.mock.calls.length; + const afterDispose = await settleWithin(engine.verify(oneTransfer), 3000); + expect(afterDispose.state).toBe('rejected'); + expect(afterDispose).toMatchObject({ reason: { code: 'MODULE_DESTROYED' } }); + // The SDK's dispose() leaves its `workers` array populated but its `idle` list + // holding TERMINATED workers, so a post-dispose task would happily call + // createWorker() and resurrect the pool it just tore down. + expect(createWorker).toHaveBeenCalledTimes(spawnsBeforeSecondVerify); + }, 30000); + + it('rejects a post-dispose verify even for a token that needs no worker at all', async () => { + const createWorker = vi.fn(() => new SilentWorker()); + const engine = await buildEngine(createWorker); + engine.dispose?.(); + + // A 0-transfer token never reaches the pool, so no other guard can notice the + // teardown: without the `disposed` gate on verify(), a torn-down engine keeps + // handing out verdicts as if it were live. + const outcome = await settleWithin(engine.verify(mintedOnly), 3000); + expect(outcome.state).toBe('rejected'); + expect(outcome).toMatchObject({ reason: { code: 'MODULE_DESTROYED' } }); + expect(createWorker).not.toHaveBeenCalled(); + }, 30000); + + it('a batch dispatched AFTER dispose() cannot resurrect the pool', async () => { + // Deterministic ordering, no timing guess: two transfers + poolSize 2 means + // WorkerPool.dispatch() loops twice. The FIRST worker calls dispose() from + // inside postMessage — i.e. mid-loop, before acquire() runs for the second + // batch. Without the createWorker() guard, that acquire spawns a live worker + // that dispose() has already walked past, leaking a thread per api-key change. + const spawned: SilentWorker[] = []; + let engineRef: { dispose?: () => void } | null = null; + const createWorker = vi.fn(() => { + const worker = new SilentWorker(); + if (spawned.length === 0) worker.onPost = (): void => engineRef?.dispose?.(); + spawned.push(worker); + return worker; + }); + + const engine = await buildEngine(createWorker, 2); + engineRef = engine; + + const outcome = await settleWithin(engine.verify(twoTransfers), 5000); + expect(outcome.state).toBe('rejected'); + expect(createWorker).toHaveBeenCalledTimes(1); + expect(spawned).toHaveLength(1); + expect(spawned[0].terminated).toBe(true); + }, 30000); +}); diff --git a/token-engine/factory.ts b/token-engine/factory.ts index de078b3d..4e947d43 100644 --- a/token-engine/factory.ts +++ b/token-engine/factory.ts @@ -38,13 +38,43 @@ import type { EngineConfig, ITokenEngine, VerificationWorker, VerificationWorker const DEFAULT_VERIFICATION_POOL_SIZE = 4; +/** #770(4): what a verification cancelled by `dispose()` rejects with. */ +function disposedError(): SphereError { + return new SphereError( + 'Verification worker pool disposed — the in-flight verification was cancelled', + 'MODULE_DESTROYED', + ); +} + /** * The consumer's worker factory, bound to the base SDK's pool verifier. The SDK * leaves `createWorker()` abstract so the platform choice stays with the consumer; * the cast is the port boundary (same web-`Worker` subset, payloads `unknown` on * our side so no SDK wire type escapes). + * + * ── #770(4): dispose() must SETTLE the in-flight batch ────────────────────── + * The SDK's `WorkerPool.dispose()` (3.0.1) only calls `worker.terminate()` on + * every worker it spawned. A dispatched task resolves ONLY from `worker.onmessage`, + * which a terminated worker never posts, and queued tasks are never drained — so + * `WorkerTokenVerifier.verify`, which awaits `Promise.all(pool.run(...))`, hangs + * FOREVER. `pool` is `private readonly` upstream, so the settle has to live here. + * + * Reachable in production: `Sphere.setOracleApiKey` → `PaymentsFacade.setEngine` + * disposes the engine it replaced while a receive drain may be mid-`verify`. + * + * The cancellation MUST REJECT — never resolve `{ ok: false }`. The only + * `engine.verify` caller in the money path is `modules/payments-v2/receive/Receive.ts` + * (`screen()`, the `const verdict = await engine.verify(token)` line): a falsy + * verdict there is a PERMANENT `rejectAck(entry, 'invalid')`, i.e. a VALID + * incoming token thrown away at the mailbox because an api-key change happened + * to land mid-drain. A rejection instead propagates to the drain's catch, which + * leaves the entry UNACKED so it re-lists on the next drain. */ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { + private disposed = false; + /** Lazily created so an idle verifier holds no promise at all. */ + private cancellation: { promise: Promise; reject: (error: unknown) => void } | null = null; + public constructor( private readonly spawn: () => VerificationWorker, poolSize: number @@ -52,9 +82,47 @@ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { super(poolSize); } + /** Rejects (see the class note) if `dispose()` lands before the pool answers. */ + public override verify( + ...args: Parameters + ): ReturnType { + if (this.disposed) return Promise.reject(disposedError()); + return Promise.race([super.verify(...args), this.cancelSignal()]); + } + + /** Idempotent — Sphere.setOracleApiKey disposes the replaced engine twice (facade + caller). */ + public override dispose(): void { + if (this.disposed) return; + this.disposed = true; + super.dispose(); // terminate() every spawned worker, as before + // Only AFTER the pool is down: settle whatever the terminated workers will + // now never answer. A no-op when nothing was ever verified. + this.cancellation?.reject(disposedError()); + } + protected createWorker(): IWorker { + // The SDK's dispose() leaves its `workers` array populated, so a task that + // slips in afterwards would call acquire() → createWorker() and RESURRECT + // the pool it just tore down. Fail instead. + if (this.disposed) throw disposedError(); return this.spawn() as unknown as IWorker; } + + private cancelSignal(): Promise { + if (this.cancellation === null) { + let reject!: (error: unknown) => void; + const promise = new Promise((_resolve, rejectFn) => { + reject = rejectFn; + }); + // Belt and braces: today the only caller hands this straight to + // `Promise.race`, which subscribes to it, so dispose()'s rejection is + // always observed. Park a no-op handler anyway so a future caller that + // drops the promise cannot turn a teardown into an unhandled rejection. + void promise.catch(() => undefined); + this.cancellation = { promise, reject }; + } + return this.cancellation.promise; + } } /** Workers spawn LAZILY on first verify and are reused, so this costs nothing to build. */ From 88a159080f249568708a214af0220c90cb831e10 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:50:05 +0200 Subject: [PATCH 12/43] docs(changelog): the registry entry's 'not fixed here' note is now stale Sphere.static instance and the clear()/import() cross-wallet kill were open when #767 shipped; this release fixes both, two sections above. --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fd84fe42..7a845a24 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -97,8 +97,8 @@ coin. See [docs/MIGRATION-TOKEN-REGISTRY.md](docs/MIGRATION-TOKEN-REGISTRY.md). - The ten singleton-bound free functions moved to `registry/global-readers.ts`, re-exported unchanged. The public surface is identical: same 126 root exports, same signatures. -Not fixed here: `Sphere`'s own `static instance`, and `Sphere.clear()`/`import()` destroying -whichever instance holds it regardless of the storage they were given. Tracked in #766. +`Sphere`'s own `static instance` and the `clear()`/`import()` cross-wallet kill were still +open when that shipped; both are fixed above in this release. ### Added — mainnet is a runnable network From 561ea857b416003dce190388222c1b85668c7c29 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:53:08 +0200 Subject: [PATCH 13/43] test(token-engine): prove the engine's NetworkId really comes from the trust base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The existing case built an engine with trustBaseJson.networkId = 4 and then asserted only that chainPubkey was a Uint8Array — i.e. that construction succeeded. An engine that ignored the trust base entirely and defaulted to a fixed id would have passed it just as well, so nothing pinned the claim CLAUDE.md makes about the trust base being the single source of the network id. That claim is what keeps mainnet money from verifying against another chain. decodeToken compares the token's genesis network against the engine's and does NOT verify proofs, so it reads the derived id offline against a real minted token. The test decodes the same token on a trust base declaring the minting network (resolves) and on one declaring mainnet (rejects, naming both ids). Falsified: hardcoding the engine's id to mainnet fails exactly this test. --- tests/unit/token-engine/factory.test.ts | 32 ++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/tests/unit/token-engine/factory.test.ts b/tests/unit/token-engine/factory.test.ts index a17687d1..44e3ab8f 100644 --- a/tests/unit/token-engine/factory.test.ts +++ b/tests/unit/token-engine/factory.test.ts @@ -1,8 +1,9 @@ import { describe, expect, it, vi } from 'vitest'; import { createSphereTokenEngine } from '../../../token-engine/factory'; -import { SigningService } from '../../../token-engine/sdk'; +import { NetworkId, SigningService } from '../../../token-engine/sdk'; import { logger } from '../../../core/logger'; +import { createTestEngine } from './test-engine'; // Minimal single-node trust base (sigKey = a valid compressed pubkey). Parses fine; // no network is touched (AggregatorClient connects lazily, on the first request). @@ -42,6 +43,35 @@ describe('createSphereTokenEngine', () => { expect(engine.getIdentity().chainPubkey).toBeInstanceOf(Uint8Array); }); + it('the trust base is the SINGLE SOURCE of the network id — a mainnet base yields network 1', async () => { + // Constructing is not evidence: an engine that ignored the trust base entirely and + // defaulted to some fixed id would also construct. decodeToken compares the token's + // genesis network against the engine's (SphereTokenEngine "Token network mismatch"), + // and does NOT verify proofs — so it reads the derived id offline, against a real + // token. mainnet = 1 and testnet2 = 4 are the two ids the SDK actually ships against. + const minted = createTestEngine(); // NetworkId.LOCAL — id 3, which TRUST_BASE_JSON declares + const blob = minted.encodeToken( + await minted.mint({ + recipientPubkey: minted.getIdentity().chainPubkey, + value: { assets: [{ coinId: 'a'.repeat(64), amount: 1n }] }, + }), + ); + + const sameNetwork = await createSphereTokenEngine({ + aggregatorUrl: 'http://localhost:3000', + privateKey: SigningService.generatePrivateKey(), + trustBaseJson: TRUST_BASE_JSON, + }); + await expect(sameNetwork.decodeToken(blob)).resolves.toBeDefined(); + + const onMainnet = await createSphereTokenEngine({ + aggregatorUrl: 'http://localhost:3000', + privateKey: SigningService.generatePrivateKey(), + trustBaseJson: { ...TRUST_BASE_JSON, networkId: NetworkId.MAINNET.id }, + }); + await expect(onMainnet.decodeToken(blob)).rejects.toThrow(/network 3, engine on 1/); + }, 30000); + it('warns when constructed without an apiKey', async () => { const warn = vi.spyOn(logger, 'warn').mockImplementation(() => {}); await createSphereTokenEngine({ From 56334d8006f80724e0fe4d312b8d6b55228f499d Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:55:18 +0200 Subject: [PATCH 14/43] test(mutation): probes for the transport-swap and verifier-dispose guards MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nine new probes across the two #770 fixes and the trust-base networkId derivation. The money one is verifier-cancel-resolves-not-rejects: a cancelled verification that RESOLVES a falsy verdict makes Receive.screen() permanently rejectAck a valid incoming token, so the mutant must not survive. Not yet run as a full suite — the tree is still carrying in-flight work; the whole set runs before the push. --- tests/mutation/probes.json | 90 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 0e7b43f4..56e332a2 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -841,5 +841,95 @@ "tests/unit/impl/backing-store-id.test.ts", "tests/integration/sphere-instance-scoping.test.ts" ] + }, + { + "name": "transport-orphans-old-client-on-failed-swap", + "note": "#770.1: a failed setIdentity connect must dispose ONLY the replacement; without it the half-built client leaks one socket per retry", + "file": "transport/NostrTransportProvider.ts", + "find": " try { nextClient.disconnect(); } catch { /* best-effort cleanup */ }\n throw error;", + "replace": " throw error;", + "tests": [ + "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" + ] + }, + { + "name": "transport-swaps-client-before-connect", + "note": "#770.1: the field must not move until the new client is connected, or a failed swap orphans the old client beyond disconnect()'s reach", + "file": "transport/NostrTransportProvider.ts", + "find": " try {\n await this.connectWithDeadline(\n nextClient,", + "replace": " this.nostrClient = nextClient;\n try {\n await this.connectWithDeadline(\n nextClient,", + "tests": [ + "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" + ] + }, + { + "name": "transport-keeps-old-client-when-subscribe-throws", + "note": "#770.1: after the swap the old client is unreachable, so it must be disposed on EVERY path out of subscribeToEvents", + "file": "transport/NostrTransportProvider.ts", + "find": " try {\n await this.subscribeToEvents();\n } finally {\n try { oldClient.disconnect(); } catch { /* best-effort cleanup */ }\n }", + "replace": " await this.subscribeToEvents();\n try { oldClient.disconnect(); } catch { /* best-effort cleanup */ }", + "tests": [ + "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" + ] + }, + { + "name": "transport-leaves-connect-deadline-timer-pending", + "note": "#770.1: Promise.race does not cancel the loser \u2014 an un-cleared setTimeout pins Node's event loop for config.timeout", + "file": "transport/NostrTransportProvider.ts", + "find": " if (timer !== undefined) clearTimeout(timer);", + "replace": " if (timer === undefined) clearTimeout(timer);", + "tests": [ + "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" + ] + }, + { + "name": "verifier-dispose-no-cancel", + "note": "#770.4: dispose() terminates the workers but the SDK pool never settles their tasks; without the race, verify() hangs forever and stop()/destroy() hang with it", + "file": "token-engine/factory.ts", + "find": " return Promise.race([super.verify(...args), this.cancelSignal()]);", + "replace": " return super.verify(...args);", + "tests": [ + "tests/unit/token-engine/worker-verification.test.ts" + ] + }, + { + "name": "verifier-cancel-resolves-not-rejects", + "note": "#770.4 MONEY: a cancelled verification must REJECT. Resolving a falsy verdict makes Receive.screen() rejectAck(entry,'invalid') \u2014 a VALID incoming token destroyed because an api-key change landed mid-drain", + "file": "token-engine/factory.ts", + "find": " const promise = new Promise((_resolve, rejectFn) => {\n reject = rejectFn;\n });", + "replace": " const promise = new Promise((resolveFn) => {\n reject = resolveFn as unknown as (error: unknown) => void;\n });", + "tests": [ + "tests/unit/token-engine/worker-verification.test.ts" + ] + }, + { + "name": "verifier-post-dispose-gate", + "note": "#770.4: a verify() started after dispose() must reject \u2014 a token needing no worker would otherwise be answered by a torn-down engine", + "file": "token-engine/factory.ts", + "find": " if (this.disposed) return Promise.reject(disposedError());", + "replace": " // probe", + "tests": [ + "tests/unit/token-engine/worker-verification.test.ts" + ] + }, + { + "name": "verifier-pool-resurrection", + "note": "#770.4: the SDK leaves workers/idle populated after dispose(), so an unguarded createWorker() resurrects the pool or hands back a terminated worker", + "file": "token-engine/factory.ts", + "find": " if (this.disposed) throw disposedError();", + "replace": " // probe", + "tests": [ + "tests/unit/token-engine/worker-verification.test.ts" + ] + }, + { + "name": "factory-networkid-not-from-trustbase", + "note": "#764/#765: the trust base is the SINGLE source of the engine's NetworkId. Hardcoding it (here: mainnet 1) makes a wallet verify money against the wrong chain \u2014 factory.test.ts decodes a real token to read the derived id offline", + "file": "token-engine/factory.ts", + "find": " networkId: trustBase.networkId,", + "replace": " networkId: trustBase.networkId.constructor.fromId(1),", + "tests": [ + "tests/unit/token-engine/factory.test.ts" + ] } ] From 6e84b9bec06b5dd0454a9b1ccccd37ad4388b43d Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 01:58:08 +0200 Subject: [PATCH 15/43] fix(payments-v2): the mailbox poll drain must hold stop() open (#770.2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Receive.start() spawned both its wake callback and its 30s poll with a bare `void this.drainOnce()`, registering neither with the facade's quiescence gate. So PaymentsFacade.stop() could return while a drain was still doing wallet-api I/O, engine.verify, scoped-KV writes and transfer:incoming emission — and destroy() carried on tearing down underneath it. The wake path was accidentally covered: the facade subscribes 'mailbox' at the top of start() and tracks the SingleFlight promise the port's onWake bridge later joins. That held only because the facade subscribes before receiveLoop.start() runs. The poll had nothing watching it at all. Both now go through one spawn helper that registers the op via the existing FacadeHooks.track. No ordering change in stop(): track() inserts synchronously and stop() is synchronous through receiveLoop.stop(), so a drain in flight at stop time is already registered and the interval can produce no new one. Ordered deliberately after a328052c — tracking a drain wedged in a worker-pool verify would otherwise have turned a silent leak into a permanent hang of stop(). --- modules/payments-v2/compose.ts | 2 ++ modules/payments-v2/receive/Receive.ts | 16 +++++---- tests/unit/payments-v2/facade-harness.ts | 13 +++++-- tests/unit/payments-v2/facade.test.ts | 44 ++++++++++++++++++++++++ 4 files changed, 65 insertions(+), 10 deletions(-) diff --git a/modules/payments-v2/compose.ts b/modules/payments-v2/compose.ts index 78002a3b..9ea8e7f4 100644 --- a/modules/payments-v2/compose.ts +++ b/modules/payments-v2/compose.ts @@ -321,6 +321,8 @@ function buildReceive( ); }, syncEpoch: deps.syncEpoch, + // #770: the poll/wake drains Receive spawns itself must hold stop() too. + track: hooks.track, ...(deps.now !== undefined ? { now: deps.now } : {}), }); } diff --git a/modules/payments-v2/receive/Receive.ts b/modules/payments-v2/receive/Receive.ts index 6d46bdc9..00381320 100644 --- a/modules/payments-v2/receive/Receive.ts +++ b/modules/payments-v2/receive/Receive.ts @@ -57,6 +57,7 @@ export interface ReceiveDeps { readonly refreshView?: () => void; readonly attention: AttentionEmitter; readonly syncEpoch: () => string; + readonly track?: (op: Promise) => void; readonly now?: () => number; } @@ -116,13 +117,14 @@ export class Receive { } start(pollIntervalMs: number = POLL_INTERVAL_MS): void { - this.unsubscribeWake ??= - this.deps.delivery.onWake?.(() => { - void this.drainOnce(); - }) ?? null; - this.pollTimer ??= setInterval(() => { - void this.drainOnce(); - }, pollIntervalMs); + this.unsubscribeWake ??= this.deps.delivery.onWake?.(() => this.spawnDrain()) ?? null; + this.pollTimer ??= setInterval(() => this.spawnDrain(), pollIntervalMs); + } + + private spawnDrain(): void { + const op = this.drainOnce(); + if (this.deps.track !== undefined) this.deps.track(op); + else void op; } stop(): void { diff --git a/tests/unit/payments-v2/facade-harness.ts b/tests/unit/payments-v2/facade-harness.ts index 367d3d7e..82f0e584 100644 --- a/tests/unit/payments-v2/facade-harness.ts +++ b/tests/unit/payments-v2/facade-harness.ts @@ -69,6 +69,8 @@ export interface Hooks { complete?: (transferId: string) => Promise; applyDelta?: () => Promise; deliver?: () => Promise; + /** Runs before the mailbox listing yields — gates a drain mid-flight. */ + incoming?: () => Promise; } export interface Counters { @@ -131,7 +133,10 @@ function hookedDelivery(inner: DeliveryPort, hooks: Hooks): DeliveryPort { if (hooks.deliver) await hooks.deliver(); return inner.deliver(recipient, blob, options); }, - incoming: (since) => inner.incoming(since), + incoming: async function* incoming(since) { + if (hooks.incoming) await hooks.incoming(); + yield* inner.incoming(since); + }, incomingEpoch: () => inner.incomingEpoch(), ack: (id, disposition, reason) => inner.ack(id, disposition, reason), ...(inner.ackBatch !== undefined ? { ackBatch: (acks) => inner.ackBatch!(acks) } : {}), @@ -157,7 +162,7 @@ export interface World { peers: Map; seed(amount: bigint): Promise; peerDeliver(token: SphereToken, transferId: string): Promise; - gate(name: 'putIntent' | 'deliver' | 'listOpen' | 'applyDelta'): Gate; + gate(name: 'putIntent' | 'deliver' | 'listOpen' | 'applyDelta' | 'incoming'): Gate; } const worlds: World[] = []; @@ -202,6 +207,8 @@ export function makeWorld( restartOf?: World; /** The wallet's own Unicity ID, as Sphere supplies it (a live getter, never a snapshot). */ ownNametag?: () => string | undefined; + /** Receive's poll backstop; the default parks it far outside any test's clock. */ + receivePollMs?: number; } = {} ): World { const prior = options.restartOf; @@ -277,7 +284,7 @@ export function makeWorld( requestMemo: stubRequestMemoCodec, syncEpoch: () => session.currentEpoch(), newId: () => `tid-${idPrefix}${String(++ids)}`, - receivePollMs: 60 * 60 * 1000, + receivePollMs: options.receivePollMs ?? 60 * 60 * 1000, }); const world: World = { diff --git a/tests/unit/payments-v2/facade.test.ts b/tests/unit/payments-v2/facade.test.ts index 888388ed..8f5de55a 100644 --- a/tests/unit/payments-v2/facade.test.ts +++ b/tests/unit/payments-v2/facade.test.ts @@ -63,6 +63,11 @@ class DetMintEngine extends RealizationEngine implements DeterministicMintCapabl afterEach(cleanupWorlds); +/** Yield enough microtask generations for a promise-only pass to settle. */ +async function microtasks(turns = 256): Promise { + for (let i = 0; i < turns; i++) await Promise.resolve(); +} + describe('PaymentsFacade — send policy', () => { it('a multi-token send reports each leg as it certifies, so the UI can show real progress', async () => { const world = makeWorld(); @@ -572,6 +577,45 @@ describe('PaymentsFacade — lifecycle', () => { expect(ok.status).toBe('delivered'); }); + it('stop() awaits the POLL drain: a drain the 30s backstop spawned holds stop() until it settles (#770)', async () => { + vi.useFakeTimers(); + try { + const world = makeWorld({ receivePollMs: 30_000 }); + const gift = await world.engine.mint({ + recipientPubkey: hexToBytes(OWN_PUB), + value: { assets: [{ coinId: COIN, amount: 42n }] }, + }); + await world.peerDeliver(gift, 'gift-poll'); + await world.facade.start(); + const gate = world.gate('incoming'); + const timeline: string[] = []; + let incomingAtStop = -1; + + await vi.advanceTimersByTimeAsync(30_000); + await microtasks(); + expect(gate.entered).toBe(true); // the backstop fired and is inside the listing + + const stopping = world.facade.stop().then(() => { + timeline.push('stop-resolved'); + incomingAtStop = eventsOf(world, 'transfer:incoming').length; + }); + await vi.advanceTimersByTimeAsync(1_000); + await microtasks(); + // Nothing else observes this drain: unregistered, stop() would already be done + // here and the rest of destroy() would run against a live wallet-api drain. + expect(timeline).toEqual([]); + + gate.release(); + await stopping; + expect(timeline).toEqual(['stop-resolved']); + // Settled means SETTLED: verified, stored, acked and announced before stop returned. + expect(incomingAtStop).toBe(1); + expect(world.facade.tokens().map((t) => t.id)).toContain(gift.blob.tokenId); + } finally { + vi.useRealTimers(); + } + }); + it('setEngine mid-flight: the old op finishes on the OLD engine, the old engine is disposed, future ops use the new one', async () => { const world = makeWorld(); const engineB = new RealizationEngine({ chainPubkey: hexToBytes(OWN_PUB) }); From deb7492e720cff625aecf42371b44b254baa5537 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:00:51 +0200 Subject: [PATCH 16/43] fix(sphere): a switch racing destroy() must not re-arm the wallet (#770.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit There was no destroyed latch: _initialized is cleared LAST by destroy(), after every teardown step, so it cannot mark 'teardown has begun'. switchToAddress calls ensureReady() ONCE and then awaits ~8 times, and destroy() can land in any of those gaps. Two vectors, both reachable: - transport: initializeAddressModules -> ensureTransportMux BUILDS and connects a fresh mux whenever _transportMux is null, which is exactly what destroy() leaves behind. Fresh sockets after teardown returned. - payments: stopThenStartPaymentsV2 queues stop-then-start on the lifecycle mutex with no liveness gate, while destroy() queues only a stop. If the pair lands after destroy's stop, the start composes a whole new vertical — wallet-api session, wake socket, stream pulls, receive poll — that nothing will ever stop again. _destroyed is now destroy()'s FIRST statement, before any await and before it queues its own stop; that position is load-bearing. The mutex gate settles the facade vector on its own (a closure that BEGINS after destroy was called sees true; one already past the check has start() in flight, which destroy's queued stop is ordered after). The ensureAlive() calls cover the transport vector, which is not on that mutex. Also guards the current-index persist, which sat unguarded between two of them: a refused switch still wrote the address it never finished moving to, so the NEXT boot loaded a different address than the user was last on. No early return in destroy(): every step is null-guarded and idempotent, and an early return would change what a double call means. Two of the ensureAlive() calls are defence in depth and are not individually falsifiable — an earlier guard throws first on every path that reaches them. Their comments say so rather than implying a test covers them. --- core/Sphere.ts | 82 ++++++++++++- .../sphere-payments-v2-wiring.test.ts | 116 ++++++++++++++++++ .../unit/core/Sphere.destroy-secrets.test.ts | 22 +++- 3 files changed, 212 insertions(+), 8 deletions(-) diff --git a/core/Sphere.ts b/core/Sphere.ts index c0630ced..5dae3cf6 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -474,6 +474,13 @@ export class Sphere { // State private _initialized = false; + /** + * Destroyed latch (#770). Distinct from `_initialized`, which destroy() clears LAST — after + * every teardown step — so it cannot mark "teardown has begun". This is set as destroy()'s + * very FIRST statement, so the WHOLE teardown window is guarded, not just the instant after + * it. Read by ensureAlive() and by the §7 lifecycle mutex. + */ + private _destroyed = false; private _trackedAddressesLoaded = false; private _identity: MutableFullIdentity | null = null; private _masterKey: MasterKey | null = null; @@ -1513,9 +1520,11 @@ export class Sphere { // Keep the active address's OWN record in step — it is what a later switch // back reads, and a stale entry would hand that address a disposed engine. if (active) active.tokenEngine = this._tokenEngine; - // The facade snapshots its engine per operation — swap what FUTURE - // operations use; in-flight ones finish on the old engine (disposed below, - // which only tears down a verification worker pool). + // The facade snapshots its engine per operation — swap what FUTURE operations use. An + // in-flight op keeps the OLD handle, which is NOT the same as finishing on it: disposal + // cancels in-flight verification deterministically (the verify REJECTS), so an op that is + // mid-verify when the key changes fails instead of completing. `verify` is a receive-path + // read, never a spend — no money moves either way, but the caller sees an error. if (this._paymentsV2Active && this._tokenEngine) { this._paymentsV2Active.facade.setEngine(this._tokenEngine); } @@ -2320,6 +2329,12 @@ export class Sphere { // storage keys. Without this, modules would load the previous address's data. this._storage.setIdentity(newIdentity); + // #770: every await below re-checks liveness. switchToAddress calls ensureReady() ONCE + // at entry and then awaits ~8 times; destroy() can land in any of those gaps. This step + // is the transport vector: initializeAddressModules → ensureTransportMux BUILDS and + // connect()s a fresh mux whenever `_transportMux` is null — exactly what destroy() + // leaves behind — so an unguarded switch opens new sockets after teardown returned. + this.ensureAlive(); await this.initializeAddressModules({ index, identity: newIdentity }); } else if (nametag !== this._addressModules.get(index)!.identity.nametag) { // Modules already exist — only the nametag label changed. @@ -2346,8 +2361,14 @@ export class Sphere { // so two verticals never write one per-address KV; the lifecycle mutex // serializes overlapping switches. Re-visits compose a FRESH vertical // (durable state lives in the scoped KV; a stopped session can't restart). + this.ensureAlive(); await this.stopThenStartPaymentsV2(index, newIdentity); + // #770: a refused switch must not leave its index on disk. Without this guard a switch + // that destroy() overtook still persisted the address it never finished moving to, so the + // NEXT boot loaded a different wallet address than the one the user was last on — and the + // write can race destroy()'s provider disconnect besides. + this.ensureAlive(); // Persist current index await this._storage.set(STORAGE_KEYS_GLOBAL.CURRENT_ADDRESS_INDEX, index.toString()); @@ -2360,12 +2381,14 @@ export class Sphere { this._transport.setFallbackSince(fallbackTs); } + this.ensureAlive(); await this._transport.setIdentity(this._identity); // The transport recreates its NostrClient on identity change (the // SDK's client doesn't support runtime key swaps). When the Mux is // sharing that client (#123), it must rebind to the new instance // and re-establish its wallet/chat subscriptions on the new socket. + this.ensureAlive(); if (this._transportMux && typeof (this._transportMux as { rebindToSharedClient?: () => Promise }).rebindToSharedClient === 'function') { await (this._transportMux as { rebindToSharedClient: () => Promise }).rebindToSharedClient(); } @@ -2390,6 +2413,12 @@ export class Sphere { * Runs after switchToAddress returns so L3 queries can start immediately. */ private async postSwitchSync(index: number, newNametag?: string): Promise { + // Fire-and-forget from switchToAddress, so this is the one switch step that can outlive + // its caller (#770). Defensive only: it guards ENTRY, and the ensureAlive() before + // setIdentity already refuses earlier on this path — it stays so that a future reordering + // of switchToAddress cannot silently restart nametag registration / transport rebinding. + this.ensureAlive(); + // Sync identity with transport — recovers nametag from existing Nostr bindings if (!newNametag) { await this.syncIdentityWithTransport(); @@ -3530,7 +3559,24 @@ export class Sphere { } } + /** + * Tear this Sphere down: stop the vertical, destroy the modules, disconnect the providers + * and zero the key material. Idempotent by construction (every step is null-guarded) and + * deliberately WITHOUT an early return on re-entry, which would change what a double call + * means. + * + * #770: the FIRST statement flips `_destroyed` — before any await, and before this queues + * its own stop on the §7 lifecycle mutex. That position is load-bearing. Every guard reads + * the flag, so from that instant any switchToAddress step not yet begun refuses; and + * because it flips SYNCHRONOUSLY at entry, a stop/start pair the mutex runs after this call + * sees `true` and skips its start. Set it beside `_initialized` at the bottom instead and a + * concurrent switch re-arms a wallet whose owner already had destroy() return: fresh + * sockets, a fresh wallet-api session, a whole fresh vertical nothing will ever stop. + */ async destroy(): Promise { + // #770 — MUST stay the first statement; see the note above. + this._destroyed = true; + // FIRST, before anything that can throw. Module teardown and // MultiAddressTransportMux.disconnect() propagate, so any later placement would let a // single failure leave this Sphere's registry fetching forever. Nothing below needs it. @@ -4078,10 +4124,25 @@ export class Sphere { return run; } - /** Switch/boot: stop whatever runs, then start `index`'s vertical — atomically vs other lifecycle ops. */ + /** + * Switch/boot: stop whatever runs, then start `index`'s vertical — atomically vs other + * lifecycle ops. + * + * #770: the destroyed check between the two halves settles the FACADE vector on its own. + * `_destroyed` flips synchronously at destroy() entry, so any closure that BEGINS executing + * after destroy() was called sees `true`, and any closure already past the check has + * facade.start() in flight — which destroy()'s own queued stop is necessarily ordered + * after. The TRANSPORT vector is not on this mutex at all; the ensureAlive() calls in + * switchToAddress are what cover it. + */ private stopThenStartPaymentsV2(index: number, identity: FullIdentity): Promise { return this.queuePaymentsV2Op(async () => { await this.stopPaymentsV2Inner(); + // #770: destroy() may have run — or merely begun — while this pair waited its turn on + // the mutex. Starting now would attach a LIVE vertical (wallet-api session, wake + // socket, stream pulls, receive poll) to an owner whose destroy() already returned, + // and nothing would ever stop it again. + if (this._destroyed) return; await this.startPaymentsV2Inner(index, identity); }); } @@ -4140,7 +4201,20 @@ export class Sphere { await active.facade.stop(); } + /** + * Refuse once destroy() has STARTED (#770). `_initialized` cannot carry this: destroy() + * clears it last, so every teardown step is a window in which a concurrent call still reads + * a ready Sphere and re-arms it. + */ + private ensureAlive(): void { + if (this._destroyed) { + throw new SphereError('Sphere destroyed', 'NOT_INITIALIZED'); + } + } + private ensureReady(): void { + // Every existing ensureReady() caller inherits the destroyed check. + this.ensureAlive(); if (!this._initialized) { throw new SphereError('Sphere not initialized', 'NOT_INITIALIZED'); } diff --git a/tests/integration/sphere-payments-v2-wiring.test.ts b/tests/integration/sphere-payments-v2-wiring.test.ts index b37e07a1..a22c62b5 100644 --- a/tests/integration/sphere-payments-v2-wiring.test.ts +++ b/tests/integration/sphere-payments-v2-wiring.test.ts @@ -25,6 +25,7 @@ import { getPublicKey, hexToBytes } from '../../core/crypto'; import { decryptDeliveryBundle, deriveDeliveryEncryptionKey } from '../../core/delivery-envelope'; import { FileStorageProvider } from '../../impl/nodejs/storage/FileStorageProvider'; import { TRUSTBASE_TESTNET2 } from '../../assets/trustbase'; +import { STORAGE_KEYS_GLOBAL } from '../../constants'; import { TokenRegistry } from '../../registry'; import type { PeerInfo, TransportProvider } from '../../transport'; import type { OracleProvider } from '../../oracle'; @@ -809,6 +810,121 @@ describe('Sphere payments wiring — defaults (P11 flip: the vertical is default expect((caught as SphereError).code).toBe('NOT_INITIALIZED'); }, 20_000); + it('destroy() racing an unawaited switchToAddress leaves nothing running', async () => { + // #770: switchToAddress calls ensureReady() ONCE at entry and then awaits ~8 times, and + // `_initialized` is cleared LAST by destroy() — so every one of those hops is a window in + // which a concurrent teardown is invisible. The switch's stop/start pair is queued on the + // §7 mutex, so destroy()'s own stop is ordered AFTER it: without a destroyed latch the + // pair's start composes a whole new vertical (wallet-api session, wake socket, stream + // pulls, receive poll) for an owner whose destroy() has already RESOLVED, and nothing is + // left that could ever stop it. + const world = makeWorld(); + const sphere = await buildSphere({ walletApi: world.walletApi }); + const transport = (sphere as unknown as { _transport: TransportProvider })._transport; + const setIdentity = transport.setIdentity as unknown as ReturnType; + + // Hold the boot vertical's stop at quiescence so the switch parks INSIDE its stop/start + // pair — the exact window destroy() has to land in. + let release!: () => void; + const opened = new Promise((resolve) => (release = resolve)); + world.gates.listMailbox = async () => opened; + const receiving = sphere.payments.receive(); + + // UNAWAITED: the caller's switch is still in flight when the owner tears the wallet down. + const switching = sphere.switchToAddress(1).then( + () => 'resolved' as const, + (err: unknown) => err + ); + await sleep(200); + expect(world.transports).toHaveLength(1); + + const identityCallsBeforeDestroy = setIdentity.mock.calls.length; + const storage = (sphere as unknown as { _storage: FileStorageProvider })._storage; + const indexBeforeDestroy = await storage.get(STORAGE_KEYS_GLOBAL.CURRENT_ADDRESS_INDEX); + const destroying = sphere.destroy(); + await sleep(50); + delete world.gates.listMailbox; + release(); + await receiving.catch(() => undefined); + await destroying; + const outcome = await switching; + // Give anything the switch might still have queued a chance to actually run. + await sleep(200); + + // The invariant: after destroy() resolves, NO vertical is left started-but-not-stopped. + expect( + world.transports.filter((t) => t.session.startCalls === 1 && t.session.stopCalls === 0) + ).toHaveLength(0); + // Stronger: the switch never composed a second vertical at all. + expect(world.transports).toHaveLength(1); + expect(world.transports[0]!.session.stopCalls).toBe(1); + + // The switch refused instead of re-arming the transport (the vector the §7 mutex cannot + // cover — it is not a lifecycle op). + expect(outcome).toBeInstanceOf(SphereError); + expect((outcome as SphereError).code).toBe('NOT_INITIALIZED'); + expect(setIdentity.mock.calls.length).toBe(identityCallsBeforeDestroy); + expect((sphere as unknown as { _transportMux: unknown })._transportMux).toBeNull(); + // A refused switch must not leave its index on disk: persisting it would send the NEXT + // boot to an address the user never finished moving to. + expect(await storage.get(STORAGE_KEYS_GLOBAL.CURRENT_ADDRESS_INDEX)).toBe(indexBeforeDestroy); + + let caught: unknown; + try { + void sphere.payments; + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(SphereError); + expect((caught as SphereError).code).toBe('NOT_INITIALIZED'); + }, 30_000); + + it('destroy() racing a switch parked BEFORE module bring-up never rebuilds the module set', async () => { + // The other half of #770, and the one the §7 mutex provably cannot reach: a switch parked + // on an await that is NOT a lifecycle op. initializeAddressModules → ensureTransportMux + // BUILDS and connect()s a fresh MultiAddressTransportMux whenever `_transportMux` is null + // — which is exactly the state destroy() leaves behind — so an unguarded resume opens new + // sockets and refills the per-address module map that destroy() just cleared. + const world = makeWorld(); + const sphere = await buildSphere({ walletApi: world.walletApi }); + const transport = (sphere as unknown as { _transport: TransportProvider })._transport; + const modules = (sphere as unknown as { _addressModules: Map })._addressModules; + + // Park at the nametag availability probe — the hop immediately BEFORE module bring-up. + let release!: () => void; + const opened = new Promise((resolve) => (release = resolve)); + (transport as unknown as { resolveNametag: () => Promise }).resolveNametag = async () => { + await opened; + return null; + }; + + const switching = sphere.switchToAddress(1, { nametag: 'zed' }).then( + () => 'resolved' as const, + (err: unknown) => err + ); + await sleep(200); + + await sphere.destroy(); + expect(modules.size).toBe(0); + + release(); + const outcome = await switching; + await sleep(200); + + // THE invariant: after destroy() resolved, nothing is left started-but-not-stopped. This + // ordering is the one that leaves a PERMANENT orphan when unguarded — destroy()'s stop + // has already run, so the switch's start has no stop behind it, ever. + expect( + world.transports.filter((t) => t.session.startCalls === 1 && t.session.stopCalls === 0) + ).toHaveLength(0); + expect(world.transports).toHaveLength(1); + // Nothing rebuilt: no module set, no mux. + expect(modules.size).toBe(0); + expect((sphere as unknown as { _transportMux: unknown })._transportMux).toBeNull(); + expect(outcome).toBeInstanceOf(SphereError); + expect((outcome as SphereError).code).toBe('NOT_INITIALIZED'); + }, 30_000); + it('setOracleApiKey rebuilds the engine and swaps it via facade.setEngine; the replaced engine is disposed', async () => { const world = makeWorld(); const sphere = await buildSphere({ walletApi: world.walletApi }); diff --git a/tests/unit/core/Sphere.destroy-secrets.test.ts b/tests/unit/core/Sphere.destroy-secrets.test.ts index 5b40dc53..ddec1a24 100644 --- a/tests/unit/core/Sphere.destroy-secrets.test.ts +++ b/tests/unit/core/Sphere.destroy-secrets.test.ts @@ -96,10 +96,24 @@ describe('Sphere.destroy() secret hygiene', () => { await sphere.destroy(); // Silently answering false/null/a half-empty WalletInfo is worse than throwing: a - // caller cannot tell "no master key" from "the wallet is gone". - expect(() => sphere.getMnemonic()).toThrow('Sphere not initialized'); - expect(() => sphere.hasMasterKey()).toThrow('Sphere not initialized'); - expect(() => sphere.getWalletInfo()).toThrow('Sphere not initialized'); + // caller cannot tell "no master key" from "the wallet is gone". The CODE is the + // contract. Since #770 the refusal comes from the destroyed latch — set at destroy() + // ENTRY, so it covers the WHOLE teardown window and not merely the instant after it — + // and the message says so instead of the vaguer "not initialized". + for (const call of [ + () => sphere.getMnemonic(), + () => sphere.hasMasterKey(), + () => sphere.getWalletInfo(), + ]) { + expect(call).toThrow('Sphere destroyed'); + let code: unknown; + try { + call(); + } catch (err) { + code = (err as { code?: string }).code; + } + expect(code).toBe('NOT_INITIALIZED'); + } }); }); From cdd5ee86a9a247b178a6e0fcde1c5afb2c374dee Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:01:12 +0200 Subject: [PATCH 17/43] test(aggregator): a WRONG root key must refuse a token the service really certified MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLAUDE.md stated this guard as fact; grep -rn INVALID_TRUSTBASE tests/ returned zero. Without it, 'verify() passes against a real aggregator' could be true because verification never consults the trust base at all — the one assertion in the repo that no fake can make was unprotected against going vacuous. The tamper replaces every rootNodes[].sigKey with a fresh valid compressed pubkey and changes NOTHING else — node ids stay, so the quorum rule FINDS each node and rejects it on its key rather than through 'No root node defined' (asserted absent); networkId stays, since it is checked before any signature. Two layers, because either alone is dishonest in a different direction: the engine-level assertion is literally the call Receive.screen() makes, but can only say 'refused'; the trace-level assertion names INVALID_TRUSTBASE at SignatureVerificationRule. The reconstructed verification context is self-checking — the CORRECT trust base is run through the same context first and must come back OK, so a miswired reconstruction cannot fabricate the failure. CLAUDE.md's sentence corrected with it: engine.verify returns the aggregated FAIL, not INVALID_TRUSTBASE, and the compose stack is single-root-node, so mainnet's 4-node/quorum-3 shape remains unexercised. --- CLAUDE.md | 8 +- tests/aggregator/aggregator-v3.test.ts | 72 ++++++++++++- tests/aggregator/support/trustBase.ts | 134 +++++++++++++++++++++++++ 3 files changed, 209 insertions(+), 5 deletions(-) create mode 100644 tests/aggregator/support/trustBase.ts diff --git a/CLAUDE.md b/CLAUDE.md index 2d7303ae..8a5a772d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -808,8 +808,12 @@ Key test areas: Mint / transfer / split / same-transferId resume, each verified against the service's own generated trust base. `verify()` passing is the assertion no fake can make: the leaf value this client computes — `H(transactionHash, referenceTime)` since 3.x — reproduces the leaf the Go - service inserted. Guard against it going vacuous: with a well-formed but WRONG root key it must - fail `INVALID_TRUSTBASE`. + service inserted. Guard against it going vacuous: with a well-formed but WRONG root key the SAME + certified token must be refused — `engine.verify` reports the aggregated `FAIL` (the + granular status lives in the SDK's nested trace, which the test walks to name + `INVALID_TRUSTBASE` at the quorum-signature rule). The compose stack runs a SINGLE + bft-root node, so this exercises "wrong key", not a real quorum — mainnet's + 4-node/threshold-3 shape is still unexercised anywhere in the repo. - `tests/mutation/probes.json` — mutation probes over `modules/payments-v2/*`, `token-engine/{proof-wait,SphereTokenEngine}.ts`, `impl/wallet-api-v2/*`, the `core/` wiring and `transport/NostrTransportProvider.ts`; `npm run test:mutation` must report every one KILLED diff --git a/tests/aggregator/aggregator-v3.test.ts b/tests/aggregator/aggregator-v3.test.ts index 8cba733a..4eb42bb7 100644 --- a/tests/aggregator/aggregator-v3.test.ts +++ b/tests/aggregator/aggregator-v3.test.ts @@ -24,8 +24,15 @@ import { readFileSync } from 'node:fs'; import { afterAll, beforeAll, describe, expect, it } from 'vitest'; import { createSphereTokenEngine, type ITokenEngine } from '../../token-engine'; -import { SigningService } from '../../token-engine/sdk'; +import { + HexConverter, + InclusionProofVerificationStatus, + RootTrustBase, + SigningService, + VerificationStatus, +} from '../../token-engine/sdk'; import { startAggregatorStack, type AggregatorStack } from './support/aggregatorStack'; +import { flattenTrace, verificationContextFor, withWrongRootKeys } from './support/trustBase'; const COIN = 'aa'.repeat(32); @@ -41,10 +48,11 @@ afterAll(async () => { await stack?.stop(); }, 120_000); -function newEngine(): Promise { +/** @param trustBase Defaults to the stack's real trust base; the vacuity guard passes a doctored one. */ +function newEngine(trustBase: unknown = trustBaseJson): Promise { return createSphereTokenEngine({ aggregatorUrl: stack.url, - trustBaseJson, + trustBaseJson: trustBase, privateKey: SigningService.generatePrivateKey(), proofTimeoutMs: 30_000, proofPollIntervalMs: 500, @@ -132,4 +140,62 @@ describe('engine ↔ real aggregator-go v3', () => { expect(Buffer.from(resumed.blob.token).equals(Buffer.from(first.blob.token))).toBe(true); expect(await recipient.verify(resumed)).toEqual({ ok: true }); }, 180_000); + + // The guard against every `{ ok: true }` above being vacuous. A verify() that + // never consulted the trust base would pass all four; only a NO it can be made + // to say proves it looked. The token is genuinely certified by the running + // service — the sole defect is the key the seal's signatures are checked against. + it('refuses that same certified token when the trust base carries a WRONG root key', async () => { + const engine = await newEngine(); + const token = await engine.mint({ + recipientPubkey: engine.getIdentity().chainPubkey, + value: { assets: [{ coinId: COIN, amount: 5n }] }, + }); + // Both directions in one run: a suite that only ever showed a failure would + // stay green if verification rejected everything. + expect(await engine.verify(token)).toEqual({ ok: true }); + + const wrongJson = withWrongRootKeys(trustBaseJson); + const real = RootTrustBase.fromJSON(trustBaseJson); + const wrong = RootTrustBase.fromJSON(wrongJson); + // Well-formed, and identical everywhere the certificate verifier looks before + // it reaches a signature — so nothing but the key can explain the refusal. + expect(wrong.networkId.id).toBe(real.networkId.id); + expect(wrong.quorumThreshold).toBe(real.quorumThreshold); + expect([...wrong.rootNodes.keys()]).toEqual([...real.rootNodes.keys()]); + for (const [nodeId, node] of wrong.rootNodes) { + expect(SigningService.isPublicKeyValid(node.signingKey)).toBe(true); + expect(HexConverter.encode(node.signingKey)).not.toBe( + HexConverter.encode(real.rootNodes.get(nodeId)!.signingKey), + ); + } + + // 1. The gate the money path actually calls (receive/Receive.ts screens on it). + // It reports only the aggregated status, so this proves refusal, not cause. + const wrongEngine = await newEngine(wrongJson); + expect(await wrongEngine.verify(token)).toEqual({ ok: false, reason: VerificationStatus.FAIL }); + + // 2. The cause, from the trace the port collapses. Running the CORRECT trust + // base through the same reconstructed context first is what makes the + // reconstruction trustworthy: a miswired context would fail here too. + const okTrace = flattenTrace(await token.sdkToken.verify(verificationContextFor(trustBaseJson))); + expect(okTrace[0].status).toBe(VerificationStatus.OK); + expect(okTrace.map((entry) => entry.status)).not.toContain(InclusionProofVerificationStatus.INVALID_TRUSTBASE); + + const failTrace = flattenTrace(await token.sdkToken.verify(verificationContextFor(wrongJson))); + expect(failTrace[0].status).toBe(VerificationStatus.FAIL); + expect(failTrace.map((entry) => entry.status)).toContain(InclusionProofVerificationStatus.INVALID_TRUSTBASE); + // Failed ON THE KEY: the node was found and its signature rejected. A lookup + // miss ('No root node defined') would reach INVALID_TRUSTBASE too, while + // proving only that an unknown node id is unknown. + expect( + failTrace.some( + (entry) => + entry.rule.startsWith('SignatureVerificationRule[') && + entry.status === VerificationStatus.FAIL && + entry.message === 'Signature verification failed', + ), + ).toBe(true); + expect(failTrace.map((entry) => entry.message)).not.toContain('No root node defined'); + }, 120_000); }); diff --git a/tests/aggregator/support/trustBase.ts b/tests/aggregator/support/trustBase.ts new file mode 100644 index 00000000..2c0f6325 --- /dev/null +++ b/tests/aggregator/support/trustBase.ts @@ -0,0 +1,134 @@ +/** + * Instruments for the vacuity guard in `aggregator-v3.test.ts`. + * + * The four positive cases in that suite all assert `verify() === { ok: true }` + * against a real aggregator-go. That is only evidence if verification can also + * say NO — a `verify()` that returned OK unconditionally, or one that never + * consulted the trust base at all, would make every one of them green. So the + * suite needs one case where the ONLY thing wrong is the trust base, and the + * trust base is wrong in the one way that is hard to fake: a root key that is a + * perfectly good secp256k1 public key but not the one that signed the seal. + * + * Everything here is deliberately narrow: substitute keys, rebuild the SDK's own + * verification pipeline exactly as `token-engine/factory.ts` does, and flatten a + * verification trace. No assertions live in this file. + */ + +import { decodeSpherePaymentData } from '../../../token-engine/SpherePaymentData'; +import { + HexConverter, + MintJustificationVerifierService, + PredicateVerifierService, + RootTrustBase, + Secp256k1SignatureVerifier, + SigningService, + SplitMintJustificationVerifier, + TokenIssuanceVerifierService, + UnicityCertificateVerifier, + UnicitySealQuorumSignaturesVerificationRule, + VerificationContext, + VerificationResult, + VerifiedSealCache, +} from '../../../token-engine/sdk'; + +/** The two fields of the trust base JSON this module touches; the rest rides along untyped. */ +interface RootNodeJson { + readonly nodeId: string; + readonly sigKey: string; + readonly stake: string; +} + +interface TrustBaseJson { + rootNodes: RootNodeJson[]; + readonly [field: string]: unknown; +} + +/** A fresh, valid, compressed secp256k1 public key that no node already claims. */ +function unusedSigningKey(taken: Set): string { + // `generatePrivateKey` is rejection-sampled, so a duplicate is not reachable in + // practice; the bound exists so a broken generator fails loudly instead of hanging. + for (let attempt = 0; attempt < 32; attempt++) { + const publicKey = new SigningService(SigningService.generatePrivateKey()).publicKey; + const hex = HexConverter.encode(publicKey); + if (!SigningService.isPublicKeyValid(publicKey) || taken.has(hex)) continue; + taken.add(hex); + return hex; + } + throw new Error('Could not generate a distinct root signing key.'); +} + +/** + * The same trust base with every root node's `sigKey` replaced by a different, + * valid public key. + * + * What is deliberately NOT changed, because each would make the guard prove + * something weaker than "the root key is checked": + * + * - the **node ids**, so the quorum rule still FINDS each node and rejects it on + * its key. Renaming a node makes the seal's signer unknown to the trust base, + * which fails through `'No root node defined'` — a lookup miss, not a key check. + * - the **networkId**, which `UnicityCertificateVerifier` compares against the + * seal before it ever reaches a signature. + * - the **quorumThreshold**, stakes, epoch, hashes and the trust base's own + * `signatures` map, so the result still parses as a valid `RootTrustBase` and + * still demands the same number of good signatures as the real one. + * + * @param trustBaseJson The trust base the aggregator stack generated. + * @returns A structurally identical trust base whose root keys are all wrong. + */ +export function withWrongRootKeys(trustBaseJson: unknown): unknown { + const tampered = structuredClone(trustBaseJson) as TrustBaseJson; + if (!Array.isArray(tampered.rootNodes) || tampered.rootNodes.length === 0) { + throw new Error('Trust base declares no root nodes — there is no key to get wrong.'); + } + const taken = new Set(tampered.rootNodes.map((node) => node.sigKey.toLowerCase())); + tampered.rootNodes = tampered.rootNodes.map((node) => ({ ...node, sigKey: unusedSigningKey(taken) })); + return tampered; +} + +/** + * The verification pipeline `createSphereTokenEngine` builds, over a given trust + * base. + * + * `ITokenEngine.verify` answers `{ ok, reason }` where `reason` is the AGGREGATED + * `VerificationStatus` — `'FAIL'`. The granular status the rules produce, including + * `INVALID_TRUSTBASE`, survives only in the nested trace, which the port does not + * expose. Naming the reason therefore means driving `Token.verify` directly. + * + * This is a reconstruction of the engine's context, not the engine's own, so the + * caller must keep it honest by also running the CORRECT trust base through it: + * a context assembled wrongly here would fail that case too. + */ +export function verificationContextFor(trustBaseJson: unknown): VerificationContext { + const mintJustificationVerifier = new MintJustificationVerifierService(); + mintJustificationVerifier.register(new SplitMintJustificationVerifier(decodeSpherePaymentData)); + return new VerificationContext( + RootTrustBase.fromJSON(trustBaseJson), + PredicateVerifierService.create(), + new UnicityCertificateVerifier( + new UnicitySealQuorumSignaturesVerificationRule(new Secp256k1SignatureVerifier(), new VerifiedSealCache(256)), + ), + mintJustificationVerifier, + new TokenIssuanceVerifierService(false), + ); +} + +/** One node of a flattened verification trace. */ +export interface TraceEntry { + readonly rule: string; + readonly status: string; + readonly message: string; +} + +/** + * Depth-first flattening of a verification trace, root first. + * + * Rules nest their children in `results`, and statuses are of mixed enum types + * down the tree, so they are compared as strings. + */ +export function flattenTrace(result: VerificationResult): TraceEntry[] { + return [ + { message: result.message, rule: result.rule, status: String(result.status) }, + ...result.results.flatMap((child) => flattenTrace(child)), + ]; +} From 0a0e44790e94c498708c844e464fa06465c0b5e8 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:01:24 +0200 Subject: [PATCH 18/43] docs: the #766/#767/#772 blast radius of #773 (7 items) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Only the items this PR created or exposed; the other ~25 in #773 are pre-existing rot for a separate docs PR. The one that matters: docs/INTEGRATION.md published the StorageProvider interface with saveTrackedAddresses and NO contract, so a custom provider written from that doc clobbers addresses — the exact #766 data-loss bug. It now carries the write contract (union by index, greater updatedAt supplies hidden, earlier createdAt survives, serialize per backing store, a failed write must reject without bricking later ones), the uint32 rule, why a union is safe, and a worked implementation. That sample was extracted, type-checked under strict and EXECUTED: it reproduces the lost-update fix, the conflict merge, the uint32 drops and the tolerant read. Also: backingStoreId documented on the published interface; Sphere.clear() and import() documented as destroying live Spheres scoped by BACKING STORE; Sphere.init's option block filled in (network is required at runtime though optional in the type, plus verification/debug/onProgress and the module flags); TrackedAddressEntry.index uint32 rule; isReady and networkId added to the properties table (the CHANGELOG told consumers to move to isReady and no reference doc listed it); a TokenRegistry section for create/dispose/isDisposed and the instance waitForReady; and MIGRATION-TOKEN-REGISTRY.md linked from the README, having been reachable only from the CHANGELOG. --- README.md | 11 ++++ docs/API.md | 147 ++++++++++++++++++++++++++++++++++++++++++- docs/INTEGRATION.md | 148 ++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 300 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 84497c4f..c680ea95 100644 --- a/README.md +++ b/README.md @@ -667,6 +667,14 @@ const sphere = await Sphere.import({ }); ``` +> **`Sphere.import()` wipes first, and that wipe destroys live Spheres.** When a wallet already +> exists on the given storage — or a Sphere is live on it — import calls `Sphere.clear()` before +> writing, which calls `destroy()` on every live `Sphere` built on that **backing store**: their +> payments verticals stop, their providers disconnect, and every `sphere.on()` handler goes with +> them. The scope is the store, not the provider object: two provider objects reporting the same +> `backingStoreId` share the teardown, while a Sphere on unrelated storage is left alone. Drop +> your references to the old instance rather than reusing it. + ## Wallet Export/Import (JSON) ```typescript @@ -880,6 +888,9 @@ Design and migration references: - [Payments vertical design](./docs/PAYMENTS-V2-DESIGN.md) — the authoritative money design - [Payments migration guide](./docs/MIGRATION-PAYMENTS-V2.md) — what the P11 flip moved +- [Token registry migration guide](./docs/MIGRATION-TOKEN-REGISTRY.md) — the per-Sphere token + registry, the removed `Sphere.getInstance()` / `isInitialized()` lifecycle globals, and + `Sphere.clear()` / `import()` becoming backing-store-scoped ## Browser Providers diff --git a/docs/API.md b/docs/API.md index fac36d8e..7f2102bd 100644 --- a/docs/API.md +++ b/docs/API.md @@ -21,6 +21,10 @@ Primary entry point. Creates a new wallet or loads an existing one automatically ```typescript const { sphere, created, generatedMnemonic } = await Sphere.init({ storage, transport, oracle, + network: 'testnet2', // REQUIRED in practice: 'testnet' | 'testnet2' | 'mainnet'. + // Compared to walletApi.network as an exact STRING — 'testnet' + // vs 'testnet2' is a mismatch (INVALID_CONFIG), alias or not. + // Also selects the token registry for this Sphere. walletApi, // REQUIRED: wallet-api transport config // { network, baseUrl, deviceId?, fetchFn?, webSocketFactory?, // paymentsV2Transport? } — createWalletApiProviders builds it; @@ -33,9 +37,33 @@ const { sphere, created, generatedMnemonic } = await Sphere.init({ price: priceProvider, // Optional PriceProvider derivationPath: "m/44'/0'/0'", // Optional custom path dmSince: Math.floor(Date.now() / 1000) - 86400, // Optional: DM history fallback (unix seconds) + + groupChat: true, // Optional: NIP-29 group chat — true = network-default relays, + // or a GroupChatModuleConfig. Omit and sphere.groupChat is null + market: true, // Optional: market intents — true | MarketModuleConfig. + // Omit and sphere.market is null + communications: { maxPerConversation: 200 }, // Optional CommunicationsModuleConfig + discoverAddresses: true, // Optional: scan Nostr bindings for previously used HD addresses. + // Only applies when the wallet is newly created. + // true | DiscoverAddressesOptions + verification: { createWorker }, // Optional: opt in to PARALLEL token verification — + // { createWorker, poolSize? }, where createWorker spawns YOUR + // bundled worker entry script. Omit for the sequential + // verifier. See docs/VERIFICATION-WORKERS.md + debug: true, // Optional: debug logging. The flag is process-global — an + // explicit false turns it off, omitting it leaves it as-is + onProgress: (p) => console.log(p.step, p.message), // Optional: init progress callback }); ``` +**`network` is required even though the type marks it optional.** `Sphere.init` resolves the +payments composition first, and that resolution throws `INVALID_CONFIG` unless `network` is a +known network AND string-equal to `walletApi.network` — so all three of +`createBrowserProviders`/`createNodeProviders`, the `walletApi` config, and `Sphere.init` must +carry the *same literal*. The provider factories do not return `network` in their bundle, so +spreading `{ ...providers }` does not supply it. `Sphere.create`, `Sphere.load` and +`Sphere.import` take the same option and enforce the same rule. + **Removed options:** `accounting: true` / `swap: true` **throw** a typed `INVALID_CONFIG` — invoicing and swaps no longer exist in the SDK, and that refusal is kept deliberately through 0.15.0 (a silent no-op would hide the removal in exactly the release where consumers @@ -75,16 +103,42 @@ mnemonic with it. The 0.15.0 scoped-KV generation bump needs nothing from you await Sphere.clear({ storage: providers.storage }); ``` +**It destroys live Spheres — scoped by BACKING STORE, not by provider object.** Before wiping, +`clear()` calls `destroy()` on every live `Sphere` built on the store `storage` addresses: the +payments vertical stops, providers disconnect, and every `sphere.on()` handler goes with them. +Those instances are dead afterwards; hold no references across a `clear()`. + +Which instances that covers is decided by +[`StorageProvider.backingStoreId`](./INTEGRATION.md#storage-provider-interface). Two provider +objects that report the same value address the same data, so clearing through either destroys +the Spheres of both — the bundled providers report one (resolved wallet path for +`FileStorageProvider`; `dbName` + key prefix for `IndexedDBStorageProvider`; the `Storage` +object + prefix for `LocalStorageProvider`). A provider that declares none is scoped to itself, +so a second object over the same data is treated as unrelated. A Sphere on *other* storage is +never touched. + +`Sphere.import(options)` inherits all of this: it calls `Sphere.clear({ storage })` first +whenever a wallet exists on that storage or a live Sphere is registered on it, so importing over +storage B tears down the Spheres on B — and only those. + ### Properties | Property | Type | Description | |----------|------|-------------| | `identity` | `FullIdentity \| null` | Current wallet identity (after init/load) | +| `isReady` | `boolean` | Whether this Sphere is initialized. `true` once `init`/`create`/`load`/`import` has finished building it; back to `false` after `destroy()` | +| `networkId` | `number \| undefined` | The active network id, read from the oracle's root trust base (`RootTrustBase.networkId` — testnet2 = `4`, mainnet = `1`). `undefined` when the oracle has no trust base | | `payments` | `PaymentsV2` | The payments facade (assets/tokens/history/send/mint/receive/requests). **Throws** `NOT_INITIALIZED` while no vertical runs — init in flight, mid address-switch, or destroyed | | `communications` | `CommunicationsModule` | Messaging operations | | `groupChat` | `GroupChatModule \| null` | NIP-29 group chat (null unless enabled) | | `market` | `MarketModule \| null` | Market intents (null unless enabled) | +`isReady` is the replacement for the removed `Sphere.isInitialized()` static. `Sphere.getInstance()`, +`Sphere.isInitialized()` and the `getSphere` export are **gone** — hold the instance the entry point +returned and read `sphere.isReady` on it. There was never a safe deprecation: once a second Sphere +had been created *and* destroyed, `getInstance()` answered `null` while the first was alive and +serving money. See [MIGRATION-TOKEN-REGISTRY.md](./MIGRATION-TOKEN-REGISTRY.md#also-removed-the-sphere-lifecycle-globals). + ### Instance Methods #### `signMessage(message: string): string` @@ -637,13 +691,26 @@ Minimal data stored in persistent storage for a tracked address. ```typescript interface TrackedAddressEntry { - readonly index: number; // HD derivation index + readonly index: number; // HD derivation index — must be a uint32 (see below) hidden: boolean; // Whether hidden from UI readonly createdAt: number; // Timestamp (ms) when first activated updatedAt: number; // Timestamp (ms) of last modification } ``` +`index` is a **BIP32 child number, so it must be a uint32**: an integer in `0` … `0xffffffff`. +A row whose `index` is not — fractional, negative, out of range, or not a number — is **dropped +when the registry is read**, not repaired. It is a drop rather than a repair because `1.5` would +`parseInt()` down to index 1's derivation path and alias a real address, and anything above +`0xffffffff` pads to more than 8 hex digits and derives off-standard. + +`createdAt` / `updatedAt` are repaired instead: a missing or non-finite value reads as `0`, and +`hidden` reads as `true` only for an exact `true`. + +Custom `StorageProvider` implementations own this: see +[the tracked-address write contract](./INTEGRATION.md#the-tracked-address-write-contract) for +the merge rules `saveTrackedAddresses` must obey. + ### TrackedAddress Full tracked address with derived fields (available in memory via `getActiveAddresses()`, etc.). @@ -891,3 +958,81 @@ Network configuration: - **testnet2:** `https://gateway.testnet2.unicity.network` (networkId 4) - **mainnet:** live v3 gateway (`gateway.mainnet.unicity.network`, network id 1). The chain is live; there is no mainnet wallet-api deployment yet, so the money path is not reachable. The `dev` preset was removed with the v1 network. +--- + +## TokenRegistry + +Token metadata (symbol, name, decimals, icons) by coin ID — fetched from the network's registry +URL, cached in the `StorageProvider`, refreshed hourly. The lookup methods +(`getDefinition`, `getSymbol`, `getDecimals`, `getCoinIdBySymbol`, `getAllDefinitions`, …) are +covered in the [Browser](./QUICKSTART-BROWSER.md#look-up-asset-metadata) and +[Node.js](./QUICKSTART-NODEJS.md#look-up-asset-metadata) quick starts. This section is the +**lifecycle** surface. + +### Two kinds of registry + +| | Process-global singleton | Owned instance | +|---|---|---| +| Obtain | `TokenRegistry.getInstance()`, configured by `TokenRegistry.configure(options)` | `TokenRegistry.create(options)` | +| Who else can repoint it | **anyone** — `configure()` reaches into whatever instance exists, and every `Sphere.init()` calls it | nobody | +| Stopping it | `TokenRegistry.resetInstance()` / `TokenRegistry.destroy()` | `registry.dispose()` | + +A `Sphere` **builds and owns its own registry** (`TokenRegistry.create`) and the payments facade +presents from that one, so two Spheres on different networks can no longer overwrite each +other's metadata. `sphere.destroy()` disposes it. The global is still configured by +`Sphere.init()` for code that reads it directly, and it is deliberately left running. + +`TokenRegistry.configure()` and `TokenRegistry.create()` take the same options: + +```typescript +interface TokenRegistryConfig { + remoteUrl?: string; // registry JSON URL — NETWORKS[network].tokenRegistryUrl + storage?: StorageProvider; // persistent cache + refreshIntervalMs?: number; // default 1 hour + autoRefresh?: boolean; // default true +} +``` + +### `TokenRegistry.create(options: TokenRegistryConfig): TokenRegistry` + +Build an **independent** registry rather than touching the singleton. The options are applied +immediately — a cache read first, then the remote fetch, which is awaited only when the cache +misses — exactly as `configure()` does on the global. Dispose it when its owner goes away. + +```typescript +import { TokenRegistry, NETWORKS } from '@unicitylabs/sphere-sdk'; + +const registry = TokenRegistry.create({ + remoteUrl: NETWORKS.testnet2.tokenRegistryUrl, + storage: providers.storage, +}); + +await registry.waitForReady(); +const uct = registry.getDefinitionBySymbol('UCT'); + +registry.dispose(); +``` + +### `registry.dispose(): void` + +Stop this registry for good: no refresh timer, no late apply of an in-flight fetch, no late +cache write — the request already in the air is aborted, not merely ignored. Idempotent. + +Required for any registry you `create()`: nothing in `registry/` calls `unref()`, so an +undisposed registry keeps an hourly fetch running and, under Node, keeps the event loop alive. + +Reads still work after disposal; they are simply **frozen** at the last-applied definitions. +Disposal is permanent — a disposed registry cannot be revived, so build a new one with +`create()`. + +### `registry.isDisposed: boolean` + +Whether `dispose()` has been called. + +### `registry.waitForReady(timeoutMs?: number): Promise` + +Wait for the initial load (cache, else remote) to settle. Resolves `true` when definitions were +loaded, `false` on timeout or when there was no data source. `timeoutMs` defaults to `10_000`; +pass `0` to wait without a timeout. The static `TokenRegistry.waitForReady()` is the same +contract against the singleton. + diff --git a/docs/INTEGRATION.md b/docs/INTEGRATION.md index 19cc74d6..dcc0bbee 100644 --- a/docs/INTEGRATION.md +++ b/docs/INTEGRATION.md @@ -347,6 +347,13 @@ This is a wipe, not a maintenance step: it clears the store with no prefix, so t with it. It is not the way to migrate the 0.15.0 scoped-KV generation — that needs nothing from you. +It also **destroys the live Spheres on that backing store** before wiping: each one's payments +vertical stops, its providers disconnect, and every `sphere.on()` handler goes with it. Drop +your references afterwards. The scope is the store the provider addresses, not the provider +object — two provider objects reporting the same +[`backingStoreId`](#storage-provider-interface) share the teardown, and a Sphere on other +storage is never touched. `Sphere.import()` clears first, so it carries the same consequence. + ### Multi-Address Derivation The SDK supports HD (Hierarchical Deterministic) address derivation following BIP32/BIP44 standards. @@ -799,10 +806,16 @@ interface StorageProvider { getStatus(): ProviderStatus; /** - * Optional, but supply it if two provider objects can address one store: two - * instances returning the same value share erasure, so `Sphere.clear()` tears - * down the live Spheres of both. Identify the STORE (path / database + prefix), - * never the class. Omitted, liveness falls back to per-object identity. + * Stable identity of the BACKING STORE this provider addresses — not of this + * object, and not of the class. Optional, but supply it if two provider objects + * can address one store: two instances returning the same value share erasure, + * so `Sphere.clear()` (and `Sphere.import()`, which clears first) tears down the + * live Spheres of both. Compose it from everything that selects the store (file + * path, database name, key prefix) behind a scheme prefix, so two kinds of store + * can never collide on one string. It must not change over the provider's + * lifetime — it is read again on teardown. Omitted, liveness falls back to + * per-object identity, i.e. a second provider over the same data is treated as + * unrelated. */ readonly backingStoreId?: string; @@ -814,12 +827,137 @@ interface StorageProvider { keys(prefix?: string): Promise; clear(prefix?: string): Promise; - // Tracked addresses registry + // Tracked addresses registry. + // saveTrackedAddresses MUST MERGE, NEVER REPLACE — see the write contract below. + // A replacing implementation silently loses addresses. saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise; loadTrackedAddresses(): Promise; } ``` +#### The tracked-address write contract + +`saveTrackedAddresses` **must merge, never replace.** `entries` is one writer's snapshot, not +the whole truth: every Sphere sharing this storage keeps its own copy of the registry and +persists all of it. Writing the argument verbatim is a lost update — A activates index 1, B +(whose snapshot predates that) activates index 2, and B's write erases index 1 while A still +reports it. This happens on a single network with a single provider, and it is the #766 +data-loss bug; do **not** try to fix it by renaming or network-scoping the key. + +The contract — `storage/tracked-addresses.ts` is the in-repo reference implementation: + +- read the stored registry and **union it with `entries` by `index`**; +- on a conflicting index, the entry with the greater `updatedAt` supplies `hidden` (ties keep + the incoming entry), and `createdAt` keeps the **earlier** value; +- **serialize concurrent calls**, so one call's read cannot interleave with another's write. + Per provider instance is the floor; because `backingStoreId` explicitly permits several + provider objects over one store, serialize per backing store wherever the platform allows + it (see below); +- a failed write must **not brick later writes**, and must still reject to its own caller. + +A union is safe because there is no delete path: entries are only ever added, and wiping the +wallet removes the key itself (`Sphere.clear()`). Adding a per-entry delete would require +revisiting this contract. + +`index` must be a **uint32** — an integer in `0` … `0xffffffff`, because it is a BIP32 child +number. Rows that are not are **dropped on read**, not repaired (see +[`TrackedAddressEntry`](./API.md#trackedaddressentry)). `loadTrackedAddresses` is otherwise +tolerant: unusable or corrupt storage must read as `[]`, never throw. + +```typescript +import type { StorageProvider, TrackedAddressEntry } from '@unicitylabs/sphere-sdk'; + +/** Tolerant read: unusable JSON and a wrong top-level shape both read as absent. */ +export function parseRegistry(raw: string | null): TrackedAddressEntry[] { + if (!raw) return []; + let parsed: unknown; + try { + parsed = JSON.parse(raw); + } catch { + return []; + } + const rows = (parsed as { addresses?: unknown } | null)?.addresses; + if (!Array.isArray(rows)) return []; + + const num = (v: unknown) => (typeof v === 'number' && Number.isFinite(v) ? v : 0); + return rows.flatMap((row) => { + const e = (row ?? {}) as Record; + const index = e.index; + // Repair the timestamps, but DROP an underivable index: it would alias a real address. + if (typeof index !== 'number' || !Number.isInteger(index) || index < 0 || index > 0xffffffff) { + return []; + } + return [{ + ...e, + index, + hidden: e.hidden === true, + createdAt: num(e.createdAt), + updatedAt: num(e.updatedAt), + } as TrackedAddressEntry]; + }); +} + +/** One write chain per BACKING STORE, so two provider objects over one store cannot interleave. */ +const trackedWrites = new Map>(); + +export async function saveTrackedAddressesMerging( + kv: Pick, + storeId: string, + entries: readonly TrackedAddressEntry[], +): Promise { + const run = (trackedWrites.get(storeId) ?? Promise.resolve()).then(async () => { + const merged = new Map(); + for (const e of parseRegistry(await kv.get('tracked_addresses'))) { + merged.set(e.index, e); + } + for (const e of entries) { + const existing = merged.get(e.index); + if (!existing) { + merged.set(e.index, e); + continue; + } + const winner = e.updatedAt >= existing.updatedAt ? e : existing; // ties keep the incoming entry + merged.set(e.index, { + ...existing, + ...winner, + index: e.index, + createdAt: Math.min(existing.createdAt, e.createdAt), + }); + } + const addresses = [...merged.values()].sort((a, b) => a.index - b.index); + await kv.set('tracked_addresses', JSON.stringify({ version: 1, addresses })); + }); + // The chain tail swallows the rejection so one failed write cannot brick every later + // one; the caller still sees the error by awaiting `run`. + trackedWrites.set(storeId, run.then(() => undefined, () => undefined)); + await run; +} +``` + +The provider then delegates, and reads through the same tolerant parse: + +```typescript +// inside your StorageProvider class +readonly backingStoreId = `mystore:${this.path}`; + +async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { + await saveTrackedAddressesMerging(this, this.backingStoreId, entries); +} + +async loadTrackedAddresses(): Promise { + return parseRegistry(await this.get('tracked_addresses')); +} +``` + +A module-level chain covers several provider objects in **one JS realm** and nothing more. Where +the platform offers a real transaction, use it instead: `IndexedDBStorageProvider` does the whole +read-merge-write in one `readwrite` transaction, which IndexedDB orders across every connection +and every tab. + +Conformance is enforced by `tests/unit/storage/contracts/tracked-addresses.contract.ts` — +run a custom provider through its `describeTrackedAddressesContract()` suite, the same one the +three bundled providers are held to. + ### Transport Provider Interface ```typescript From a6d816d10c26fb04a0404171da72e4d5df7bb5c7 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:02:05 +0200 Subject: [PATCH 19/43] test(mutation): probes for the destroyed latch and the tracked receive drain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eight more, covering both #770 fixes that landed in core/Sphere.ts and modules/payments-v2/. Deliberately NOT probing the two ensureAlive() calls that are defence in depth — an earlier guard throws first on every path that reaches them, so no test kills those and a probe would just report SURVIVED forever. --- tests/mutation/probes.json | 80 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 56e332a2..5affe735 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -931,5 +931,85 @@ "tests": [ "tests/unit/token-engine/factory.test.ts" ] + }, + { + "name": "sphere-switch-starts-vertical-after-destroy", + "note": "#770.3: the lifecycle mutex orders a switch's queued start AFTER destroy()'s stop, so without this gate the pair composes a whole new vertical for an owner whose destroy() already returned", + "file": "core/Sphere.ts", + "find": " if (this._destroyed) return;\n await this.startPaymentsV2Inner(index, identity);", + "replace": " await this.startPaymentsV2Inner(index, identity);", + "tests": [ + "tests/integration/sphere-payments-v2-wiring.test.ts" + ] + }, + { + "name": "sphere-destroyed-latch-set-too-late", + "note": "#770.3: _destroyed must flip as destroy()'s FIRST statement \u2014 set later (or not at all) and the whole teardown window is unguarded, which is exactly why _initialized could not carry it", + "file": "core/Sphere.ts", + "find": " this._destroyed = true;\n", + "replace": "", + "tests": [ + "tests/integration/sphere-payments-v2-wiring.test.ts" + ] + }, + { + "name": "sphere-switch-rebuilds-transport-mux-after-destroy", + "note": "#770.3: ensureTransportMux BUILDS and connect()s a fresh mux whenever _transportMux is null \u2014 which is what destroy() leaves behind \u2014 so an unguarded switch opens new sockets after teardown returned", + "file": "core/Sphere.ts", + "find": " this.ensureAlive();\n await this.initializeAddressModules({ index, identity: newIdentity });", + "replace": " await this.initializeAddressModules({ index, identity: newIdentity });", + "tests": [ + "tests/integration/sphere-payments-v2-wiring.test.ts" + ] + }, + { + "name": "sphere-switch-sets-transport-identity-after-destroy", + "note": "#770.3: setIdentity recreates the NostrClient, so an unguarded switch reconnects the transport after disconnectProvidersIndependently() already ran", + "file": "core/Sphere.ts", + "find": " this.ensureAlive();\n await this._transport.setIdentity(this._identity);", + "replace": " await this._transport.setIdentity(this._identity);", + "tests": [ + "tests/integration/sphere-payments-v2-wiring.test.ts" + ] + }, + { + "name": "sphere-refused-switch-persists-its-index", + "note": "#770.3: a switch destroy() overtook must not leave its index on disk \u2014 persisting it sends the NEXT boot to an address the user never finished moving to", + "file": "core/Sphere.ts", + "find": " this.ensureAlive();\n // Persist current index", + "replace": " // Persist current index", + "tests": [ + "tests/integration/sphere-payments-v2-wiring.test.ts" + ] + }, + { + "name": "sphere-ensureready-ignores-destroyed", + "note": "#770.3: ensureReady() is what every existing caller inherits the destroyed check from; without it a destroyed Sphere answers 'not initialized' semantics only after _initialized is finally cleared", + "file": "core/Sphere.ts", + "find": " this.ensureAlive();\n if (!this._initialized) {", + "replace": " if (!this._initialized) {", + "tests": [ + "tests/unit/core/Sphere.destroy-secrets.test.ts" + ] + }, + { + "name": "receive-drain-untracked-by-quiescence", + "note": "#770.2: a self-spawned poll drain must hold stop() open, or teardown proceeds under live wallet-api I/O, engine.verify and scoped-KV writes", + "file": "modules/payments-v2/receive/Receive.ts", + "find": " const op = this.drainOnce();\n if (this.deps.track !== undefined) this.deps.track(op);\n else void op;", + "replace": " void this.drainOnce();", + "tests": [ + "tests/unit/payments-v2/facade.test.ts" + ] + }, + { + "name": "compose-drops-receive-track-hook", + "note": "#770.2: track is optional on ReceiveDeps, so dropping it here compiles cleanly and silently un-tracks every self-spawned drain", + "file": "modules/payments-v2/compose.ts", + "find": " track: hooks.track,\n", + "replace": "", + "tests": [ + "tests/unit/payments-v2/facade.test.ts" + ] } ] From 55165c4d6a01a91294ca055150dc665f2a4cb192 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:14:29 +0200 Subject: [PATCH 20/43] =?UTF-8?q?test(mutation):=20the=20runner=20rejects?= =?UTF-8?q?=20an=20empty=20replace=20=E2=80=94=20use=20a=20marker=20commen?= =?UTF-8?q?t?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two probes deleted their target line by replacing it with the empty string; the manifest loader requires every field to be a non-empty string, so the whole run aborted before any probe executed. --- tests/mutation/probes.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 5affe735..577ba227 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -947,7 +947,7 @@ "note": "#770.3: _destroyed must flip as destroy()'s FIRST statement \u2014 set later (or not at all) and the whole teardown window is unguarded, which is exactly why _initialized could not carry it", "file": "core/Sphere.ts", "find": " this._destroyed = true;\n", - "replace": "", + "replace": " // probe: the destroyed latch is never set\n", "tests": [ "tests/integration/sphere-payments-v2-wiring.test.ts" ] @@ -1007,7 +1007,7 @@ "note": "#770.2: track is optional on ReceiveDeps, so dropping it here compiles cleanly and silently un-tracks every self-spawned drain", "file": "modules/payments-v2/compose.ts", "find": " track: hooks.track,\n", - "replace": "", + "replace": " // probe: the receive drain is left untracked\n", "tests": [ "tests/unit/payments-v2/facade.test.ts" ] From f0a24bfacc8018270ce861c1a62d1deadbb9d175 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:32:42 +0200 Subject: [PATCH 21/43] fix(health): a present blockNumber is not a height (Copilot round 5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit readBlockNumber accepted any string or number, so {blockNumber:'error'}, an empty string, 1.5 or -1 all reported a healthy aggregator — the same too-permissive reading the HTTP-status check had, one layer down. Heights arrive as decimal strings and eventually outgrow a JS number, so the string form is validated as digits rather than parsed. Also swaps out the one probe that SURVIVED the 101-probe run: sphere-switch-sets-transport-identity-after-destroy. It is not a weak test — the guard it mutates is now SHADOWED by the index-persist guard added later in the same commit, which throws earlier on every path that reaches it. A probe nothing can kill is worse than no probe, so it is removed rather than left reporting SURVIVED forever; the guard stays, marked as defence in depth like the two others in that method. --- core/network-health.ts | 8 ++++++- tests/mutation/probes.json | 20 +++++++++--------- tests/unit/core/network-health.test.ts | 29 ++++++++++++++++++++++++++ 3 files changed, 46 insertions(+), 11 deletions(-) diff --git a/core/network-health.ts b/core/network-health.ts index 0981e38d..a4a498f6 100644 --- a/core/network-health.ts +++ b/core/network-health.ts @@ -261,7 +261,13 @@ function readBlockNumber(body: unknown): string | null { const result = (body as { result?: unknown }).result; if (typeof result !== 'object' || result === null) return null; const n = (result as { blockNumber?: unknown }).blockNumber; - return typeof n === 'string' || typeof n === 'number' ? String(n) : null; + // A height, not merely a present field. The gateway sends it as a decimal STRING + // ("40932"), and heights outstrip Number.MAX_SAFE_INTEGER eventually, so the string + // form is checked as digits rather than parsed. Without this, `{blockNumber: "error"}` + // or `""` reads as a healthy aggregator. + if (typeof n === 'string') return /^\d+$/.test(n) ? n : null; + if (typeof n === 'number') return Number.isInteger(n) && n >= 0 ? String(n) : null; + return null; } /** The `error` member of a JSON-RPC body, or the gateway's bare `{"error": "..."}`. */ diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 577ba227..2edad478 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -962,16 +962,6 @@ "tests/integration/sphere-payments-v2-wiring.test.ts" ] }, - { - "name": "sphere-switch-sets-transport-identity-after-destroy", - "note": "#770.3: setIdentity recreates the NostrClient, so an unguarded switch reconnects the transport after disconnectProvidersIndependently() already ran", - "file": "core/Sphere.ts", - "find": " this.ensureAlive();\n await this._transport.setIdentity(this._identity);", - "replace": " await this._transport.setIdentity(this._identity);", - "tests": [ - "tests/integration/sphere-payments-v2-wiring.test.ts" - ] - }, { "name": "sphere-refused-switch-persists-its-index", "note": "#770.3: a switch destroy() overtook must not leave its index on disk \u2014 persisting it sends the NEXT boot to an address the user never finished moving to", @@ -1011,5 +1001,15 @@ "tests": [ "tests/unit/payments-v2/facade.test.ts" ] + }, + { + "name": "health-accepts-any-blocknumber", + "note": "#769.1: the field being PRESENT is not a height. Accepting any string/number reports a gateway healthy on a body carrying no usable answer ({blockNumber:'error'}, '', 1.5, -1)", + "file": "core/network-health.ts", + "find": " if (typeof n === 'string') return /^\\d+$/.test(n) ? n : null;\n if (typeof n === 'number') return Number.isInteger(n) && n >= 0 ? String(n) : null;\n return null;", + "replace": " return typeof n === 'string' || typeof n === 'number' ? String(n) : null;", + "tests": [ + "tests/unit/core/network-health.test.ts" + ] } ] diff --git a/tests/unit/core/network-health.test.ts b/tests/unit/core/network-health.test.ts index 638d962c..b39f9dee 100644 --- a/tests/unit/core/network-health.test.ts +++ b/tests/unit/core/network-health.test.ts @@ -68,6 +68,35 @@ describe('checkNetworkHealth', () => { expect(result.services.oracle!.error).toContain('no such method'); }); + it.each([ + ['a non-numeric string', 'error'], + ['an empty string', ''], + ['a fractional number', 1.5], + ['a negative number', -1], + ])('reports unhealthy when result.blockNumber is %s', async (_label, blockNumber) => { + // The field being PRESENT is not a height. Accepting any string or number here + // reports a gateway healthy on a body that carries no usable answer. + fetchSpy.mockResolvedValueOnce( + new Response(JSON.stringify({ jsonrpc: '2.0', result: { blockNumber }, id: 1 }), { + status: 200, + }), + ); + + const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); + + expect(result.services.oracle!.healthy).toBe(false); + }); + + it('accepts a decimal-string height beyond Number.MAX_SAFE_INTEGER', async () => { + // Heights arrive as decimal strings and eventually outgrow a JS number, so the + // string form is validated as digits rather than parsed. + fetchSpy.mockResolvedValueOnce(blockHeightBody('90071992547409910')); + + const result = await checkNetworkHealth('testnet', { services: ['oracle'] }); + + expect(result.services.oracle!.healthy).toBe(true); + }); + it("surfaces the gateway's own error string from a 400 rather than a bare status", async () => { fetchSpy.mockResolvedValueOnce( new Response(JSON.stringify({ error: 'Shard ID not found: 0' }), { From e232826e3232aa0f8476de86d2eba8bf86342718 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:33:11 +0200 Subject: [PATCH 22/43] docs(changelog): the #770 teardown fixes, #769.1 and the docs sweep --- CHANGELOG.md | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a845a24..30109bb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,73 @@ worker pool at the documented entry point silently gave you the sequential verif The logger stays process-global by design: most of its 370 call sites are in providers constructed before any Sphere exists and shared between them. +### Fixed — teardown no longer leaves live work behind (#770) + +Four defects of one shape: work spawned by a component outlived the component. + +- **`setIdentity` orphaned the previous `NostrClient`** (`transport/`). It assigned the + replacement to `this.nostrClient` *before* connecting, so a failed connect left the old + client's sockets, ping intervals and auto-reconnect chain running and unreachable — + `disconnect()` only ever reaches the field. `setIdentity` never touches `status`, so the + caller's next retry orphaned another. The field now moves only after a successful connect; + a failed connect disposes the replacement and leaves the provider on the working client, + and a throwing `subscribeToEvents()` disposes the old one from a `finally`. Deliberately + not "tear down both": `MultiAddressTransportMux` shares that client. Folds in the + same-class timer leak in `connect()` — `Promise.race` does not cancel the loser, so the + deadline timer pinned Node's event loop for the full timeout after the call returned. +- **`engine.dispose()` mid-verify never settled the batch** (`token-engine/`). The base + SDK's `WorkerPool.dispose()` only `terminate()`s its workers; a dispatched task resolves + solely from `worker.onmessage`, so `verify` hung forever. Reachable through + `setOracleApiKey` → `PaymentsFacade.setEngine`. Disposal now **rejects** the in-flight + verification, and that is load-bearing rather than stylistic: the only `engine.verify` + caller in the money path turns a falsy verdict into a permanent mailbox rejection, so + resolving `{ ok: false }` would have destroyed a **valid incoming token** because an + api-key change happened to land mid-drain. A rejection leaves the entry unacked to + re-list. `createWorker()` also refuses once disposed, so a late task cannot resurrect the + pool. Only affects consumers who opt into `verification.createWorker`. +- **The mailbox poll drain was untracked** (`modules/payments-v2/`). `Receive.start()` + spawned its 30 s poll with a bare `void drainOnce()`, so `PaymentsFacade.stop()` could + return while a drain was still doing wallet-api I/O, verification, scoped-KV writes and + `transfer:incoming` emission — and `destroy()` carried on underneath it. Both spawns now + register with the existing quiescence gate. (The wake path was accidentally covered, but + only because of subscriber ordering.) +- **A `switchToAddress` racing `destroy()` re-armed the wallet** (`core/`). + `switchToAddress` checked liveness once and then awaited ~8 times, and `_initialized` is + cleared *last*, so it could not mark "teardown has begun". Two live vectors: a switch + rebuilt and reconnected the transport mux (`destroy()` leaves it null, which is exactly + the condition to build one), and its stop/start pair — queued behind `destroy()`'s own + stop on the lifecycle mutex — started a whole new vertical for an owner whose `destroy()` + had already resolved. A destroyed latch, set as `destroy()`'s first statement, now gates + both. A refused switch also no longer persists the address index it never finished moving + to, which would have sent the next boot to the wrong address. + +### Fixed — `checkNetworkHealth` reported healthy gateways as unhealthy (#769) + +It POSTed `get_round_number` with empty params and keyed the verdict off `response.ok`. The +gateway is a routing layer: it refuses any call carrying neither `stateId` nor `shardId` +with HTTP 400 before it looks at the method. So the check that exists to gate a network +cutover answered "unhealthy" for every live gateway. It now sends `get_block_height` with a +32-byte `stateId` and reads the JSON-RPC body — the status code cannot answer this in +either direction, since a healthy gateway answers a routing mistake with a 400 plus a body +and JSON-RPC puts application errors inside a 200. Verified live against testnet2 and +mainnet. + +### Documentation + +`docs/INTEGRATION.md` published the `StorageProvider` interface with `saveTrackedAddresses` +and no contract, so a custom provider written from it reproduced the #766 data-loss bug; it +now carries the write contract, the uint32 rule and a worked implementation. `Sphere.init`'s +option block, `TrackedAddressEntry.index`, `sphere.isReady`/`networkId`, the backing-store +scoping of `clear()`/`import()` and the new `TokenRegistry` instance API are documented, and +the token-registry migration guide is reachable from the README. Four CLAUDE.md claims the +code does not honour were corrected (there is no durable receive seen-set; coin symbols are +not registry-resolved on the money path). + +`tests/aggregator/` gained the `INVALID_TRUSTBASE` vacuity guard CLAUDE.md already claimed: +a token the real service certified must be refused when the trust base carries a wrong root +key. Without it, "verify() passes against a real aggregator" could have been true because +verification never consulted the trust base. + ### Fixed — `TokenRegistry.resetInstance()` now disposes (#770) It called only `stopAutoRefresh()`, leaving `disposed` unset, the generation unchanged and From c966b1e07f39d2fac38f98ce21b36e064baa629b Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:40:53 +0200 Subject: [PATCH 23/43] fix(transport): the client swap must lose to a concurrent teardown, and the identity moves with it (Copilot round 5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two findings on the setIdentity swap, both real. 1. Resurrect-after-teardown. Moving `this.nostrClient` to AFTER a successful connect closed the orphan leak but opened a window: disconnect() nulls the field and sets status='disconnected', and nothing tied the pending connect to the provider that started it — so when it resolved, the swap installed a live client plus subscriptions owned by nobody, and told the caller the identity took. It now re-checks ownership before any field moves, disposes the replacement and REJECTS: with the staging below, neither the client nor the identity was applied, so resolving would be a lie. Covers a competing setIdentity for free — the loser disposes its own replacement. 2. Half-applied identity. identity, keyManager and the dedup window were written before any client work, so a failed swap — which now correctly keeps the OLD client and its old-address subscriptions — ran them under the NEW key: old gift wraps decrypted with the wrong key, the new address subscribed nowhere. They are staged and committed with the client. Staging rather than restoring because every mid-flight reader of keyManager (getNostrPubkey, gift-wrap send, subscribeToEvents, publishIdentityBinding) then reads the key the still-installed old client is actually subscribed with. Behaviour change worth knowing: a failed setIdentity no longer half-applies, so a caller retrying after a transient relay failure re-attempts the whole swap instead of resuming on a provider that had silently already taken the new key. Note for follow-up, not fixed here: the `else if (this.isConnected())` branch is unreachable — isConnected() is exactly the first branch's condition. Left in place, staged correctly, rather than deleting dead code outside the finding. --- ...NostrTransportProvider.setIdentity.test.ts | 78 ++++++++++++++++++- transport/NostrTransportProvider.ts | 69 +++++++++++----- 2 files changed, 126 insertions(+), 21 deletions(-) diff --git a/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts b/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts index 9f7d94ea..6f43a162 100644 --- a/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts +++ b/tests/unit/transport/NostrTransportProvider.setIdentity.test.ts @@ -14,6 +14,12 @@ * `oldClient.disconnect()` only on the success tail — so a failed connect * orphaned the old client (open socket, unreachable from the provider, so * `disconnect()` could not reach it either) and every retry leaked another one. + * + * Two further invariants, from the #772 review: + * - a `disconnect()` that lands mid-swap wins — the replacement is disposed, + * never installed into a provider nobody owns any more; + * - identity + key manager + dedup window move WITH the client, so a failed + * swap leaves the old client running under its own key, not the new one. */ import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; @@ -43,6 +49,8 @@ const clients: MockClient[] = []; const rejectConnectFor = new Set(); const hangConnectFor = new Set(); const throwSubscribeFor = new Set(); +/** Connects that stay pending until the test resolves the gate, keyed by index. */ +const gateConnectFor = new Map>(); vi.mock('@unicitylabs/nostr-js-sdk', async (importOriginal) => { const actual = await importOriginal(); @@ -55,6 +63,8 @@ vi.mock('@unicitylabs/nostr-js-sdk', async (importOriginal) => { connect: vi.fn(async () => { if (rejectConnectFor.has(id)) throw new Error(`relay refused (client ${id})`); if (hangConnectFor.has(id)) await new Promise(() => { /* never settles */ }); + const gate = gateConnectFor.get(id); + if (gate) await gate; }), disconnect: vi.fn(), isConnected: vi.fn().mockReturnValue(true), @@ -82,12 +92,12 @@ const { NostrTransportProvider } = await import('../../../transport/NostrTranspo const TIMEOUT_MS = 50; -function createProvider() { +function createProvider(timeout: number = TIMEOUT_MS) { return new NostrTransportProvider({ relays: ['wss://relay1.test'], // Inert: the (mocked) SDK NostrClient owns its own sockets. createWebSocket: (() => {}) as unknown as WebSocketFactory, - timeout: TIMEOUT_MS, + timeout, autoReconnect: false, }); } @@ -105,6 +115,17 @@ function currentClient(provider: InstanceType): M return (provider as unknown as { nostrClient: MockClient | null }).nostrClient; } +type ProviderInternals = { + identity: { privateKey: string; chainPubkey: string } | null; + processedEventIds: Set; + lastDmEventTs: number; +}; + +/** No public accessor exists for these three — the cast is the only reader. */ +function internals(provider: InstanceType): ProviderInternals { + return provider as unknown as ProviderInternals; +} + /** * A client is leaked when it is neither disposed nor the provider's current * client: nothing can ever reach it again, and its socket stays open. @@ -127,6 +148,7 @@ describe('NostrTransportProvider — setIdentity() client swap', () => { rejectConnectFor.clear(); hangConnectFor.clear(); throwSubscribeFor.clear(); + gateConnectFor.clear(); }); afterEach(() => { @@ -204,6 +226,58 @@ describe('NostrTransportProvider — setIdentity() client swap', () => { } }); + it('disposes the replacement when disconnect() lands mid-swap', async () => { + // Deadline well past the gate: the rejection must come from the ownership + // check, not from connectWithDeadline timing the gated connect out. + const provider = createProvider(10_000); + await provider.connect(); // client 0 + await provider.setIdentity(identity(1)); // client 1 + + let openTheRelay!: () => void; + gateConnectFor.set(2, new Promise((resolve) => { openTheRelay = resolve; })); + const swap = provider.setIdentity(identity(2)); // client 2, connect pending + await vi.waitFor(() => expect(clients).toHaveLength(3)); + + await provider.disconnect(); + openTheRelay(); + await expect(swap).rejects.toThrow(/during setIdentity - identity not applied/); + + expect(currentClient(provider)).toBeNull(); // no resurrection + expect(clients[2].disconnect).toHaveBeenCalledTimes(1); + expect(clients[2].subscribe).not.toHaveBeenCalled(); // no post-teardown subs + expect(clients[1].disconnect).toHaveBeenCalledTimes(1); + expect(leakedClients(provider)).toEqual([]); + expect(provider.getStatus()).toBe('disconnected'); + }); + + it('keeps the OLD identity, key manager and dedup window when the swap fails', async () => { + const provider = createProvider(); + await provider.connect(); + await provider.setIdentity(identity(1)); + + const oldPubkey = provider.getNostrPubkey(); + internals(provider).processedEventIds.add('event-seen-on-old-address'); + internals(provider).lastDmEventTs = 1234; + + rejectConnectFor.add(2); + await expect(provider.setIdentity(identity(2))).rejects.toThrow(/relay refused/); + + // The old client is still installed and still subscribed with the old key. + expect(currentClient(provider)).toBe(clients[1]); + expect(provider.getNostrPubkey()).toBe(oldPubkey); + expect(internals(provider).identity).toEqual(identity(1)); + expect(internals(provider).processedEventIds.has('event-seen-on-old-address')).toBe(true); + expect(internals(provider).lastDmEventTs).toBe(1234); + + // ...and a swap that SUCCEEDS still applies everything. + await provider.setIdentity(identity(3)); + expect(currentClient(provider)).toBe(clients[3]); + expect(provider.getNostrPubkey()).not.toBe(oldPubkey); + expect(internals(provider).identity).toEqual(identity(3)); + expect(internals(provider).processedEventIds.size).toBe(0); + expect(internals(provider).lastDmEventTs).toBe(0); + }); + it('leaves no pending connect-deadline timer behind', async () => { vi.useFakeTimers(); const provider = createProvider(); diff --git a/transport/NostrTransportProvider.ts b/transport/NostrTransportProvider.ts index 263dc7df..93d035b0 100644 --- a/transport/NostrTransportProvider.ts +++ b/transport/NostrTransportProvider.ts @@ -491,25 +491,36 @@ export class NostrTransportProvider implements TransportProvider { // =========================================================================== async setIdentity(identity: FullIdentity): Promise { - this.identity = identity; - - // Clear per-address state so stale dedup entries from previous address - // don't block legitimate events for the new address. - this.processedEventIds.clear(); - this.lastEventTs = 0; - this.lastDmEventTs = 0; - this.fallbackDmSince = null; - - // Create NostrKeyManager from private key + // Staged, NOT applied: the key material belongs to the client that will + // carry it. Applying it up front left a FAILED swap running the old client + // and its old-address subscriptions under the NEW key — gift wraps for the + // old address decrypted with the wrong key, the new address subscribed + // nowhere, and no error path that could put either back. const secretKey = Buffer.from(identity.privateKey, 'hex'); - this.keyManager = NostrKeyManager.fromPrivateKey(secretKey); - - // Use Nostr-format pubkey (32 bytes / 64 hex chars) from keyManager - const nostrPubkey = this.keyManager.getPublicKeyHex(); - logger.debug('Nostr', 'Identity set, Nostr pubkey:', nostrPubkey.slice(0, 16) + '...'); + const nextKeyManager = NostrKeyManager.fromPrivateKey(secretKey); + + // Nostr-format pubkey (32 bytes / 64 hex chars) + const nostrPubkey = nextKeyManager.getPublicKeyHex(); + logger.debug('Nostr', 'Identity staged, Nostr pubkey:', nostrPubkey.slice(0, 16) + '...'); + + /** + * Commit the staged identity — always together with the client it belongs + * to, never before it. The per-address dedup window is reset here for the + * same reason: stale entries from the previous address must not block the + * new address's events, and wiping them for a swap that then FAILS would + * re-admit already-processed events on the address still subscribed. + */ + const applyStagedIdentity = (): void => { + this.identity = identity; + this.keyManager = nextKeyManager; + this.processedEventIds.clear(); + this.lastEventTs = 0; + this.lastDmEventTs = 0; + this.fallbackDmSince = null; + }; - // If we already have a NostrClient with a temp key, we need to reconnect with the real key - // NostrClient doesn't support changing key at runtime + // NostrClient cannot swap its key at runtime, so an identity change while + // connected means building a replacement client and handing it the socket. if (this.nostrClient && this.status === 'connected') { logger.debug('Nostr', 'Identity changed while connected - recreating NostrClient'); const oldClient = this.nostrClient; @@ -519,7 +530,7 @@ export class NostrTransportProvider implements TransportProvider { // orphans `oldClient` — socket open but unreachable from the provider, so // `disconnect()` cannot reach it either — and setIdentity never touches // `status`, so the caller's next retry re-enters here and leaks another. - const nextClient = new NostrClient(this.keyManager, { + const nextClient = new NostrClient(nextKeyManager, { autoReconnect: this.config.autoReconnect, reconnectIntervalMs: this.config.reconnectDelay, maxReconnectIntervalMs: this.config.reconnectDelay * 16, @@ -557,18 +568,38 @@ export class NostrTransportProvider implements TransportProvider { throw error; } + // A `disconnect()` (or a competing swap) during the await above already + // tore this provider down: installing a freshly connected client now + // resurrects it — live socket plus subscriptions, owned by nobody, with + // the caller told the identity took. Dispose the replacement and reject, + // because neither the client nor the identity was applied. + if (this.nostrClient !== oldClient || this.status !== 'connected') { + try { nextClient.disconnect(); } catch { /* best-effort cleanup */ } + throw new SphereError( + 'Transport client changed or left the connected state during setIdentity' + + ' - identity not applied', + 'TRANSPORT_ERROR' + ); + } + // The swap is committed. From here on `oldClient` is unreachable from the // provider, so it must be disposed on EVERY path out — including a // throwing `subscribeToEvents()`. this.nostrClient = nextClient; + applyStagedIdentity(); try { await this.subscribeToEvents(); } finally { try { oldClient.disconnect(); } catch { /* best-effort cleanup */ } } } else if (this.isConnected()) { - // Already connected with right key, just subscribe + // No client swap needed — commit first, since subscribeToEvents() reads + // `identity`/`keyManager` to build its filters. + applyStagedIdentity(); await this.subscribeToEvents(); + } else { + // Not connected: no client work to fail, so nothing to stage against. + applyStagedIdentity(); } } From ac47c5a0f0ddd0e9b7ff09851d3b8b359cd9fbc8 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:41:21 +0200 Subject: [PATCH 24/43] test(mutation): probes for the swap ownership check and the staged identity --- tests/mutation/probes.json | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 2edad478..dc075d52 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -1011,5 +1011,25 @@ "tests": [ "tests/unit/core/network-health.test.ts" ] + }, + { + "name": "transport-resurrects-client-after-teardown", + "note": "#772 review: a disconnect() during the swap's connect must win \u2014 installing nextClient afterwards leaves a live socket + subscriptions owned by nobody, with the caller told the identity took", + "file": "transport/NostrTransportProvider.ts", + "find": " if (this.nostrClient !== oldClient || this.status !== 'connected') {", + "replace": " if (false) {", + "tests": [ + "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" + ] + }, + { + "name": "transport-applies-identity-before-client", + "note": "#772 review: identity + key manager + dedup window move WITH the client; applied eagerly, a failed swap runs the OLD client and its old-address subscriptions under the NEW key", + "file": "transport/NostrTransportProvider.ts", + "find": " const applyStagedIdentity = (): void => {\n this.identity = identity;\n this.keyManager = nextKeyManager;", + "replace": " this.identity = identity;\n this.keyManager = nextKeyManager;\n const applyStagedIdentity = (): void => {", + "tests": [ + "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" + ] } ] From f4d9eea1d724cd83c3cdf99c2beaeb46ab2c5e63 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:44:18 +0200 Subject: [PATCH 25/43] fix(token-engine): the verification cancellation must be per call, not one shared promise (Copilot round 5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Racing every verify() against a single long-lived never-settling promise leaks: Promise.race subscribes to every input and does not detach when another input wins, so each finished verification pinned a reaction record to that promise until dispose(). Retention grew with every token the wallet had ever verified — inside the very class added to fix a teardown bug. Receive.screen() calls engine.verify once per incoming token, so it grew with received volume. Each call now registers its own rejector and drops it the instant the call settles, so retention is bounded by CONCURRENT verifications rather than by lifetime count. Not the reviewer's deferred-plus-race shape, which allocates three promises per call (the deferred, the Promise.resolve wrapper, the race result). verify() constructs the promise it already owes the caller and drives it directly: one promise per call, and the unhandled-rejection question dissolves rather than being managed — the only promise this class creates per call is the RETURNED one, so its rejection is the caller's to observe exactly as before, and nothing outlives the call that created it. A verdict arriving after a cancellation re-settles an already-settled promise, which is a no-op, so a late verdict can never downgrade the MODULE_DESTROYED rejection into a falsy verdict. Scope: only bites consumers who pass verification.createWorker; without it the class is never constructed. pendingCancellations makes the bookkeeping assertable without heap introspection — the new test pins peak 1 over 300 sequential verifications, and a concurrency case keeps a hard-wired 0 from passing it vacuously. The four existing guards were each re-reverted after the refactor and still red, including the money pin (a cancellation that resolves a falsy verdict). --- .../token-engine/worker-verification.test.ts | 91 ++++++++++++++++++- token-engine/factory.ts | 76 +++++++++++----- 2 files changed, 144 insertions(+), 23 deletions(-) diff --git a/tests/unit/token-engine/worker-verification.test.ts b/tests/unit/token-engine/worker-verification.test.ts index 5e6ff488..61be97be 100644 --- a/tests/unit/token-engine/worker-verification.test.ts +++ b/tests/unit/token-engine/worker-verification.test.ts @@ -16,7 +16,22 @@ import { beforeAll, describe, expect, it, vi } from 'vitest'; import { createSphereTokenEngine, createWorkerTokenVerifier } from '../../../token-engine/factory'; import type { SphereToken, VerificationWorker } from '../../../token-engine'; -import { SigningService, VerificationStatus, WorkerTokenVerifier } from '../../../token-engine/sdk'; +import { + MintJustificationVerifierService, + PredicateVerifierService, + RootTrustBase, + Secp256k1SignatureVerifier, + SigningService, + SplitMintJustificationVerifier, + TokenIssuanceVerifierService, + UnicityCertificateVerifier, + UnicitySealQuorumSignaturesVerificationRule, + VerificationContext, + VerificationStatus, + VerifiedSealCache, + WorkerTokenVerifier, +} from '../../../token-engine/sdk'; +import { decodeSpherePaymentData } from '../../../token-engine/SpherePaymentData'; import { TestAggregatorClient } from './support/TestAggregatorClient'; import { createTestEngine, freshPubkey } from './test-engine'; @@ -321,6 +336,80 @@ describe('dispose() during an in-flight verification (#770 item 4)', () => { expect(createWorker).not.toHaveBeenCalled(); }, 30000); + /** + * The same context the factory builds — the verifier needs a REAL one: it + * verifies the genesis on the calling thread and only fans the transfers out. + */ + const verificationContext = (): VerificationContext => { + const trustBase = RootTrustBase.fromJSON(trustBaseJson); + const mintJustificationVerifier = new MintJustificationVerifierService(); + mintJustificationVerifier.register(new SplitMintJustificationVerifier(decodeSpherePaymentData)); + return new VerificationContext( + trustBase, + PredicateVerifierService.create(), + new UnicityCertificateVerifier( + new UnicitySealQuorumSignaturesVerificationRule(new Secp256k1SignatureVerifier(), new VerifiedSealCache(256)) + ), + mintJustificationVerifier, + new TokenIssuanceVerifierService(false) + ); + }; + + /** + * The cancellation `dispose()` fires must not be paid for by every verification + * that ever SUCCEEDED. One shared never-settling promise raced against every + * call would be: `Promise.race` subscribes to every input and never detaches + * when another input wins, so a finished verification leaves its reaction + * pinned to that promise until dispose() — unbounded retention in a wallet that + * verifies a token on every receive, mint and resync. + * + * `pendingCancellations` is that retention made observable (no heap assertions: + * they would be flaky and prove nothing about the structure). + */ + describe('cancellation retention tracks concurrency, not history', () => { + it('counts the verifications IN FLIGHT — up while they run, back to 0 when they settle', async () => { + const verifier = createWorkerTokenVerifier({ createWorker: () => new FakeWorker(), poolSize: 3 }); + const context = verificationContext(); + expect(verifier.pendingCancellations).toBe(0); + + // Registered synchronously by verify(), so this reads the true in-flight set. + // Without this half the leak guard below would pass on a counter stuck at 0. + const inFlight = [ + verifier.verify(oneTransfer.sdkToken, context), + verifier.verify(oneTransfer.sdkToken, context), + verifier.verify(oneTransfer.sdkToken, context), + ]; + expect(verifier.pendingCancellations).toBe(3); + + const results = await Promise.all(inFlight); + expect(results.map((r) => r.status)).toEqual([ + VerificationStatus.OK, + VerificationStatus.OK, + VerificationStatus.OK, + ]); + expect(verifier.pendingCancellations).toBe(0); + verifier.dispose(); + }, 30000); + + it('retains nothing per completed verification — 300 sequential verifies leave 0', async () => { + const verifier = createWorkerTokenVerifier({ createWorker: () => new FakeWorker() }); + const context = verificationContext(); + + let peak = 0; + for (let i = 0; i < 300; i++) { + const verifying = verifier.verify(oneTransfer.sdkToken, context); + peak = Math.max(peak, verifier.pendingCancellations); + expect((await verifying).status).toBe(VerificationStatus.OK); + } + + // Sequential calls: at most ONE cancellation is ever live, and none survives + // the call that created it. A shape that only clears on dispose() reads 300. + expect(peak).toBe(1); + expect(verifier.pendingCancellations).toBe(0); + verifier.dispose(); + }, 60000); + }); + it('a batch dispatched AFTER dispose() cannot resurrect the pool', async () => { // Deterministic ordering, no timing guess: two transfers + poolSize 2 means // WorkerPool.dispatch() loops twice. The FIRST worker calls dispose() from diff --git a/token-engine/factory.ts b/token-engine/factory.ts index 4e947d43..1dc1d974 100644 --- a/token-engine/factory.ts +++ b/token-engine/factory.ts @@ -52,7 +52,7 @@ function disposedError(): SphereError { * the cast is the port boundary (same web-`Worker` subset, payloads `unknown` on * our side so no SDK wire type escapes). * - * ── #770(4): dispose() must SETTLE the in-flight batch ────────────────────── + * ── #770(4): dispose() must SETTLE the in-flight batch ────────────────────────── * The SDK's `WorkerPool.dispose()` (3.0.1) only calls `worker.terminate()` on * every worker it spawned. A dispatched task resolves ONLY from `worker.onmessage`, * which a terminated worker never posts, and queued tasks are never drained — so @@ -69,11 +69,24 @@ function disposedError(): SphereError { * incoming token thrown away at the mailbox because an api-key change happened * to land mid-drain. A rejection instead propagates to the drain's catch, which * leaves the entry UNACKED so it re-lists on the next drain. + * + * ── Why the cancellation is PER CALL, not one shared promise ────────────────── + * Racing every verify() against a single long-lived never-settling promise + * LEAKS: `Promise.race` subscribes to every input and does not detach when + * another input wins, so each finished verification pins a reaction record to + * that promise until dispose() — retention growing with every token the wallet + * has ever verified, inside the very class added to fix a teardown bug. So each + * call registers its own rejector in `cancellations` and drops it the instant + * the call settles: retention is bounded by CONCURRENT verifications, never by + * lifetime count. No parked `.catch()` is needed to keep a cancellation from + * becoming an unhandled rejection either — the only promise built per call is + * the one RETURNED, so its rejection is the caller's to observe, as before, and + * no promise this class creates outlives the call that created it. */ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { private disposed = false; - /** Lazily created so an idle verifier holds no promise at all. */ - private cancellation: { promise: Promise; reject: (error: unknown) => void } | null = null; + /** One rejector per IN-FLIGHT verify(), dropped on settle; empty when idle. */ + private readonly cancellations = new Set<() => void>(); public constructor( private readonly spawn: () => VerificationWorker, @@ -82,12 +95,35 @@ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { super(poolSize); } + /** Verifications awaiting a verdict — back to 0 whenever the verifier is idle. */ + public get pendingCancellations(): number { + return this.cancellations.size; + } + /** Rejects (see the class note) if `dispose()` lands before the pool answers. */ public override verify( ...args: Parameters ): ReturnType { if (this.disposed) return Promise.reject(disposedError()); - return Promise.race([super.verify(...args), this.cancelSignal()]); + // Started before its cancellation is registered, but nothing can interleave: + // an async function runs synchronously up to its first await. + const verdict = super.verify(...args); + return new Promise((resolve, reject) => { + const cancel = (): void => reject(disposedError()); + this.cancellations.add(cancel); + // Settling twice is a no-op, so a verdict landing after dispose() cancelled + // the call can never downgrade that rejection into a falsy verdict. + void verdict.then( + (result) => { + this.cancellations.delete(cancel); + resolve(result); + }, + (error: unknown) => { + this.cancellations.delete(cancel); + reject(error); + } + ); + }); } /** Idempotent — Sphere.setOracleApiKey disposes the replaced engine twice (facade + caller). */ @@ -96,8 +132,10 @@ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { this.disposed = true; super.dispose(); // terminate() every spawned worker, as before // Only AFTER the pool is down: settle whatever the terminated workers will - // now never answer. A no-op when nothing was ever verified. - this.cancellation?.reject(disposedError()); + // now never answer. A no-op when nothing is in flight. + const cancelling = [...this.cancellations]; + this.cancellations.clear(); + for (const cancel of cancelling) cancel(); } protected createWorker(): IWorker { @@ -107,26 +145,20 @@ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { if (this.disposed) throw disposedError(); return this.spawn() as unknown as IWorker; } +} - private cancelSignal(): Promise { - if (this.cancellation === null) { - let reject!: (error: unknown) => void; - const promise = new Promise((_resolve, rejectFn) => { - reject = rejectFn; - }); - // Belt and braces: today the only caller hands this straight to - // `Promise.race`, which subscribes to it, so dispose()'s rejection is - // always observed. Park a no-op handler anyway so a future caller that - // drops the promise cannot turn a teardown into an unhandled rejection. - void promise.catch(() => undefined); - this.cancellation = { promise, reject }; - } - return this.cancellation.promise; - } +/** + * A verifier whose in-flight verifications `dispose()` cancels. `pendingCancellations` + * is the observable that the bookkeeping behind that is per-call and short-lived + * (see the class note): it must return to 0 as verifications settle, never grow + * with the number of tokens verified. + */ +export interface CancellableTokenVerifier extends DisposableTokenVerifier { + readonly pendingCancellations: number; } /** Workers spawn LAZILY on first verify and are reused, so this costs nothing to build. */ -export function createWorkerTokenVerifier(config: VerificationWorkerConfig): DisposableTokenVerifier { +export function createWorkerTokenVerifier(config: VerificationWorkerConfig): CancellableTokenVerifier { const poolSize = config.poolSize ?? DEFAULT_VERIFICATION_POOL_SIZE; if (!Number.isInteger(poolSize) || poolSize < 1) { throw new TypeError(`verification.poolSize must be a positive integer, got ${String(config.poolSize)}`); From 9627469a417b81f4a4cecfbaa72f69766921b17b Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:44:44 +0200 Subject: [PATCH 26/43] test(mutation): re-point two verifier probes after the per-call refactor, add the leak probe MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two find strings stopped existing with the shared-promise shape; both were re-pointed to the equivalent mutation rather than deleted, and the new verifier-cancellation-leak probe pins the release itself (a single-line find on the delete would be ambiguous — it occurs twice). --- tests/mutation/probes.json | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index dc075d52..f1e716f5 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -884,10 +884,10 @@ }, { "name": "verifier-dispose-no-cancel", - "note": "#770.4: dispose() terminates the workers but the SDK pool never settles their tasks; without the race, verify() hangs forever and stop()/destroy() hang with it", + "note": "#770.4: dispose() terminates the workers but the SDK pool never settles their tasks; with the call's cancellation unregistered, verify() hangs forever and stop()/destroy() hang with it", "file": "token-engine/factory.ts", - "find": " return Promise.race([super.verify(...args), this.cancelSignal()]);", - "replace": " return super.verify(...args);", + "find": " this.cancellations.add(cancel);", + "replace": " // probe: the call's cancellation is never registered", "tests": [ "tests/unit/token-engine/worker-verification.test.ts" ] @@ -896,8 +896,8 @@ "name": "verifier-cancel-resolves-not-rejects", "note": "#770.4 MONEY: a cancelled verification must REJECT. Resolving a falsy verdict makes Receive.screen() rejectAck(entry,'invalid') \u2014 a VALID incoming token destroyed because an api-key change landed mid-drain", "file": "token-engine/factory.ts", - "find": " const promise = new Promise((_resolve, rejectFn) => {\n reject = rejectFn;\n });", - "replace": " const promise = new Promise((resolveFn) => {\n reject = resolveFn as unknown as (error: unknown) => void;\n });", + "find": " const cancel = (): void => reject(disposedError());", + "replace": " const cancel = (): void => resolve({ status: 'FAIL' } as unknown as Awaited>);", "tests": [ "tests/unit/token-engine/worker-verification.test.ts" ] @@ -1031,5 +1031,15 @@ "tests": [ "tests/unit/transport/NostrTransportProvider.setIdentity.test.ts" ] + }, + { + "name": "verifier-cancellation-leak", + "note": "#770.4 review: a call's cancellation entry must be released when that call SETTLES. Holding it until dispose() is the shared-promise leak again \u2014 one retained entry per token the wallet has ever verified, inside the class added to fix a teardown bug", + "file": "token-engine/factory.ts", + "find": " (result) => {\n this.cancellations.delete(cancel);\n resolve(result);\n },\n (error: unknown) => {\n this.cancellations.delete(cancel);\n reject(error);\n }", + "replace": " (result) => {\n resolve(result);\n },\n (error: unknown) => {\n reject(error);\n }", + "tests": [ + "tests/unit/token-engine/worker-verification.test.ts" + ] } ] From 3c0d73a03a1bc9fac8b2ed0697f0adac18a20dfc Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:58:26 +0200 Subject: [PATCH 27/43] fix(sphere): refuse to publish over a store cleared under the init; enforce uint32 before derivation (Copilot round 5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two findings, both real. 1. clear() could wipe a store out from under an in-flight init. An init is invisible to clear() until it PUBLISHES (#767's invariant), so create() — which writes the mnemonic before bring-up — could have its KV emptied and then publish an isReady Sphere over nothing. A per-backing-store clear generation, recorded before the first storage work and re-checked at publication. NOT a per-store mutex: import() clears internally, so a mutex round-trips through itself, and a clear() blocked behind a slow network-bound bring-up is a worse outage than a typed refusal. Bumped on ENTRY and again in a finally — one bump is insufficient in either position, and both directions have their own test: exit-only misses an init publishing DURING the clear, entry-only misses an init that starts mid-clear and records the already-bumped value. Publication moved INSIDE withOwnedRegistry so check and registration are one synchronous step (split by an await, a clear lands between them) and a refusal reuses the existing teardown. import() records its generation AFTER its own internal clear — recording earlier would make every import refuse itself. 2. The uint32 index rule was enforced only on read. mergeTrackedAddresses now throws on an underivable INCOMING index while still filtering bad stored rows (one bad row must not brick every later write), and derivation is guarded at _deriveAddressInternal — the single point every index reaches derivation through. Storage-layer refusal alone is too late: ensureAddressTracked mutates the in-memory registry before persisting, and the falsification shows 7 aliasing rows where 2 were expected. Deliberately NOT a separate guard at switchToAddress, as the review suggested: derivation refuses first on every reachable path, so it would be exactly the non-falsifiable guard whose probe was just deleted. The old index < 0 check is removed rather than widened — negatives now throw the same INVALID_CONFIG with a better message. Known gap, documented rather than hidden: on IndexedDB the write-side throw lands inside read.onsuccess and aborts the transaction, so the caller sees the generic abort rather than the typed reason. The port docstring and docs/INTEGRATION.md now tell implementations to validate before opening the transaction. --- core/Sphere.ts | 97 +++++++++++++-- docs/API.md | 21 +++- docs/INTEGRATION.md | 18 ++- storage/storage-provider.ts | 8 +- storage/tracked-addresses.ts | 17 ++- .../sphere-instance-scoping.test.ts | 114 ++++++++++++++++++ .../tracked-addresses-concurrent.test.ts | 43 +++++++ .../contracts/tracked-addresses.contract.ts | 17 +++ tests/unit/storage/tracked-addresses.test.ts | 62 +++++++++- 9 files changed, 373 insertions(+), 24 deletions(-) diff --git a/core/Sphere.ts b/core/Sphere.ts index 5dae3cf6..3789088a 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -61,6 +61,7 @@ import type { } from '../types'; import { SphereError } from './errors'; import type { StorageProvider } from '../storage'; +import { isDerivableIndex } from '../storage/tracked-addresses'; import type { TransportProvider, PeerInfo } from '../transport'; import { MultiAddressTransportMux, AddressTransportAdapter } from '../transport/MultiAddressTransportMux'; import type { OracleProvider } from '../oracle'; @@ -469,6 +470,15 @@ export class Sphere { private static readonly _objectStoreKeys = new WeakMap(); private static _objectStoreSeq = 0; + /** + * How many times each backing store has been cleared. An init is invisible to `clear()` + * until it PUBLISHES (#767), so a clear cannot destroy one in flight and would wipe the + * store under it. Every init records this number before its first storage work and + * publication refuses if it moved (#772). Only cleared stores get an entry, so the + * strongly-held keys are bounded by the stores a process actually clears. + */ + private static readonly _clearGenerations = new Map(); + // One-time best-effort cleanup of the orphaned vesting cache (prior versions). private static _orphanCacheCleaned = false; @@ -831,17 +841,20 @@ export class Sphere { * its caller — until then nobody holds anything they could destroy, so a * registry left behind is unreachable and its hourly fetch runs for the life of the * process. Guarding a named subset of the steps is what failed twice: the guarded region - * and the fallible region were separate things, and drifted. + * and the fallible region were separate things, and drifted. Publication is the LAST + * guarded step for the same reason: it can refuse, and a refusal must tear down. */ private static async withOwnedRegistry( sphere: Sphere, storage: StorageProvider, network: NetworkType | undefined, + clearGeneration: number, bringUp: () => Promise, ): Promise { sphere._registry = Sphere.createOwnedRegistry(storage, network); try { await bringUp(); + Sphere.publishLive(sphere, clearGeneration); } catch (err) { // Tear down the WHOLE half-built Sphere, not just its registry. By this point // providers may be connected and the payments vertical running, and publication @@ -883,6 +896,11 @@ export class Sphere { throw new SphereError('Invalid mnemonic', 'INVALID_IDENTITY'); } + // #772: recorded BEFORE the first storage read/write and re-checked at publication — + // a clear() cannot see this init to destroy it, so it would otherwise wipe the keys + // written below out from under a Sphere that goes on to report itself ready. + const clearGeneration = Sphere.clearGenerationOf(options.storage); + // Check if wallet already exists if (await Sphere.exists(options.storage)) { throw new SphereError('Wallet already exists. Use Sphere.load() or Sphere.clear() first.', 'ALREADY_INITIALIZED'); @@ -924,7 +942,7 @@ export class Sphere { // Initialize everything progress?.({ step: 'initializing', message: 'Initializing wallet...' }); - await Sphere.withOwnedRegistry(sphere, options.storage, options.network, async () => { + await Sphere.withOwnedRegistry(sphere, options.storage, options.network, clearGeneration, async () => { await sphere.initializeProviders(); await sphere.initializeModules(); @@ -976,7 +994,6 @@ export class Sphere { progress?.({ step: 'complete', message: 'Wallet created' }); }); - Sphere.registerLive(sphere); return sphere; } @@ -993,6 +1010,9 @@ export class Sphere { Sphere.refuseRetiredModuleOptions(options); const composition = resolvePaymentsV2Composition(options.walletApi, options.network); + // #772: see create() — recorded before the first storage read, refused at publication. + const clearGeneration = Sphere.clearGenerationOf(options.storage); + // Check if wallet exists if (!(await Sphere.exists(options.storage))) { throw new SphereError('No wallet found. Use Sphere.create() to create a new wallet.', 'NOT_INITIALIZED'); @@ -1031,7 +1051,7 @@ export class Sphere { // Initialize everything progress?.({ step: 'initializing', message: 'Initializing wallet...' }); - await Sphere.withOwnedRegistry(sphere, options.storage, options.network, async () => { + await Sphere.withOwnedRegistry(sphere, options.storage, options.network, clearGeneration, async () => { await sphere.initializeProviders(); await sphere.initializeModules(); @@ -1061,7 +1081,6 @@ export class Sphere { progress?.({ step: 'complete', message: 'Wallet loaded' }); }); - Sphere.registerLive(sphere); return sphere; } @@ -1108,6 +1127,10 @@ export class Sphere { logger.debug('Sphere', 'Storage reconnected'); } + // #772: recorded AFTER import's OWN clear above, which bumps the generation. Recording + // it earlier would make every import refuse its own publication. + const clearGeneration = Sphere.clearGenerationOf(options.storage); + // Configure TokenRegistry for THIS network in the main bundle context. // import() previously omitted this (unlike init/create/load), leaving the // registry on a stale/default network — so imported wallets resolved tokens @@ -1163,7 +1186,7 @@ export class Sphere { // Initialize everything progress?.({ step: 'initializing', message: 'Initializing wallet...' }); logger.debug('Sphere', 'Initializing providers...'); - await Sphere.withOwnedRegistry(sphere, options.storage, options.network, async () => { + await Sphere.withOwnedRegistry(sphere, options.storage, options.network, clearGeneration, async () => { await sphere.initializeProviders(); await sphere.initializeModules(); logger.debug('Sphere', 'Modules initialized'); @@ -1218,7 +1241,6 @@ export class Sphere { logger.debug('Sphere', 'Import complete'); }); - Sphere.registerLive(sphere); return sphere; } @@ -1237,7 +1259,19 @@ export class Sphere { */ static async clear(options: { storage: StorageProvider }): Promise { const storage = options.storage; + // Bumped on ENTRY and again on exit, so an init that publishes anywhere in this + // window is refused too — not only one that publishes after the wipe (#772). The + // snapshot below is taken before the wipe, so a Sphere that registers between the + // two is neither destroyed here nor able to keep its data. + Sphere.bumpClearGeneration(storage); + try { + await Sphere.clearStore(storage); + } finally { + Sphere.bumpClearGeneration(storage); + } + } + private static async clearStore(storage: StorageProvider): Promise { // 1. Destroy Sphere instance — stops the payments vertical (quiescence), // then closes all connections. // Scoped on purpose: destroying whichever Sphere was constructed last silently @@ -1334,6 +1368,33 @@ export class Sphere { return key; } + /** The clears this store has seen. Absent means none — `clear()` creates the entry. */ + private static clearGenerationOf(storage: StorageProvider): number { + return Sphere._clearGenerations.get(Sphere.storeKeyOf(storage)) ?? 0; + } + + private static bumpClearGeneration(storage: StorageProvider): void { + const key = Sphere.storeKeyOf(storage); + Sphere._clearGenerations.set(key, (Sphere._clearGenerations.get(key) ?? 0) + 1); + } + + /** + * Make a fully-built Sphere reachable — or refuse, if `clear()` emptied the store + * under it while it was building. Check and registration are one SYNCHRONOUS step on + * purpose: split by an await, a clear could land between them and wipe a store whose + * Sphere is already published. The refusal throws inside `withOwnedRegistry`, whose + * teardown then destroys the half-built Sphere the caller never received. + */ + private static publishLive(sphere: Sphere, clearGeneration: number): void { + if (Sphere.clearGenerationOf(sphere._storage) !== clearGeneration) { + throw new SphereError( + 'The wallet store was cleared while this wallet was initializing, so the keys and journals this init wrote are gone. Nothing was published; re-run Sphere.init() once the clear has settled.', + 'STORAGE_ERROR', + ); + } + Sphere.registerLive(sphere); + } + /** Record a fully-built Sphere against the storage it owns. See `_liveByStorage`. */ private static registerLive(sphere: Sphere): void { const key = Sphere.storeKeyOf(sphere._storage); @@ -2255,10 +2316,6 @@ export class Sphere { throw new SphereError('HD derivation requires master key with chain code. Cannot switch addresses.', 'INVALID_CONFIG'); } - if (index < 0) { - throw new SphereError('Address index must be non-negative', 'INVALID_CONFIG'); - } - // If nametag requested, normalize and validate format early const newNametag = options?.nametag ? this.cleanNametag(options.nametag) : undefined; if (newNametag && !isValidNametag(newNametag)) { @@ -2381,6 +2438,9 @@ export class Sphere { this._transport.setFallbackSince(fallbackTs); } + // Defence in depth, and NOT individually falsifiable: the index-persist guard above + // throws first on every path that reaches here, so deleting this one changes no + // observable behaviour. It stays for the await between them (#770). this.ensureAlive(); await this._transport.setIdentity(this._identity); @@ -2572,6 +2632,15 @@ export class Sphere { return adapter; } + /** A BIP32 child number: an integer in 0…0xffffffff. Anything else aliases an address. */ + private static assertDerivableIndex(index: number): void { + if (isDerivableIndex(index)) return; + throw new SphereError( + `Address index ${String(index)} is not a BIP32 child number: it must be an integer in 0…0xffffffff.`, + 'INVALID_CONFIG', + ); + } + /** * Derive address at a specific index * @@ -2619,6 +2688,12 @@ export class Sphere { * when _initialized is still false. */ private _deriveAddressInternal(index: number, isChange: boolean = false): AddressInfo { + // The path segment is parseInt()ed downstream, so 1.5 silently derives index 1's keys, + // and a child number past 0xffffffff pads to more than 8 hex digits and derives + // off-standard. Refused at the one point every index reaches derivation through — + // deriveAddress, ensureAddressTracked, discovery and switchToAddress alike. + Sphere.assertDerivableIndex(index); + if (!this._masterKey) { throw new SphereError('HD derivation requires master key with chain code', 'INVALID_CONFIG'); } diff --git a/docs/API.md b/docs/API.md index 7f2102bd..132ed195 100644 --- a/docs/API.md +++ b/docs/API.md @@ -121,6 +121,13 @@ never touched. whenever a wallet exists on that storage or a live Sphere is registered on it, so importing over storage B tears down the Spheres on B — and only those. +**It also refuses an `init` / `create` / `load` / `import` that is in flight on that store.** A +wallet being built is not yet registered, so `clear()` cannot destroy it; instead the bring-up +checks at the very end whether the store was cleared under it and, if so, tears itself down and +rejects with a `SphereError` of code `STORAGE_ERROR` rather than handing back a ready Sphere +over an emptied KV. Retry the init once the clear has settled — it is a fresh wallet by then, +so `Sphere.init` reports `created: true`. + ### Properties | Property | Type | Description | @@ -221,6 +228,9 @@ console.log(sphere.getCurrentAddressIndex()); // 1 console.log(sphere.identity!.directAddress); // DIRECT://... (address at index 1) ``` +`index` must be a uint32 (see [`TrackedAddressEntry`](#trackedaddressentry)); anything else +throws `INVALID_CONFIG` before anything is derived, tracked or written. + #### `getActiveAddresses(): TrackedAddress[]` Get all non-hidden tracked addresses, sorted by index. @@ -699,10 +709,13 @@ interface TrackedAddressEntry { ``` `index` is a **BIP32 child number, so it must be a uint32**: an integer in `0` … `0xffffffff`. -A row whose `index` is not — fractional, negative, out of range, or not a number — is **dropped -when the registry is read**, not repaired. It is a drop rather than a repair because `1.5` would -`parseInt()` down to index 1's derivation path and alias a real address, and anything above -`0xffffffff` pads to more than 8 hex digits and derives off-standard. +It is enforced at both ends. A **write** carrying such a row — `saveTrackedAddresses`, and every +address-index API that derives keys (`switchToAddress`, `deriveAddress`, `trackScannedAddresses`, +`discoverAddresses`) — is **refused** with a typed `SphereError`; a row already **stored** is +**dropped when the registry is read**, not repaired, so one bad row cannot brick later writes. +Neither is repaired because `1.5` would `parseInt()` down to index 1's derivation path and hand +back that address's keys, and anything above `0xffffffff` pads to more than 8 hex digits and +derives off-standard. `createdAt` / `updatedAt` are repaired instead: a missing or non-finite value reads as `0`, and `hidden` reads as `true` only for an exact `true`. diff --git a/docs/INTEGRATION.md b/docs/INTEGRATION.md index dcc0bbee..c1a9d0b2 100644 --- a/docs/INTEGRATION.md +++ b/docs/INTEGRATION.md @@ -860,9 +860,17 @@ wallet removes the key itself (`Sphere.clear()`). Adding a per-entry delete woul revisiting this contract. `index` must be a **uint32** — an integer in `0` … `0xffffffff`, because it is a BIP32 child -number. Rows that are not are **dropped on read**, not repaired (see -[`TrackedAddressEntry`](./API.md#trackedaddressentry)). `loadTrackedAddresses` is otherwise -tolerant: unusable or corrupt storage must read as `[]`, never throw. +number. An `entries` row that is not one must make the **write reject** (the reference +`mergeTrackedAddresses` throws a `VALIDATION_ERROR` `SphereError`): dropping it silently would +report a save that never happened, and the row would derive another address's keys. Rows already +**stored** are **dropped on read** instead, not repaired (see +[`TrackedAddressEntry`](./API.md#trackedaddressentry)), so one bad row cannot brick every later +write. `loadTrackedAddresses` is otherwise tolerant: unusable or corrupt storage must read as +`[]`, never throw. + +If your platform runs the merge inside a transaction whose abort replaces the failure reason — +IndexedDB does — validate the argument before opening it, or callers see a generic abort instead +of the reason. ```typescript import type { StorageProvider, TrackedAddressEntry } from '@unicitylabs/sphere-sdk'; @@ -911,6 +919,10 @@ export async function saveTrackedAddressesMerging( merged.set(e.index, e); } for (const e of entries) { + // Refuse the WRITE: a dropped row here would report a save that never happened. + if (!Number.isInteger(e.index) || e.index < 0 || e.index > 0xffffffff) { + throw new Error(`tracked address index ${e.index} is not a BIP32 child number`); + } const existing = merged.get(e.index); if (!existing) { merged.set(e.index, e); diff --git a/storage/storage-provider.ts b/storage/storage-provider.ts index dd9a6d78..e2b62125 100644 --- a/storage/storage-provider.ts +++ b/storage/storage-provider.ts @@ -90,11 +90,15 @@ export interface StorageProvider extends BaseProvider { * - a failed write must not brick later writes, and must still reject to its * own caller. * - * A stored `index` must be a UINT32 — a BIP32 child number. `deriveKeyAtPath` parseInt()s + * An `index` must be a UINT32 — a BIP32 child number. `deriveKeyAtPath` parseInt()s * that path segment, so `1.5` derives index 1's keys and the row aliases a real * address. The ceiling matters too: `deriveChildKey` pads the child number to 8 hex * digits, so anything above `0xffffffff` emits extra bytes and derives off-standard. - * Such rows are dropped on read rather than repaired. + * An `entries` row that is not one must REJECT the whole call (`mergeTrackedAddresses` + * throws `VALIDATION_ERROR`); dropping it silently on a write reports a save that + * never happened. Already-stored rows are dropped on READ instead, so one bad row + * cannot brick every later write. Validate before opening the write transaction if + * your platform would otherwise replace the reason with a generic abort. * * A union is safe because there is no delete path: entries are only ever added, * and wiping the wallet removes the key itself (`Sphere.clear()`). Adding a diff --git a/storage/tracked-addresses.ts b/storage/tracked-addresses.ts index 43fd75d8..8e241a24 100644 --- a/storage/tracked-addresses.ts +++ b/storage/tracked-addresses.ts @@ -1,3 +1,4 @@ +import { SphereError } from '../core/errors'; import type { TrackedAddressEntry } from '../types'; /** On-disk shape of the global `tracked_addresses` key. */ @@ -6,8 +7,8 @@ export interface TrackedAddressesFile { addresses: TrackedAddressEntry[]; } -/** A BIP32 child number is a uint32 — see the port docstring for why. */ -function isDerivableIndex(value: unknown): value is number { +/** A BIP32 child number is a uint32 — see the port docstring. */ +export function isDerivableIndex(value: unknown): value is number { return typeof value === 'number' && Number.isInteger(value) && value >= 0 && value <= 0xffffffff; } @@ -54,15 +55,25 @@ export function parseTrackedAddresses(raw: string | null): TrackedAddressEntry[] * On a conflict the greater `updatedAt` supplies `hidden` (ties keep `incoming`) and the * earlier `createdAt` survives — safe because nothing removes a single entry; see * `StorageProvider.saveTrackedAddresses` (#766 item 5). + * An underivable `incoming` index REJECTS the write; `onDisk` is filtered instead, since + * it is read tolerantly and one bad stored row must not brick every later write. */ export function mergeTrackedAddresses( onDisk: readonly TrackedAddressEntry[], incoming: readonly TrackedAddressEntry[], ): TrackedAddressEntry[] { const merged = new Map(); - for (const entry of onDisk) merged.set(entry.index, entry); + for (const entry of onDisk) { + if (isDerivableIndex(entry.index)) merged.set(entry.index, entry); + } for (const entry of incoming) { + if (!isDerivableIndex(entry.index)) { + throw new SphereError( + `Tracked address index ${String(entry.index)} is not a BIP32 child number: it must be an integer in 0…0xffffffff. Refusing the write — such a row derives another address's keys (1.5 parses to 1) instead of its own.`, + 'VALIDATION_ERROR', + ); + } const existing = merged.get(entry.index); if (!existing) { merged.set(entry.index, entry); diff --git a/tests/integration/sphere-instance-scoping.test.ts b/tests/integration/sphere-instance-scoping.test.ts index a86609be..c17e6a7d 100644 --- a/tests/integration/sphere-instance-scoping.test.ts +++ b/tests/integration/sphere-instance-scoping.test.ts @@ -384,6 +384,120 @@ describe('Sphere lifecycle statics are scoped to the storage they are handed (#7 expect(sphereA.isReady).toBe(false); expect(sphereB.isReady, 'undeclared stores stay scoped per object').toBe(true); }); + + describe('a bring-up publishes only if its store survived it (#772)', () => { + /** A pause point: `reached` settles when the code arrives, `open()` lets it through. */ + function gate(): { arrive: () => void; reached: Promise; open: () => void; passed: Promise } { + let arrive!: () => void; + const reached = new Promise((r) => { arrive = r; }); + let open!: () => void; + const passed = new Promise((r) => { open = r; }); + return { arrive, reached, open, passed }; + } + + /** + * Park an init inside its bring-up: the mnemonic and the created marker are already + * on disk, and it has not published, so `clear()` cannot see it to destroy it. + */ + function parkBringUp(wallet: Wallet): { reached: Promise; open: () => void } { + const g = gate(); + const publish = wallet.transport.publishIdentityBinding as unknown as ReturnType; + publish.mockImplementation(async () => { + g.arrive(); + await g.passed; + return true; + }); + return { reached: g.reached, open: g.open }; + } + + /** Park `Sphere.clear()` with its live snapshot taken and the wipe not yet applied. */ + function parkWipe(wallet: Wallet): { reached: Promise; open: () => void } { + const g = gate(); + const wipe = wallet.storage.clear.bind(wallet.storage); + vi.spyOn(wallet.storage, 'clear').mockImplementation(async (prefix?: string) => { + g.arrive(); + await g.passed; + await wipe(prefix); + }); + return { reached: g.reached, open: g.open }; + } + + function initOf(wallet: Wallet, mnemonic: string): Promise { + return Sphere.init({ + storage: wallet.storage, + transport: wallet.transport, + oracle: wallet.oracle, + walletApi: wallet.world.walletApi, + network: NET, + mnemonic, + }); + } + + const CLEARED = /cleared while this wallet was initializing/; + + it('refuses to publish over a store clear() emptied while it was building', async () => { + const a = makeWallet('wiped-under-init'); + const park = parkBringUp(a); + + const init = initOf(a, MNEMONIC_A); + await park.reached; + expect(await Sphere.exists(a.storage), 'the parked init has written its keys').toBe(true); + + // Nothing is registered until publication (#767), so this clear finds no Sphere to + // destroy and wipes the KV the init is standing on. + await Sphere.clear({ storage: a.storage }); + park.open(); + + await expect(init).rejects.toThrow(CLEARED); + // What a published Sphere would have been reporting `isReady` over. + expect(await Sphere.exists(a.storage)).toBe(false); + expect(liveStoreKeys().some((k) => k.includes(a.dataDir))).toBe(false); + expect(a.transport.disconnect, 'the refused Sphere is torn down, not leaked').toHaveBeenCalled(); + }); + + it('refuses to publish DURING a clear, before the wipe that would empty it', async () => { + const a = makeWallet('publish-mid-clear'); + const park = parkBringUp(a); + const wipe = parkWipe(a); + + const init = initOf(a, MNEMONIC_A); + await park.reached; + + const cleared = Sphere.clear({ storage: a.storage }); + await wipe.reached; + + // Publishing here is past the clear's snapshot: it would be destroyed by nobody and + // wiped a moment later. A generation bumped only when clear RETURNS misses this. + park.open(); + await expect(init).rejects.toThrow(CLEARED); + + wipe.open(); + await cleared; + expect(await Sphere.exists(a.storage)).toBe(false); + }); + + it('refuses an init that began mid-clear, whose keys the wipe then erased', async () => { + const a = makeWallet('init-mid-clear'); + const wipe = parkWipe(a); + + const cleared = Sphere.clear({ storage: a.storage }); + await wipe.reached; + + // This init records the generation with the clear's entry already counted, so only a + // second bump when the clear FINISHES can tell it the wipe erased what it wrote. + const park = parkBringUp(a); + const init = initOf(a, MNEMONIC_A); + await park.reached; + expect(await Sphere.exists(a.storage)).toBe(true); + + wipe.open(); + await cleared; + park.open(); + + await expect(init).rejects.toThrow(CLEARED); + expect(await Sphere.exists(a.storage)).toBe(false); + }); + }); }); /** diff --git a/tests/integration/tracked-addresses-concurrent.test.ts b/tests/integration/tracked-addresses-concurrent.test.ts index c1aa1368..e6b426c5 100644 --- a/tests/integration/tracked-addresses-concurrent.test.ts +++ b/tests/integration/tracked-addresses-concurrent.test.ts @@ -237,6 +237,49 @@ describe('Tracked addresses — concurrent Spheres (#766 item 5)', () => { expect((await storage.loadTrackedAddresses()).map((e) => e.index)).toEqual([0, 1, 2]); }); + /** + * The uint32 rule at the LIVE end, where a bad index costs more than a dropped row: + * `deriveKeyAtPath` parseInt()s the path segment, so index 1.5 hands back index 1's + * private key. The wallet would then track, persist and spend at "1.5" — one address + * under two registry entries — and only the next reload would notice, by deleting it. + */ + it('refuses a non-uint32 index at every public entry point, before it derives or persists', async () => { + const { sphere } = await Sphere.init({ + storage, + transport: createMockTransport(), + oracle: createMockOracle(), + network: TEST_NETWORK, + walletApi: makePv2World().walletApi, + autoGenerate: true, + }); + + await sphere.switchToAddress(1); + expect(await readPersistedIndices(storage)).toEqual([0, 1]); + const activePubkey = sphere.identity!.chainPubkey; + + for (const index of [1.5, -1, 0x100000000, Number.NaN]) { + await expect(sphere.switchToAddress(index)).rejects.toThrow(/not a BIP32 child number/); + } + + // The bulk-tracking path reaches derivation through ensureAddressTracked instead, so + // it is refused at the derivation choke point rather than at switchToAddress's door. + await expect( + sphere.trackScannedAddresses([{ index: 2.5, hidden: false }]), + ).rejects.toThrow(/not a BIP32 child number/); + + // Nothing moved: not the in-memory registry, not the file, not the active identity. + // Refusing at the storage write alone is too late — ensureAddressTracked has already + // put the aliasing entry in `_trackedAddresses`, where getActiveAddresses() reports it. + expect(sphere.getAllTrackedAddresses().map((a) => a.index)).toEqual([0, 1]); + expect(await readPersistedIndices(storage)).toEqual([0, 1]); + expect(await storage.get(STORAGE_KEYS_GLOBAL.CURRENT_ADDRESS_INDEX)).toBe('1'); + expect(sphere.identity!.chainPubkey).toBe(activePubkey); + + expect(() => sphere.deriveAddress(1.5)).toThrow(/not a BIP32 child number/); + + await sphere.destroy(); + }); + describe('merge semantics (deterministic, no clock)', () => { it('unions by index, greater updatedAt wins hidden, earlier createdAt is kept', () => { const merged = mergeTrackedAddresses( diff --git a/tests/unit/storage/contracts/tracked-addresses.contract.ts b/tests/unit/storage/contracts/tracked-addresses.contract.ts index bd5d9b2a..9d80d6d3 100644 --- a/tests/unit/storage/contracts/tracked-addresses.contract.ts +++ b/tests/unit/storage/contracts/tracked-addresses.contract.ts @@ -141,6 +141,23 @@ export function describeTrackedAddressesContract( }); }); + it('refuses an underivable index instead of storing a row the next load drops', async () => { + await withProvider(async (provider) => { + await provider.saveTrackedAddresses([entry(0)]); + + // `1.5` derives index 1's keys (the path segment is parseInt()ed), so a stored row + // aliases a real address. Enforced only on read, this save reported success and the + // next load silently dropped the address the caller believes it activated. + const underivable = { ...entry(0), index: 1.5 } as TrackedAddressEntry; + await expect(provider.saveTrackedAddresses([entry(2), underivable])).rejects.toThrow(); + + // Nothing from the refused call landed — not even its well-formed companion. + expect(indices(await provider.loadTrackedAddresses())).toEqual([0]); + await provider.saveTrackedAddresses([entry(1)]); + expect(indices(await provider.loadTrackedAddresses())).toEqual([0, 1]); + }); + }); + if (options.crossObject !== true) return; it('merges across SEPARATE provider objects over the same backing store', async () => { diff --git a/tests/unit/storage/tracked-addresses.test.ts b/tests/unit/storage/tracked-addresses.test.ts index ec0bf17b..de717047 100644 --- a/tests/unit/storage/tracked-addresses.test.ts +++ b/tests/unit/storage/tracked-addresses.test.ts @@ -11,7 +11,9 @@ */ import { describe, expect, it } from 'vitest'; -import { parseTrackedAddresses } from '../../../storage/tracked-addresses'; +import { SphereError } from '../../../core/errors'; +import { mergeTrackedAddresses, parseTrackedAddresses } from '../../../storage/tracked-addresses'; +import type { TrackedAddressEntry } from '../../../types'; function stored(...addresses: unknown[]): string { return JSON.stringify({ version: 1, addresses }); @@ -146,3 +148,61 @@ describe('parseTrackedAddresses — the index must fit a BIP32 child number (uin expect(parsed.map((e) => e.index)).toEqual([0]); }); }); + +/** + * The same rule on the WRITE, where it is a refusal rather than a drop. + * + * Enforced only on read, `saveTrackedAddresses([{ index: 1.5, ... }])` STORED the row and + * reported success; the next load silently dropped it, so the address the caller believes + * it activated is simply absent — and before any of that, the live Sphere derives index 1's + * keys for it. Dropping the row here instead of throwing would keep the false success. + */ +describe('mergeTrackedAddresses — an underivable incoming index refuses the write', () => { + const ok = (index: number): TrackedAddressEntry => + ({ index, hidden: false, createdAt: 1, updatedAt: 1 }); + const bad = (index: unknown): TrackedAddressEntry => + ({ index, hidden: false, createdAt: 1, updatedAt: 1 }) as unknown as TrackedAddressEntry; + + it.each([ + ['a fractional index, which parseInt()s onto another address', 1.5], + ['a negative index, which has no BIP32 derivation at all', -1], + ['one past the uint32 ceiling, where the child number grows a 9th hex digit', 0x100000000], + ['NaN', Number.NaN], + ['Infinity', Number.POSITIVE_INFINITY], + ['a numeric string, which Number.isInteger rejects', '1'], + ['undefined', undefined], + ])('refuses %s', (_why, index) => { + expect(() => mergeTrackedAddresses([ok(0)], [bad(index)])).toThrow( + /not a BIP32 child number/, + ); + }); + + it('refuses with a typed VALIDATION_ERROR, so a caller can tell it from a disk failure', () => { + try { + mergeTrackedAddresses([], [bad(1.5)]); + expect.unreachable('the merge must not accept an underivable index'); + } catch (err) { + expect(err).toBeInstanceOf(SphereError); + expect((err as SphereError).code).toBe('VALIDATION_ERROR'); + expect((err as SphereError).message).toContain('1.5'); + } + }); + + it('refuses the WHOLE call, so no half-written registry reaches the store', () => { + // The good rows travel with the bad one; returning them would let the provider + // persist a partial snapshot and call the save a success. + expect(() => mergeTrackedAddresses([ok(0)], [ok(1), bad(2.5), ok(3)])).toThrow(SphereError); + }); + + it('accepts the whole legal range — 0, hardened, and 0xffffffff', () => { + const merged = mergeTrackedAddresses([], [ok(0), ok(0x80000000), ok(0xffffffff)]); + expect(merged.map((e) => e.index)).toEqual([0, 2147483648, 4294967295]); + }); + + it('FILTERS a bad row already on disk instead of refusing, so one cannot brick writes', () => { + // Stored rows are read tolerantly; throwing on them would make every later write of a + // legitimate address fail for as long as the bad row sits in the file. + const merged = mergeTrackedAddresses([bad(1.5), ok(0)], [ok(1)]); + expect(merged.map((e) => e.index)).toEqual([0, 1]); + }); +}); From dd6ecca62335ec30210ec8c68fa29a340859cf6b Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 02:58:51 +0200 Subject: [PATCH 28/43] test(mutation): probes for the clear generation and the derivation index guard Also repairs init-failure-strands-registry, which went stale when publication moved inside withOwnedRegistry. Swept all 109 probes: every find string now matches its file exactly once. --- tests/mutation/probes.json | 55 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 53 insertions(+), 2 deletions(-) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index f1e716f5..ef280370 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -545,8 +545,8 @@ "name": "init-failure-strands-registry", "note": "#767: a rejection inside the guarded bring-up must actually reach the catch \u2014 swallowing it would leave a half-built Sphere published and reported as success", "file": "core/Sphere.ts", - "find": " try {\n await bringUp();\n } catch (err) {", - "replace": " try {\n await bringUp().catch(() => undefined);\n } catch (err) {", + "find": " try {\n await bringUp();\n Sphere.publishLive(sphere, clearGeneration);\n } catch (err) {", + "replace": " try {\n await bringUp().catch(() => undefined);\n Sphere.publishLive(sphere, clearGeneration);\n } catch (err) {", "tests": [ "tests/integration/sphere-payments-v2-wiring.test.ts" ] @@ -1041,5 +1041,56 @@ "tests": [ "tests/unit/token-engine/worker-verification.test.ts" ] + }, + { + "name": "publish-refuses-cleared-store", + "note": "#772 review: an init is invisible to clear() until it publishes, so without this compare a create() whose KV was emptied mid-init publishes an isReady Sphere over nothing", + "file": "core/Sphere.ts", + "find": " if (Sphere.clearGenerationOf(sphere._storage) !== clearGeneration) {", + "replace": " if (false) {", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "clear-bumps-generation-on-entry", + "note": "#772 review: bumping only on exit misses an init that publishes DURING the clear \u2014 past its snapshot, before its wipe", + "file": "core/Sphere.ts", + "find": " Sphere.bumpClearGeneration(storage);\n try {\n await Sphere.clearStore(storage);", + "replace": " // probe\n try {\n await Sphere.clearStore(storage);", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "clear-bumps-generation-on-exit", + "note": "#772 review: bumping only on entry misses an init that STARTS mid-clear, records the already-bumped value and is wiped a moment later", + "file": "core/Sphere.ts", + "find": " } finally {\n Sphere.bumpClearGeneration(storage);\n }", + "replace": " } finally {\n void storage;\n }", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "tracked-index-refused-at-write", + "note": "#772 review: the uint32 rule must hold before the WRITE, not only after reload \u2014 a persisted 1.5 derives index 1's keys and aliases a real address", + "file": "storage/tracked-addresses.ts", + "find": " if (!isDerivableIndex(entry.index)) {\n throw new SphereError(", + "replace": " if (false) {\n throw new SphereError(", + "tests": [ + "tests/unit/storage/tracked-addresses.test.ts", + "tests/unit/storage/contracts/tracked-addresses.contract.ts" + ] + }, + { + "name": "derivation-refuses-underivable-index", + "note": "#772 review: the storage-layer refusal is too late \u2014 ensureAddressTracked mutates the in-memory registry before persisting, so derivation itself must refuse", + "file": "core/Sphere.ts", + "find": " Sphere.assertDerivableIndex(index);\n\n if (!this._masterKey) {", + "replace": " // probe\n\n if (!this._masterKey) {", + "tests": [ + "tests/integration/tracked-addresses-concurrent.test.ts" + ] } ] From 6e2a88549470c416d4ce52d22d66b7d8cb02cba1 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 03:01:34 +0200 Subject: [PATCH 29/43] docs(changelog): the four behaviour changes from the review round --- CHANGELOG.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 30109bb2..bc980714 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,33 @@ Four defects of one shape: work spawned by a component outlived the component. both. A refused switch also no longer persists the address index it never finished moving to, which would have sent the next boot to the wrong address. +### Changed — three refusals that used to be silent successes + +Found by review of the fixes above; each turns a wrong answer into a typed error. + +- **`saveTrackedAddresses` rejects a non-uint32 index** instead of writing the row and + letting the next load drop it. The uint32 rule was enforced only on read, so a malformed + index reached derivation — `deriveKeyAtPath` `parseInt()`s the path segment, so `1.5` + derives index `1`'s keys — before anything noticed. Rows already on disk are still + *filtered* rather than rejected: one corrupt stored row must not brick every later write. +- **`deriveAddress`, `switchToAddress`, `trackScannedAddresses` and address discovery throw + `INVALID_CONFIG` for any non-uint32 index.** Previously only `switchToAddress` checked, and + only for negatives. The guard sits at the one point every index reaches derivation + through — the storage-layer refusal alone is too late, because `ensureAddressTracked` + mutates the in-memory registry before persisting. +- **`Sphere.init` / `create` / `load` / `import` can reject with `STORAGE_ERROR`** when + `Sphere.clear()` empties the store while they are building. An init is invisible to + `clear()` until it publishes, so previously the KV was wiped and the init went on to + publish a Sphere reporting `isReady` over nothing. Refusing is deliberate over serializing: + `import()` clears internally (a per-store mutex would round-trip through itself), and a + `clear()` blocked behind a slow bring-up is a worse outage than a loud refusal. + +A fourth, in the transport: **a failed `setIdentity()` no longer half-applies.** Identity, +key material and the per-address dedup window are staged and committed with the client, so +a swap that fails leaves the provider entirely on the old identity — previously the old +client kept serving its old-address subscriptions under the *new* key. A caller retrying +after a transient relay failure therefore re-attempts the whole swap. + ### Fixed — `checkNetworkHealth` reported healthy gateways as unhealthy (#769) It POSTed `get_round_number` with empty params and keyed the verdict off `response.ok`. The From cd45901cffd372bd8f42da37688ff0ec3f999d12 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 03:28:24 +0200 Subject: [PATCH 30/43] fix: two more teardown/deadline holes (local codex review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. destroy() landing INSIDE initializeAddressModules. The switchToAddress guards are checks BEFORE an await, so a teardown that starts while the bring-up is itself awaiting has its module-clearing loop run first, and the continuation then registers a fully-built set — its own token engine, hence its own worker pool — on a Sphere whose destroy() already returned. Nothing disposes it and nothing holds a reference to find it by. The set is now discarded and its engine disposed, along with any mux the bring-up rebuilt, before the switch re-raises. destroy()'s teardown loop was extracted so both paths tear a module set down the same way (that extraction also retires one max-lines-per-function suppression). 2. checkNetworkHealth dropped its deadline before reading the body. fetch settles on the response HEADERS, so clearing the timer there left response.json() waiting forever on a gateway that stalls mid-response — timeoutMs stopped applying at exactly the point the endpoint is least responsive. The timer now clears in a finally, and an abort during the body read is reported as the timeout it is rather than as a malformed answer. Falsified: restoring the early clear hangs the new test to its 2s limit. NOT taken from the same review: sharing FileStorageProvider's tracked-address write chain across provider objects. The premise — that two objects over one dataDir became a supported scenario because they now share a backingStoreId — is wrong: backingStoreId scopes TEARDOWN (which Sphere clear() destroys), and the concurrent-write half is explicitly still open as #771. I implemented the change, then reverted it: that provider caches the whole store and rewrites the file on every set(), so a sibling's UNRELATED write still rolls the registry back. Serializing the tracked path alone would have made the skipped contract case pass while leaving the provider unsafe — which is what the existing carve-out comment in tracked-addresses-providers.test.ts already says. --- core/Sphere.ts | 51 ++++++++++++++---- core/network-health.ts | 52 +++++++++++++------ eslint-suppressions.json | 2 +- .../sphere-payments-v2-wiring.test.ts | 51 ++++++++++++++++++ tests/mutation/probes.json | 20 +++++++ tests/unit/core/network-health.test.ts | 25 +++++++++ 6 files changed, 173 insertions(+), 28 deletions(-) diff --git a/core/Sphere.ts b/core/Sphere.ts index 3789088a..0a59f187 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -2393,6 +2393,10 @@ export class Sphere { // leaves behind — so an unguarded switch opens new sockets after teardown returned. this.ensureAlive(); await this.initializeAddressModules({ index, identity: newIdentity }); + // The guard above is a check BEFORE an await: destroy() landing inside this + // bring-up would otherwise let the continuation register a live module set — + // and a reconnected mux — on a Sphere whose teardown had already returned. + if (this._destroyed) await this.discardModulesBuiltDuringDestroy(index); } else if (nametag !== this._addressModules.get(index)!.identity.nametag) { // Modules already exist — only the nametag label changed. this._addressModules.get(index)!.identity = newIdentity; @@ -2503,6 +2507,42 @@ export class Sphere { * independently in background. The payments vertical is NOT created here — * the caller starts it via stopThenStartPaymentsV2 (§7 single vertical). */ + /** Tear one address's module set down. Each address owns its own engine, so its own pool. */ + private static destroyModuleSet(index: number, moduleSet: AddressModuleSet): void { + try { + moduleSet.communications.destroy(); + moduleSet.groupChat?.destroy(); + moduleSet.market?.destroy(); + moduleSet.tokenEngine?.dispose?.(); + logger.debug('Sphere', `Destroyed modules for address ${index}`); + } catch (err) { + logger.warn('Sphere', `Error destroying modules for address ${index}:`, err); + } + } + + /** + * Undo a module set built while `destroy()` was running. + * + * `destroy()`'s teardown loop has already emptied `_addressModules`, so a set + * registered after it would stay live and unreachable — its own engine (and worker + * pool), and a transport mux `ensureTransportMux` rebuilt because teardown had just + * nulled the field. Then re-raise, so the switch fails rather than reporting success + * on a destroyed Sphere (#770, #772 review). + */ + private async discardModulesBuiltDuringDestroy(index: number): Promise { + const built = this._addressModules.get(index); + this._addressModules.delete(index); + if (built) Sphere.destroyModuleSet(index, built); + if (this._transportMux) { + const mux = this._transportMux; + this._transportMux = null; + await Sphere.safeDisconnect('transport mux', () => mux.disconnect()); + } + this.ensureAlive(); + /* c8 ignore next */ + throw new SphereError('Sphere destroyed', 'NOT_INITIALIZED'); + } + private async initializeAddressModules( spec: { index: number; identity: FullIdentity }, ): Promise { @@ -3671,16 +3711,7 @@ export class Sphere { // Destroy all per-address module sets for (const [idx, moduleSet] of this._addressModules.entries()) { - try { - moduleSet.communications.destroy(); - moduleSet.groupChat?.destroy(); - moduleSet.market?.destroy(); - // Each address has its OWN engine, so each may own its own worker pool. - moduleSet.tokenEngine?.dispose?.(); - logger.debug('Sphere', `Destroyed modules for address ${idx}`); - } catch (err) { - logger.warn('Sphere', `Error destroying modules for address ${idx}:`, err); - } + Sphere.destroyModuleSet(idx, moduleSet); } this._addressModules.clear(); diff --git a/core/network-health.ts b/core/network-health.ts index a4a498f6..21b5f72f 100644 --- a/core/network-health.ts +++ b/core/network-health.ts @@ -283,15 +283,41 @@ function readRpcError(body: unknown): string | null { return null; } +/** The verdict for a probe that came back — healthy only on a real block height. */ +function oracleVerdict( + url: string, + responseTimeMs: number, + response: Response, + body: unknown, +): ServiceHealthResult { + if (readBlockNumber(body) !== null) return { healthy: true, url, responseTimeMs }; + const rpcError = readRpcError(body); + return { + healthy: false, + url, + responseTimeMs, + error: + rpcError ?? + (response.ok + ? 'aggregator answered without a block height' + : `HTTP ${String(response.status)} ${response.statusText}`), + }; +} + /** * Check oracle (aggregator) endpoint via HTTP POST. */ async function checkOracle(url: string, timeoutMs: number): Promise { const startTime = Date.now(); + const controller = new AbortController(); + // Deliberately NOT cleared when the fetch resolves. `fetch` settles on the response + // HEADERS, so a gateway that stalls mid-body would leave the read below waiting + // forever on a deadline that had already been cancelled — timeoutMs would silently + // stop applying at the one point the endpoint is least responsive. Cleared in the + // `finally` instead, once the body has been read or the abort has cut it short. + const timer = setTimeout(() => controller.abort(), timeoutMs); try { - const controller = new AbortController(); - const timer = setTimeout(() => controller.abort(), timeoutMs); const response = await fetch(url, { method: 'POST', @@ -305,25 +331,15 @@ async function checkOracle(url: string, timeoutMs: number): Promise null); - if (readBlockNumber(body) !== null) { - return { healthy: true, url, responseTimeMs }; + // The deadline can only have fired during the body read — the fetch itself would + // have rejected. Reported as the timeout it is, not as a malformed answer. + if (controller.signal.aborted) { + return { healthy: false, url, responseTimeMs, error: `Connection timeout after ${timeoutMs}ms` }; } - - const rpcError = readRpcError(body); - return { - healthy: false, - url, - responseTimeMs, - error: - rpcError ?? - (response.ok - ? 'aggregator answered without a block height' - : `HTTP ${String(response.status)} ${response.statusText}`), - }; + return oracleVerdict(url, responseTimeMs, response, body); } catch (err) { return { healthy: false, @@ -333,6 +349,8 @@ async function checkOracle(url: string, timeoutMs: number): Promise { + // The guards in switchToAddress are checks BEFORE an await. This is the gap they + // cannot close on their own: destroy() lands while initializeAddressModules is + // itself awaiting, its teardown loop empties _addressModules, and the continuation + // then registers a fully-built set — its own token engine, and so its own worker + // pool — on a Sphere whose destroy() has already returned. Nothing would ever + // dispose it, and nothing holds a reference through which it could be found. + const world = makeWorld(); + const sphere = await buildSphere({ walletApi: world.walletApi }); + const modules = (sphere as unknown as { _addressModules: Map })._addressModules; + + // Park INSIDE the bring-up, on the engine build — past ensureTransportMux, before + // the module set is registered. + let release!: () => void; + const opened = new Promise((resolve) => (release = resolve)); + const internals = sphere as unknown as { + buildTokenEngine: (identity: unknown) => Promise; + }; + const realBuild = internals.buildTokenEngine.bind(sphere); + const built: ITokenEngine[] = []; + internals.buildTokenEngine = async (identity: unknown): Promise => { + await opened; + const engine = await realBuild(identity); + engine.dispose = vi.fn(); + built.push(engine); + return engine; + }; + + const switching = sphere.switchToAddress(1).then( + () => 'resolved' as const, + (err: unknown) => err + ); + await sleep(200); + + await sphere.destroy(); + expect(modules.size).toBe(0); + + release(); + const outcome = await switching; + await sleep(200); + + // The set the continuation built is gone again, and its engine — the one destroy() + // could not have disposed, because it did not exist yet — was disposed here. + expect(modules.size).toBe(0); + expect(built).toHaveLength(1); + expect(built[0]!.dispose).toHaveBeenCalledTimes(1); + expect((sphere as unknown as { _transportMux: unknown })._transportMux).toBeNull(); + expect(outcome).toBeInstanceOf(SphereError); + expect((outcome as SphereError).code).toBe('NOT_INITIALIZED'); + }, 30_000); + it('setOracleApiKey rebuilds the engine and swaps it via facade.setEngine; the replaced engine is disposed', async () => { const world = makeWorld(); const sphere = await buildSphere({ walletApi: world.walletApi }); diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index ef280370..ad7f3f83 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -1092,5 +1092,25 @@ "tests": [ "tests/integration/tracked-addresses-concurrent.test.ts" ] + }, + { + "name": "switch-keeps-modules-built-during-destroy", + "note": "#772 codex: the switchToAddress guards are checks BEFORE an await. destroy() landing INSIDE initializeAddressModules lets the continuation register a live module set \u2014 its own engine and worker pool \u2014 on a Sphere whose destroy() already returned", + "file": "core/Sphere.ts", + "find": " if (this._destroyed) await this.discardModulesBuiltDuringDestroy(index);", + "replace": " // probe", + "tests": [ + "tests/integration/sphere-payments-v2-wiring.test.ts" + ] + }, + { + "name": "health-deadline-dropped-before-body-read", + "note": "#769.1 codex: fetch settles on the HEADERS, so clearing the deadline there leaves the body read waiting forever on a gateway that stalls mid-response \u2014 timeoutMs stops applying exactly where the endpoint is least responsive", + "file": "core/network-health.ts", + "find": " const responseTimeMs = Date.now() - startTime;\n\n const body: unknown = await response.json().catch(() => null);", + "replace": " clearTimeout(timer);\n const responseTimeMs = Date.now() - startTime;\n\n const body: unknown = await response.json().catch(() => null);", + "tests": [ + "tests/unit/core/network-health.test.ts" + ] } ] diff --git a/tests/unit/core/network-health.test.ts b/tests/unit/core/network-health.test.ts index b39f9dee..6efe487e 100644 --- a/tests/unit/core/network-health.test.ts +++ b/tests/unit/core/network-health.test.ts @@ -123,6 +123,31 @@ describe('checkNetworkHealth', () => { expect(result.healthy).toBe(false); }); + it('honours timeoutMs while the BODY is still streaming, not only the headers', async () => { + // fetch settles on the response HEADERS. Clearing the deadline there left the + // body read waiting forever on a gateway that stalled mid-response — timeoutMs + // silently stopped applying at the point the endpoint is least responsive. + fetchSpy.mockImplementationOnce((_url: string, init: RequestInit) => { + const signal = init.signal!; + return Promise.resolve({ + ok: true, + status: 200, + statusText: 'OK', + json: () => + new Promise((_resolve, reject) => { + signal.addEventListener('abort', () => { + reject(Object.assign(new Error('aborted'), { name: 'AbortError' })); + }); + }), + } as unknown as Response); + }); + + const result = await checkNetworkHealth('testnet', { services: ['oracle'], timeoutMs: 50 }); + + expect(result.services.oracle!.healthy).toBe(false); + expect(result.services.oracle!.error).toContain('timeout'); + }, 2000); + it('should report oracle unhealthy on fetch error', async () => { fetchSpy.mockRejectedValueOnce(new Error('ECONNREFUSED')); From a420baeb2af6900222c57d41cbe762c692e4b675 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 03:31:58 +0200 Subject: [PATCH 31/43] test(token-engine): pin the reentrant-dispose outcome; guard it defensively MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's finding is NOT reachable in the pinned SDK, and I checked rather than assumed: WorkerTokenVerifier.verify awaits CertifiedMintTransactionVerificationRule on the genesis BEFORE fanning transfers out, so super.verify cannot reach a worker's postMessage — and thus a reentrant dispose() — before verify() has registered this call's cancellation. A test that drives dispose() from inside postMessage passes with the proposed guard removed. The guard is kept anyway, two lines, and labelled as what it is: unreachable today, insurance against an internal reordering in a pinned dependency that has already changed shape across a major. The test is kept too, with a comment saying plainly that it pins the OUTCOME (a reentrant teardown rejects, never hangs) and is not a falsification of the guard — so the next reader does not mistake it for one. --- .../token-engine/worker-verification.test.ts | 25 +++++++++++++++++++ token-engine/factory.ts | 11 ++++++-- 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/tests/unit/token-engine/worker-verification.test.ts b/tests/unit/token-engine/worker-verification.test.ts index 61be97be..9130786a 100644 --- a/tests/unit/token-engine/worker-verification.test.ts +++ b/tests/unit/token-engine/worker-verification.test.ts @@ -322,6 +322,31 @@ describe('dispose() during an in-flight verification (#770 item 4)', () => { expect(createWorker).toHaveBeenCalledTimes(spawnsBeforeSecondVerify); }, 30000); + it('rejects when dispose() runs REENTRANTLY from inside super.verify()', async () => { + // A worker's message handler runs on THIS thread, so dispose() can fire from inside + // postMessage. Today the cancellation is already registered when that happens — the + // SDK awaits the genesis rule before fanning transfers out — so this passes with the + // post-registration re-check removed, and it is NOT a falsification of that guard. + // It pins the OUTCOME rather than the mechanism: however the pinned SDK orders its + // internals, a reentrant teardown must reject, never hang on a terminated worker. + const spawned: SilentWorker[] = []; + let engineRef: { dispose?: () => void } | null = null; + const createWorker = vi.fn(() => { + const worker = new SilentWorker(); + worker.onPost = (): void => engineRef?.dispose?.(); + spawned.push(worker); + return worker; + }); + + const engine = await buildEngine(createWorker, 1); + engineRef = engine; + + const outcome = await settleWithin(engine.verify(oneTransfer), 3000); + expect(outcome.state).toBe('rejected'); + expect(outcome).toMatchObject({ reason: { code: 'MODULE_DESTROYED' } }); + expect(spawned.every((w) => w.terminated)).toBe(true); + }, 30000); + it('rejects a post-dispose verify even for a token that needs no worker at all', async () => { const createWorker = vi.fn(() => new SilentWorker()); const engine = await buildEngine(createWorker); diff --git a/token-engine/factory.ts b/token-engine/factory.ts index 1dc1d974..9a2a083d 100644 --- a/token-engine/factory.ts +++ b/token-engine/factory.ts @@ -105,12 +105,19 @@ class ConfiguredWorkerTokenVerifier extends WorkerTokenVerifier { ...args: Parameters ): ReturnType { if (this.disposed) return Promise.reject(disposedError()); - // Started before its cancellation is registered, but nothing can interleave: - // an async function runs synchronously up to its first await. const verdict = super.verify(...args); return new Promise((resolve, reject) => { const cancel = (): void => reject(disposedError()); this.cancellations.add(cancel); + // Defensive, and currently UNREACHABLE — the SDK awaits the genesis rule before + // fanning transfers out, so a reentrant dispose() (a worker's postMessage handler + // runs on THIS thread) cannot land before this line. Kept because that ordering is + // an internal detail of a pinned dependency: fan out first and dispose() would + // clear the set BEFORE registration, hanging this promise on a dead worker. + if (this.disposed) { + this.cancellations.delete(cancel); + cancel(); + } // Settling twice is a no-op, so a verdict landing after dispose() cancelled // the call can never downgrade that rejection into a falsy verdict. void verdict.then( From e46b7ae9641573ab7f69786afef9458a004cc97e Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 03:54:17 +0200 Subject: [PATCH 32/43] fix: process-wide identity state must not split per tsup bundle (Copilot round 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tsup.shared.js defines 12 configs, each splitting:false, each emitting esm+cjs. So every subpath export carries its own copy of module state — which made this PR's own fix incomplete: a Sphere built through 'sphere-sdk' was invisible to a Sphere.clear() called through 'sphere-sdk/core', so that clear wiped the backing store and left the first Sphere isReady over an emptied KV. #766 again, across the entry-point boundary. _clearGenerations split the same way, so an in-flight init in the other bundle also missed the clear. LocalStorageProvider minted its backingStoreId tags from a module-level WeakMap plus counter, and that string is process-level identity: two copies each give their first Storage object tag 1, so two unrelated wallets collide and clear() destroys the wrong one. Its trackedWriteChains map — the read-merge-write serializer — was module-local too, which is the #766 lost update one level out. All of it now rides one versioned globalThis cell, the pattern core/logger.ts already uses for the same reason. Two version levels on purpose: the container key is versioned, and each cell NAME carries its own shape version, so changing one structure later cannot silently split the other and reintroduce the tag collision between two SDK copies on one page. _objectStoreKeys/_objectStoreSeq had to move too, and not sharing them would have been worse than the bug: with the live map shared but the counters split, two bundles hand different providers the same 'object:1' key and land two unrelated wallets in ONE bucket, so a clear() on either destroys both. IndexedDBStorageProvider and FileStorageProvider were checked and need nothing — their ids are intrinsic (dbName+prefix, resolved path), not counter-allocated. The cell is non-enumerable, validated on read (a planted or foreign value is refused, not adopted), and degrades to bundle-local rather than throwing when globalThis or the bag is frozen. It holds Sphere instances, backingStoreId strings and counters — never key material, which is stated in the helper doc and pinned by a test. In-realm secrecy was never available anyway: a script that can read the cell can already reach the same objects through any consumer variable. Cross-bundle tests use vi.resetModules() plus two dynamic imports, verified to produce genuinely distinct module objects before asserting anything; each states that it does NOT reproduce a second realm, which no in-process mechanism can join. --- core/Sphere.ts | 40 +++- core/global-cell.ts | 70 ++++++ impl/browser/storage/LocalStorageProvider.ts | 29 ++- .../sphere-cross-bundle-lifecycle.test.ts | 222 ++++++++++++++++++ tests/unit/core/global-cell.test.ts | 175 ++++++++++++++ tests/unit/impl/backing-store-id.test.ts | 49 +++- .../tracked-addresses-providers.test.ts | 46 +++- 7 files changed, 617 insertions(+), 14 deletions(-) create mode 100644 core/global-cell.ts create mode 100644 tests/integration/sphere-cross-bundle-lifecycle.test.ts create mode 100644 tests/unit/core/global-cell.test.ts diff --git a/core/Sphere.ts b/core/Sphere.ts index 0a59f187..db8fe576 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -42,6 +42,7 @@ */ import { logger } from './logger'; +import { sharedCell } from './global-cell'; import type { Identity, FullIdentity, @@ -454,6 +455,35 @@ export interface AddressModuleSet { initialized: boolean; } +/** + * The lifecycle state that must be PROCESS-wide, not per-bundle: every subpath export + * is its own tsup bundle, so a Sphere built through `sphere-sdk` was invisible to a + * `clear()` called through `sphere-sdk/core` — #766 again, across the entry points. + * The object keys travel with it: split, two bundles hand DIFFERENT providers the same + * `object:1` and one wallet's clear() destroys another's Sphere. + */ +interface SphereLifecycleCell { + readonly live: Map>; + readonly clearGenerations: Map; + readonly objectStoreKeys: WeakMap; + objectStoreSeq: number; +} + +const lifecycle = sharedCell( + 'core.sphere.lifecycle@1', + () => ({ + live: new Map(), + clearGenerations: new Map(), + objectStoreKeys: new WeakMap(), + objectStoreSeq: 0, + }), + (cell) => { + const c = cell as Partial; + return c.live instanceof Map && c.clearGenerations instanceof Map + && c.objectStoreKeys instanceof WeakMap && typeof c.objectStoreSeq === 'number'; + }, +); + // ============================================================================= // Sphere Class // ============================================================================= @@ -464,11 +494,11 @@ export class Sphere { // really erase. Object identity was too narrow — two providers over one dataDir/DB // are different objects but the same file, so clearing through one left the other's // Sphere live over an emptied KV (#766). - private static readonly _liveByStorage = new Map>(); + private static readonly _liveByStorage: Map> = lifecycle.live; /** Fallback keys for providers that declare no `backingStoreId` — one per object. */ - private static readonly _objectStoreKeys = new WeakMap(); - private static _objectStoreSeq = 0; + private static readonly _objectStoreKeys: WeakMap = + lifecycle.objectStoreKeys; /** * How many times each backing store has been cleared. An init is invisible to `clear()` @@ -477,7 +507,7 @@ export class Sphere { * publication refuses if it moved (#772). Only cleared stores get an entry, so the * strongly-held keys are bounded by the stores a process actually clears. */ - private static readonly _clearGenerations = new Map(); + private static readonly _clearGenerations: Map = lifecycle.clearGenerations; // One-time best-effort cleanup of the orphaned vesting cache (prior versions). private static _orphanCacheCleaned = false; @@ -1362,7 +1392,7 @@ export class Sphere { if (declared) return `store:${declared}`; let key = Sphere._objectStoreKeys.get(storage); if (!key) { - key = `object:${++Sphere._objectStoreSeq}`; + key = `object:${++lifecycle.objectStoreSeq}`; Sphere._objectStoreKeys.set(storage, key); } return key; diff --git a/core/global-cell.ts b/core/global-cell.ts new file mode 100644 index 00000000..6696b0ba --- /dev/null +++ b/core/global-cell.ts @@ -0,0 +1,70 @@ +/** + * Identity state that must not split when tsup bundles each subpath export separately + * (`splitting: false`) or when ESM and CJS both load: globalThis, VERSIONED key, + * validated on read, never key material. Why: tests/unit/core/global-cell.test.ts. + */ +const CELLS_KEY = '__sphere_sdk_cells_v1__'; + +type CellBag = Record; + +/** Set only when globalThis refuses the bag; cells then degrade to this bundle. */ +let bundleLocalBag: CellBag | null = null; + +function isObject(value: unknown): value is object { + return typeof value === 'object' && value !== null; +} + +function installBag(host: CellBag, fresh: CellBag): boolean { + try { + Object.defineProperty(host, CELLS_KEY, { + value: fresh, + writable: true, + configurable: true, + enumerable: false, + }); + return host[CELLS_KEY] === fresh; + } catch { + return false; + } +} + +function cellBag(): CellBag { + if (bundleLocalBag) return bundleLocalBag; + const host = globalThis as unknown as CellBag; + const found = host[CELLS_KEY]; + if (isObject(found)) return found as CellBag; + const fresh: CellBag = {}; + if (!installBag(host, fresh)) bundleLocalBag = fresh; + return fresh; +} + +function accepted(intact: (candidate: object) => boolean, candidate: object): boolean { + try { + return intact(candidate); + } catch { + return false; + } +} + +function put(bag: CellBag, name: string, cell: object): void { + try { + bag[name] = cell; + if (bag[name] === cell) return; + } catch { /* a frozen or trapped bag — this bundle keeps its cells locally */ } + if (bag !== bundleLocalBag) bundleLocalBag = { [name]: cell }; +} + +/** The cell `name` (suffix its shape version, `@1`), created once and shared by every + * bundle; `intact` rejects a foreign value rather than adopting it as SDK state. */ +export function sharedCell( + name: string, + create: () => T, + intact: (candidate: object) => boolean, +): T { + const bag = cellBag(); + const found = bag[name]; + if (isObject(found) && accepted(intact, found)) return found as T; + const fresh = create(); + put(bag, name, fresh); + return fresh; +} diff --git a/impl/browser/storage/LocalStorageProvider.ts b/impl/browser/storage/LocalStorageProvider.ts index 79886013..2ff9c61d 100644 --- a/impl/browser/storage/LocalStorageProvider.ts +++ b/impl/browser/storage/LocalStorageProvider.ts @@ -4,6 +4,7 @@ */ import { logger } from '../../../core/logger'; +import { sharedCell } from '../../../core/global-cell'; import { SphereError } from '../../../core/errors'; import type { ProviderStatus, FullIdentity, TrackedAddressEntry } from '../../../types'; import type { StorageProvider } from '../../../storage'; @@ -40,16 +41,30 @@ export interface LocalStorageProviderConfig { * Per-`Storage` tags for `backingStoreId`. The prefix alone does not identify the * store: an SSR fallback mints a private in-memory `Storage` per provider, so two * providers with the same prefix over different objects hold unrelated data. Weak, - * lazily assigned, and process-local — it is only ever compared with itself. + * lazily assigned, and PROCESS-wide — a per-bundle counter gives the first unrelated + * `Storage` of each copy the tag `1`, and `Sphere.clear()` erases the wrong wallet. + * The write chains ride the same cell: two chains over one store lose an update. */ -const storageObjectTags = new WeakMap(); -let storageObjectSeq = 0; +interface StorageIdentityCell { + readonly tags: WeakMap; + readonly writeChains: Map>; + seq: number; +} + +const storeIdentity = sharedCell( + 'impl.browser.localStorage.identity@1', + () => ({ tags: new WeakMap(), writeChains: new Map(), seq: 0 }), + (cell) => { + const c = cell as Partial; + return c.tags instanceof WeakMap && c.writeChains instanceof Map && typeof c.seq === 'number'; + }, +); function storageObjectTag(storage: Storage): string { - let tag = storageObjectTags.get(storage); + let tag = storeIdentity.tags.get(storage); if (!tag) { - tag = String(++storageObjectSeq); - storageObjectTags.set(storage, tag); + tag = String(++storeIdentity.seq); + storeIdentity.tags.set(storage, tag); } return tag; } @@ -61,7 +76,7 @@ function storageObjectTag(storage: Storage): string { * level up. This coordinates a single JS realm only; a SECOND TAB writing the same store * is genuinely not covered, and no in-process lock can cover it. */ -const trackedWriteChains = new Map>(); +const trackedWriteChains = storeIdentity.writeChains; function serializeTrackedWrite(storeId: string, task: () => Promise): Promise { const previous = trackedWriteChains.get(storeId) ?? Promise.resolve(); diff --git a/tests/integration/sphere-cross-bundle-lifecycle.test.ts b/tests/integration/sphere-cross-bundle-lifecycle.test.ts new file mode 100644 index 00000000..9e68724d --- /dev/null +++ b/tests/integration/sphere-cross-bundle-lifecycle.test.ts @@ -0,0 +1,222 @@ +/** + * #766, one boundary further out: the lifecycle registry must be PROCESS-wide. + * + * `Sphere._liveByStorage` and `Sphere._clearGenerations` decide who `clear()` may + * destroy and which init is standing on a store that was emptied under it. tsup builds + * every subpath export as its own bundle with `splitting: false` (tsup.shared.js), so + * a static on the class is per-BUNDLE: a Sphere created through `@unicitylabs/sphere-sdk` + * was invisible to a `clear()` called through `@unicitylabs/sphere-sdk/core`, which then + * wiped the KV and left that Sphere `isReady` over nothing — the exact bug the scoped + * registry fixed, resurrected across the entry points. The ESM and CJS outputs duplicate + * the same way. + * + * `vi.resetModules()` + two dynamic imports gives a genuinely separate module instance + * (asserted below) sharing one globalThis, which is exactly the two-bundle shape. It is + * NOT a second REALM: an iframe or a worker has its own globalThis and cannot be joined + * by any in-process mechanism, and nothing here claims to cover that. + * + * The object-key half is load-bearing for the same reason and cannot be split off: with + * a shared registry and a per-copy counter, the first `backingStoreId`-less provider of + * EACH copy is `object:1`, so two unrelated wallets land in one bucket and one wallet's + * clear() destroys the other's Sphere. The last test is that case. + */ + +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { FileStorageProvider } from '../../impl/nodejs/storage/FileStorageProvider'; +import type { StorageProvider } from '../../storage'; +import type { TransportProvider } from '../../transport'; +import type { FullIdentity, ProviderStatus, TrackedAddressEntry } from '../../types'; +import { makePv2World, createEngineOracle } from '../support/pv2-world'; + +type SphereModule = typeof import('../../core/Sphere'); +type SphereInstance = Awaited>['sphere']; + +const NET = 'testnet2' as const; +const MNEMONIC_A = 'test test test test test test test test test test test junk'; +const MNEMONIC_B = + 'abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about'; + +function createMockTransport(): TransportProvider { + return { + id: 'mock-transport', + name: 'Mock Transport', + type: 'p2p' as const, + description: 'Mock transport', + setIdentity: vi.fn(), + connect: vi.fn().mockResolvedValue(undefined), + disconnect: vi.fn().mockResolvedValue(undefined), + isConnected: vi.fn().mockReturnValue(true), + getStatus: vi.fn().mockReturnValue('connected' as ProviderStatus), + sendMessage: vi.fn().mockResolvedValue('event-id'), + onMessage: vi.fn().mockReturnValue(() => {}), + subscribeToBroadcast: vi.fn().mockReturnValue(() => {}), + publishBroadcast: vi.fn().mockResolvedValue('broadcast-id'), + onEvent: vi.fn().mockReturnValue(() => {}), + resolve: vi.fn().mockResolvedValue(null), + resolveNametag: vi.fn().mockResolvedValue(null), + publishIdentityBinding: vi.fn().mockResolvedValue(true), + recoverNametag: vi.fn().mockResolvedValue(null), + } as unknown as TransportProvider; +} + +/** A provider that declares no `backingStoreId`, so liveness falls back to object keys. */ +class MemoryStorage implements StorageProvider { + readonly id = 'memory'; + readonly name = 'Memory Storage'; + readonly type = 'local' as const; + private connected = false; + + private identity: FullIdentity | null = null; + + constructor(private readonly cells: Map) {} + + setIdentity(identity: FullIdentity): void { this.identity = identity; } + getIdentity(): FullIdentity | null { return this.identity; } + async connect(): Promise { this.connected = true; } + async disconnect(): Promise { this.connected = false; } + isConnected(): boolean { return this.connected; } + getStatus(): ProviderStatus { return this.connected ? 'connected' : 'disconnected'; } + async get(key: string): Promise { return this.cells.get(key) ?? null; } + async set(key: string, value: string): Promise { this.cells.set(key, value); } + async remove(key: string): Promise { this.cells.delete(key); } + async has(key: string): Promise { return this.cells.has(key); } + async keys(): Promise { return Array.from(this.cells.keys()); } + async clear(): Promise { this.cells.clear(); } + async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { + this.cells.set('__tracked_addresses', JSON.stringify(entries)); + } + async loadTrackedAddresses(): Promise { + const raw = this.cells.get('__tracked_addresses'); + return raw ? (JSON.parse(raw) as TrackedAddressEntry[]) : []; + } +} + +const dataDirs: string[] = []; +const spheres: SphereInstance[] = []; + +function tempDir(label: string): string { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), `sphere-xbundle-${label}-`)); + dataDirs.push(dir); + return dir; +} + +interface InitArgs { + storage: StorageProvider; + transport?: TransportProvider; + mnemonic?: string; +} + +async function initThrough(mod: SphereModule, args: InitArgs): Promise { + const { sphere } = await mod.Sphere.init({ + storage: args.storage, + transport: args.transport ?? createMockTransport(), + oracle: createEngineOracle(), + walletApi: makePv2World(NET).walletApi, + network: NET, + mnemonic: args.mnemonic, + }); + spheres.push(sphere); + return sphere; +} + +/** A second module instance of core/Sphere — one bundle's copy, not one shared class. */ +async function freshCopy(): Promise { + vi.resetModules(); + return import('../../core/Sphere'); +} + +describe('the Sphere lifecycle registry spans entry points (#766)', () => { + let copyA: SphereModule; + let copyB: SphereModule; + + beforeEach(async () => { + vi.stubGlobal( + 'fetch', + vi.fn(async () => ({ + ok: true, + status: 200, + statusText: 'OK', + json: async () => [], + text: async () => '[]', + } as unknown as Response)), + ); + copyA = await freshCopy(); + copyB = await freshCopy(); + expect(copyA.Sphere, 'two genuinely separate module instances').not.toBe(copyB.Sphere); + }); + + afterEach(async () => { + for (const sphere of spheres.splice(0)) { + try { await sphere.destroy(); } catch { /* the test already tore it down */ } + } + for (const dir of dataDirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true }); + vi.unstubAllGlobals(); + }); + + it('destroys a Sphere built through one copy when the OTHER copy clears its store', async () => { + const dataDir = tempDir('cleared'); + const sphere = await initThrough(copyA, { storage: new FileStorageProvider({ dataDir }), mnemonic: MNEMONIC_A }); + expect(sphere.isReady).toBe(true); + + // The consumer's second entry point: its own provider object over the same wallet.json. + await copyB.Sphere.clear({ storage: new FileStorageProvider({ dataDir }) }); + + expect(sphere.isReady, 'left ready over a KV that no longer exists').toBe(false); + expect(() => sphere.payments, 'a stopped vertical must throw, not serve').toThrow(); + }); + + it('leaves a Sphere on an unrelated store alone when the other copy clears', async () => { + const kept = tempDir('kept'); + const wiped = tempDir('wiped'); + const sphere = await initThrough(copyA, { storage: new FileStorageProvider({ dataDir: kept }), mnemonic: MNEMONIC_A }); + + await copyB.Sphere.clear({ storage: new FileStorageProvider({ dataDir: wiped }) }); + + expect(sphere.isReady, 'scoping must survive the merge, not collapse into one bucket').toBe(true); + expect(() => sphere.payments).not.toThrow(); + }); + + it('refuses a publication from one copy over a store the other copy cleared', async () => { + const dataDir = tempDir('generation'); + const transport = createMockTransport(); + // Park the init inside its bring-up: keys on disk, nothing published yet, so the + // clear cannot see it to destroy it and the generation is the only signal left. + let release!: () => void; + const parked = new Promise((resolve) => { release = resolve; }); + let reached!: () => void; + const atGate = new Promise((resolve) => { reached = resolve; }); + (transport.publishIdentityBinding as unknown as ReturnType).mockImplementation( + async () => { reached(); await parked; return true; }, + ); + + const init = initThrough(copyA, { storage: new FileStorageProvider({ dataDir }), transport, mnemonic: MNEMONIC_A }); + await atGate; + + await copyB.Sphere.clear({ storage: new FileStorageProvider({ dataDir }) }); + release(); + + await expect(init).rejects.toThrow(/cleared while this wallet was initializing/); + expect(transport.disconnect, 'the refused Sphere is torn down, not leaked').toHaveBeenCalled(); + }); + + it('does not collide two undeclared stores on one object key across copies', async () => { + // No `backingStoreId`, so each provider gets a minted key. Per-copy minting hands + // BOTH of these `object:1`, and the shared registry then files two unrelated + // wallets in one bucket — one clear() destroying a wallet it never touched. + const kept: StorageProvider = new MemoryStorage(new Map()); + const wiped: StorageProvider = new MemoryStorage(new Map()); + expect(kept.backingStoreId).toBeUndefined(); + + const sphere = await initThrough(copyA, { storage: kept, mnemonic: MNEMONIC_A }); + await initThrough(copyB, { storage: wiped, mnemonic: MNEMONIC_B }); + + await copyB.Sphere.clear({ storage: wiped }); + + expect(sphere.isReady, 'a different store, a different wallet, untouched').toBe(true); + expect(() => sphere.payments).not.toThrow(); + }); +}); diff --git a/tests/unit/core/global-cell.test.ts b/tests/unit/core/global-cell.test.ts new file mode 100644 index 00000000..a59765f1 --- /dev/null +++ b/tests/unit/core/global-cell.test.ts @@ -0,0 +1,175 @@ +/** + * `sharedCell` — the one place SDK state is allowed to be PROCESS-wide. + * + * Why it exists: tsup builds every subpath export as its own bundle with + * `splitting: false` (tsup.shared.js), so `@unicitylabs/sphere-sdk` and + * `@unicitylabs/sphere-sdk/core` each carry a private copy of every module they + * import — and the ESM and CJS outputs duplicate them again. Module-level state is + * therefore per-BUNDLE. For state that answers a question about IDENTITY ("is this + * the same backing store?", "have I seen this object?") a second copy is not a + * cache miss, it is a WRONG ANSWER: #766's `clear()` destroying a Sphere it does not + * own, one entry point at a time. `core/logger.ts` already stores its state on + * globalThis for the same reason. + * + * Two module copies are simulated with `vi.resetModules()` + two dynamic imports. + * That is a real second module instance in one realm sharing one globalThis — which + * is exactly the shape of the hazard. It does NOT reproduce a second REALM (an + * iframe, a worker): those have their own globalThis and cannot be joined by any + * in-process mechanism, and nothing here claims otherwise. + * + * The key is versioned (`_v1`) because the SHAPE of the cells is not a public + * contract: a future release that changes a cell's fields moves to `_v2` rather than + * handing an old reader a structure it cannot use. Within a version, `intact()` + * still re-validates — a globalThis key is reachable by any page script, so a + * foreign or hostile value must be refused rather than adopted, and must not throw + * on the way in. Nothing secret is ever stored: instances and counters only. + */ + +import { describe, expect, it, vi } from 'vitest'; + +import { sharedCell } from '../../../core/global-cell'; + +const CELLS_KEY = '__sphere_sdk_cells_v1__'; + +interface Counter { n: number } + +const isCounter = (candidate: object): boolean => typeof (candidate as Partial).n === 'number'; + +let seq = 0; +const uniqueName = (): string => `test.cell.${++seq}`; + +const host = (): Record => globalThis as unknown as Record; + +/** The live bag, created on demand — the tests that plant a value need it to exist. */ +function bag(): Record { + sharedCell(uniqueName(), () => ({ n: 0 }), isCounter); + return host()[CELLS_KEY] as Record; +} + +async function freshCopy(): Promise { + vi.resetModules(); + return import('../../../core/global-cell'); +} + +describe('sharedCell survives the bundle split', () => { + it('hands two module copies the SAME cell', async () => { + const copyA = await freshCopy(); + const copyB = await freshCopy(); + expect(copyA.sharedCell, 'two genuinely separate module instances').not.toBe(copyB.sharedCell); + + const name = uniqueName(); + const a = copyA.sharedCell(name, () => ({ n: 0 }), isCounter); + a.n = 7; + const b = copyB.sharedCell(name, () => ({ n: 0 }), isCounter); + + expect(b, 'the second copy must not mint its own').toBe(a); + expect(b.n).toBe(7); + }); + + it('keeps differently-named cells apart', () => { + const one = sharedCell(uniqueName(), () => ({ n: 1 }), isCounter); + const two = sharedCell(uniqueName(), () => ({ n: 2 }), isCounter); + + expect(one).not.toBe(two); + expect(two.n).toBe(2); + }); +}); + +describe('sharedCell refuses what it finds rather than trusting it', () => { + it('replaces a cell of the wrong shape instead of handing it to the SDK', () => { + const cells = bag(); + const name = uniqueName(); + cells[name] = { n: 'not a number' }; + + const cell = sharedCell(name, () => ({ n: 5 }), isCounter); + + expect(cell.n).toBe(5); + expect(cells[name], 'the impostor is evicted, not left for the next reader').toBe(cell); + }); + + it('treats a validator that throws as a refusal, not a crash', () => { + const cells = bag(); + const name = uniqueName(); + // A page script can leave anything here, including a trap that throws on read. + cells[name] = new Proxy({}, { get(): never { throw new Error('hostile'); } }); + + const cell = sharedCell(name, () => ({ n: 3 }), isCounter); + + expect(cell.n).toBe(3); + }); + + it('keeps one stable cell per bundle when the bag itself is frozen', async () => { + const saved = Object.getOwnPropertyDescriptor(host(), CELLS_KEY); + try { + // A page can freeze the bag it can reach; the write fails, and re-minting a cell + // per CALL would hand two callers in one bundle two different registries. + Object.defineProperty(host(), CELLS_KEY, { + value: Object.freeze({}), writable: true, configurable: true, enumerable: false, + }); + const copy = await freshCopy(); + const name = uniqueName(); + + const first = copy.sharedCell(name, () => ({ n: 1 }), isCounter); + first.n = 4; + const again = copy.sharedCell(name, () => ({ n: 1 }), isCounter); + + expect(again).toBe(first); + expect(again.n).toBe(4); + } finally { + if (saved) Object.defineProperty(host(), CELLS_KEY, saved); + else delete host()[CELLS_KEY]; + } + }); + + it('does not crash when a non-object is sitting at the globalThis key', async () => { + const saved = Object.getOwnPropertyDescriptor(host(), CELLS_KEY); + try { + Object.defineProperty(host(), CELLS_KEY, { + value: 'hostile', writable: true, configurable: true, enumerable: false, + }); + const copy = await freshCopy(); + + const cell = copy.sharedCell(uniqueName(), () => ({ n: 1 }), isCounter); + + expect(cell.n, 'import-time state must not depend on what the page left behind').toBe(1); + expect(typeof host()[CELLS_KEY], 'the string was replaced by a real bag').toBe('object'); + } finally { + if (saved) Object.defineProperty(host(), CELLS_KEY, saved); + else delete host()[CELLS_KEY]; + } + }); +}); + +describe('what the cell exposes to the page', () => { + it('is a single non-enumerable property — instances and counters, never key material', () => { + sharedCell(uniqueName(), () => ({ n: 0 }), isCounter); + + const descriptor = Object.getOwnPropertyDescriptor(host(), CELLS_KEY); + expect(descriptor?.enumerable, 'not something an Object.keys(globalThis) dump walks into').toBe(false); + expect(Object.keys(globalThis)).not.toContain(CELLS_KEY); + }); + + // LAST in the file on purpose: the key it plants cannot be made configurable again. + it('degrades to bundle-local cells when globalThis refuses the key', async () => { + const saved = Object.getOwnPropertyDescriptor(host(), CELLS_KEY); + try { + Object.defineProperty(host(), CELLS_KEY, { + value: 'locked', writable: true, configurable: false, enumerable: false, + }); + const copy = await freshCopy(); + const name = uniqueName(); + + const first = copy.sharedCell(name, () => ({ n: 1 }), isCounter); + first.n = 9; + const again = copy.sharedCell(name, () => ({ n: 1 }), isCounter); + + expect(again, 'one stable cell per bundle is the floor, never a fresh one per call').toBe(first); + expect(again.n).toBe(9); + expect(host()[CELLS_KEY], 'and the host property is left exactly as it was found').toBe('locked'); + } finally { + // Non-configurable now, but still writable: a valid bag at the key is all any + // later caller needs, because an existing object is never re-installed. + host()[CELLS_KEY] = saved?.value ?? {}; + } + }); +}); diff --git a/tests/unit/impl/backing-store-id.test.ts b/tests/unit/impl/backing-store-id.test.ts index 10b41716..8f744630 100644 --- a/tests/unit/impl/backing-store-id.test.ts +++ b/tests/unit/impl/backing-store-id.test.ts @@ -9,7 +9,7 @@ * (the default fallback) never collides at all and misses the case this exists for. */ -import { describe, expect, it } from 'vitest'; +import { describe, expect, it, vi } from 'vitest'; import * as os from 'os'; import * as path from 'path'; @@ -115,6 +115,53 @@ describe('LocalStorageProvider.backingStoreId', () => { }); }); +/** + * The tag that separates two `Storage` objects is minted by a counter, and a counter is + * per-MODULE. tsup ships each subpath export as its own bundle (`splitting: false`), and + * ESM/CJS duplicate them again, so two copies of this file each hand THEIR first unrelated + * `Storage` the tag `1` — two unrelated stores reporting one `backingStoreId`, which is + * what `Sphere.clear()` uses to decide whose wallet it may destroy. `vi.resetModules()` + * plus two dynamic imports is a real second module instance over one globalThis: the + * two-bundle case exactly. It is NOT a second realm (an iframe or worker has its own + * globalThis and cannot be joined at all), and nothing below claims to cover one. + */ +describe('LocalStorageProvider tags are process-wide, not per-module-copy', () => { + async function freshCopy(): Promise { + vi.resetModules(); + return import('../../../impl/browser/storage/LocalStorageProvider'); + } + + it('never gives two UNRELATED Storage objects one id across two module copies', async () => { + const copyA = await freshCopy(); + const copyB = await freshCopy(); + expect(copyA.LocalStorageProvider, 'two genuinely separate module instances').not.toBe( + copyB.LocalStorageProvider, + ); + + const a = new copyA.LocalStorageProvider({ storage: fakeStorage(), prefix: 'sphere_' }); + const b = new copyB.LocalStorageProvider({ storage: fakeStorage(), prefix: 'sphere_' }); + + expect(a.backingStoreId, 'a collision here is one wallet clearing another').not.toBe( + b.backingStoreId, + ); + }); + + it('gives ONE Storage object the same id from either copy', async () => { + const copyA = await freshCopy(); + const copyB = await freshCopy(); + // A head start for one copy, so two independent counters cannot agree by accident. + new copyA.LocalStorageProvider({ storage: fakeStorage(), prefix: 'unrelated_' }); + const storage = fakeStorage(); + + const a = new copyA.LocalStorageProvider({ storage, prefix: 'sphere_' }); + const b = new copyB.LocalStorageProvider({ storage, prefix: 'sphere_' }); + + expect(b.backingStoreId, 'one store, so one id — that is what erasure follows').toBe( + a.backingStoreId, + ); + }); +}); + describe('the three provider kinds never collide', () => { it('gives every implementation its own namespace', () => { const ids = [ diff --git a/tests/unit/storage/tracked-addresses-providers.test.ts b/tests/unit/storage/tracked-addresses-providers.test.ts index 3de59d47..0145fada 100644 --- a/tests/unit/storage/tracked-addresses-providers.test.ts +++ b/tests/unit/storage/tracked-addresses-providers.test.ts @@ -9,9 +9,10 @@ import 'fake-indexeddb/auto'; import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; -import { vi } from 'vitest'; +import { describe, expect, it, vi } from 'vitest'; import type { StorageProvider } from '../../../storage'; +import type { TrackedAddressEntry } from '../../../types'; import { IndexedDBStorageProvider } from '../../../impl/browser/storage/IndexedDBStorageProvider'; import { LocalStorageProvider } from '../../../impl/browser/storage/LocalStorageProvider'; import { FileStorageProvider } from '../../../impl/nodejs/storage/FileStorageProvider'; @@ -39,6 +40,49 @@ function failNextSet(provider: StorageProvider): (message: string) => void { }; } +/** + * The cross-object merge one level further out: the two providers come from two SEPARATE + * MODULE COPIES. tsup gives every subpath export its own bundle (`splitting: false`) and + * ESM/CJS duplicate them again, so a chain map held in module scope orders each copy's + * writes against itself and nothing else — both copies read the same empty registry and + * the later write erases the earlier one's address (#766 item 5, across the entry points). + * `vi.resetModules()` + two dynamic imports is a real second instance over one globalThis. + */ +describe('saveTrackedAddresses serializes across module copies', () => { + const AT = 1_000; + const entry = (index: number): TrackedAddressEntry => + ({ index, hidden: false, createdAt: AT, updatedAt: AT }); + + it('merges concurrent writes from providers built by two different copies', async () => { + vi.resetModules(); + const copyA = await import('../../../impl/browser/storage/LocalStorageProvider'); + vi.resetModules(); + const copyB = await import('../../../impl/browser/storage/LocalStorageProvider'); + expect(copyA.LocalStorageProvider, 'two genuinely separate module instances').not.toBe( + copyB.LocalStorageProvider, + ); + + const storage = memoryWebStorage(); + const a = new copyA.LocalStorageProvider({ prefix: 'crossbundle_', storage }); + const b = new copyB.LocalStorageProvider({ prefix: 'crossbundle_', storage }); + await a.connect(); + await b.connect(); + + // Disjoint snapshots, no await between them: unserialized, both read the empty + // registry and whichever lands last is the only one that survives. + await Promise.all([ + a.saveTrackedAddresses([entry(0), entry(1)]), + b.saveTrackedAddresses([entry(0), entry(2)]), + ]); + + const stored = (await b.loadTrackedAddresses()).map((each) => each.index).sort((x, y) => x - y); + expect(stored, 'neither writer may lose its address to the other').toEqual([0, 1, 2]); + + await a.disconnect(); + await b.disconnect(); + }); +}); + describeTrackedAddressesContract('FileStorageProvider', async () => { const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'sphere-tracked-')); const provider = new FileStorageProvider({ dataDir }); From c40c74090a2ac02ef28d2ffa8062c3f6c24375b7 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 03:55:30 +0200 Subject: [PATCH 33/43] test(mutation): probes for the cross-bundle identity cell MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five, one per split structure. The LocalStorage one deliberately replaces the whole sharedCell() call with a bundle-local literal rather than swapping a single reference — a partial edit there does not parse, and a probe killed by a transform error proves nothing about the guard. --- tests/mutation/probes.json | 51 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index ad7f3f83..d49de0df 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -1112,5 +1112,56 @@ "tests": [ "tests/unit/core/network-health.test.ts" ] + }, + { + "name": "sphere-live-registry-splits-per-bundle", + "note": "#772 round 6: every subpath export is its own tsup bundle, so a class static splits \u2014 a Sphere built through the root entry becomes invisible to a clear() called through ./core, which then wipes the store and leaves it ready over an emptied KV", + "file": "core/Sphere.ts", + "find": " private static readonly _liveByStorage: Map> = lifecycle.live;", + "replace": " private static readonly _liveByStorage = new Map>();", + "tests": [ + "tests/integration/sphere-cross-bundle-lifecycle.test.ts" + ] + }, + { + "name": "sphere-clear-generations-split-per-bundle", + "note": "#772 round 6: split per bundle, an init in the OTHER bundle never sees the clear and publishes a Sphere over a wiped store", + "file": "core/Sphere.ts", + "find": " private static readonly _clearGenerations: Map = lifecycle.clearGenerations;", + "replace": " private static readonly _clearGenerations = new Map();", + "tests": [ + "tests/integration/sphere-cross-bundle-lifecycle.test.ts" + ] + }, + { + "name": "sphere-object-store-keys-split-per-bundle", + "note": "#772 round 6: with the live map shared but the key allocator split, two bundles hand DIFFERENT providers the same object:N key and put two unrelated wallets in one bucket \u2014 a clear() on either destroys both", + "file": "core/Sphere.ts", + "find": " private static readonly _objectStoreKeys: WeakMap =\n lifecycle.objectStoreKeys;", + "replace": " private static readonly _objectStoreKeys = new WeakMap();", + "tests": [ + "tests/integration/sphere-cross-bundle-lifecycle.test.ts" + ] + }, + { + "name": "globalcell-adopts-a-hostile-value", + "note": "#772 round 6: the cell is read from globalThis, so a validator that throws (a Proxy trap) must be a refusal, not an SDK-wide crash at import time", + "file": "core/global-cell.ts", + "find": " try {\n return intact(candidate);\n } catch {\n return false;\n }", + "replace": " return intact(candidate);", + "tests": [ + "tests/unit/core/global-cell.test.ts" + ] + }, + { + "name": "localstorage-tags-split-per-bundle", + "note": "#772 round 6: the tag feeds backingStoreId, which is PROCESS-level identity. Split per module copy, each first Storage object gets tag 1 \u2014 two unrelated wallets collide and clear() destroys the wrong one", + "file": "impl/browser/storage/LocalStorageProvider.ts", + "find": "const storeIdentity = sharedCell(\n 'impl.browser.localStorage.identity@1',\n () => ({ tags: new WeakMap(), writeChains: new Map(), seq: 0 }),\n (cell) => {\n const c = cell as Partial;\n return c.tags instanceof WeakMap && c.writeChains instanceof Map && typeof c.seq === 'number';\n },\n);\n", + "replace": "const storeIdentity: StorageIdentityCell = { tags: new WeakMap(), writeChains: new Map(), seq: 0 };\nvoid sharedCell;\n", + "tests": [ + "tests/unit/impl/backing-store-id.test.ts", + "tests/unit/storage/tracked-addresses-providers.test.ts" + ] } ] From 501c3c8b061bfe8053f2d33b36c82890fa52b459 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 03:56:14 +0200 Subject: [PATCH 34/43] docs(changelog): the cross-bundle identity fix --- CHANGELOG.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index bc980714..0f5449e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,32 @@ Four defects of one shape: work spawned by a component outlived the component. both. A refused switch also no longer persists the address index it never finished moving to, which would have sent the next boot to the wrong address. +### Fixed — identity state no longer splits per bundle (#766) + +`tsup` ships each subpath export as its own bundle with `splitting: false`, and ESM and +CJS duplicate it again — so class statics are per-bundle, not per-process. That made the +lifecycle fixes above incomplete at the entry-point boundary: a Sphere built through +`@unicitylabs/sphere-sdk` was invisible to a `Sphere.clear()` called through +`@unicitylabs/sphere-sdk/core`, which wiped the backing store and left the first Sphere +`isReady` over an emptied KV — the original #766 failure, one level out. The clear +generation split the same way, so an in-flight init in the other bundle also missed the +clear. + +`LocalStorageProvider` had the same shape for a different reason: it minted its +`backingStoreId` tags from a module-level counter, so each copy gave its first unrelated +`Storage` object the tag `1` — two different stores, one id, and `clear()` erasing the +wrong wallet. Its read-merge-write serializer was module-local too, which is the +tracked-address lost update one level out. + +All of it now lives in a versioned `globalThis` cell, the pattern `core/logger.ts` +already uses. The cell is non-enumerable, validated on read (a foreign value is refused, +not adopted), degrades to bundle-local rather than throwing if `globalThis` is frozen, +and holds only Sphere instances, `backingStoreId` strings and counters — never key +material. `IndexedDBStorageProvider` and `FileStorageProvider` needed no change: their +ids are intrinsic (`dbName` + prefix, resolved path), not counter-allocated. + +No public API change. + ### Changed — three refusals that used to be silent successes Found by review of the fixes above; each turns a wrong answer into a typed error. From 5e89cba16e8227bda0a85a6b5f50aa216e43f4e9 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 04:17:18 +0200 Subject: [PATCH 35/43] test: restore the coverage two new guards had silently taken away MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 116-probe run left three survivors, all mine, and none of them a weak test in the usual sense — two were guards added THIS round shadowing older ones: - verifier-post-dispose-gate: the reentrancy re-check added for the round-6 review also rejects a post-dispose verify, so deleting the entry gate no longer changed the outcome. The entry gate's actual job is to not ENTER the torn-down SDK verifier at all, so the test now spies WorkerTokenVerifier.prototype.verify and asserts it is never called — the one seam that tells the two guards apart. - sphere-switch-rebuilds-transport-mux-after-destroy: the post-bring-up discard added for the codex review undoes whatever the bring-up built, so asserting the end state could not distinguish 'never entered' from 'entered and cleaned up'. The test now asserts buildTokenEngine was never called. - sphere-object-store-keys-split-per-bundle was simply the wrong mutation: it swapped the shared WeakMap while leaving the shared counter, which still hands out distinct keys. Repointed at the key expression itself (a constant key, the same failure mode as the file-provider probe) and renamed to match what it actually mutates. Each verified to red, and to red for the right reason: 'expected verify to not be called', 'expected bound buildTokenEngine to not be called', and 'a different store, a different wallet, untouched'. --- .../sphere-payments-v2-wiring.test.ts | 13 ++++++++++++- tests/mutation/probes.json | 11 ++++++----- .../token-engine/worker-verification.test.ts | 17 ++++++++++++++--- 3 files changed, 32 insertions(+), 9 deletions(-) diff --git a/tests/integration/sphere-payments-v2-wiring.test.ts b/tests/integration/sphere-payments-v2-wiring.test.ts index 27e7382b..6381d4c2 100644 --- a/tests/integration/sphere-payments-v2-wiring.test.ts +++ b/tests/integration/sphere-payments-v2-wiring.test.ts @@ -898,6 +898,16 @@ describe('Sphere payments wiring — defaults (P11 flip: the vertical is default return null; }; + // The bring-up must not be ENTERED, not merely undone. The guard after it discards + // whatever it built, so asserting only the end state cannot tell the two apart — + // and that is precisely what let this guard's mutation probe survive a full run. + const internals = sphere as unknown as { + buildTokenEngine: (identity: unknown) => Promise; + }; + const realBuild = internals.buildTokenEngine.bind(sphere); + const buildSpy = vi.fn(realBuild); + internals.buildTokenEngine = buildSpy; + const switching = sphere.switchToAddress(1, { nametag: 'zed' }).then( () => 'resolved' as const, (err: unknown) => err @@ -918,8 +928,9 @@ describe('Sphere payments wiring — defaults (P11 flip: the vertical is default world.transports.filter((t) => t.session.startCalls === 1 && t.session.stopCalls === 0) ).toHaveLength(0); expect(world.transports).toHaveLength(1); - // Nothing rebuilt: no module set, no mux. + // Nothing rebuilt: no module set, no mux — and nothing was built to be undone. expect(modules.size).toBe(0); + expect(buildSpy).not.toHaveBeenCalled(); expect((sphere as unknown as { _transportMux: unknown })._transportMux).toBeNull(); expect(outcome).toBeInstanceOf(SphereError); expect((outcome as SphereError).code).toBe('NOT_INITIALIZED'); diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index d49de0df..62f81a67 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -1134,13 +1134,14 @@ ] }, { - "name": "sphere-object-store-keys-split-per-bundle", - "note": "#772 round 6: with the live map shared but the key allocator split, two bundles hand DIFFERENT providers the same object:N key and put two unrelated wallets in one bucket \u2014 a clear() on either destroys both", + "name": "sphere-object-store-key-is-a-constant", + "note": "#772 round 6: a provider that declares no backingStoreId is scoped to ITSELF. A constant key puts every such provider \u2014 in every bundle \u2014 into one liveness bucket, so a clear() on one wallet destroys another's Sphere", "file": "core/Sphere.ts", - "find": " private static readonly _objectStoreKeys: WeakMap =\n lifecycle.objectStoreKeys;", - "replace": " private static readonly _objectStoreKeys = new WeakMap();", + "find": " key = `object:${++lifecycle.objectStoreSeq}`;", + "replace": " key = 'object:1';", "tests": [ - "tests/integration/sphere-cross-bundle-lifecycle.test.ts" + "tests/integration/sphere-cross-bundle-lifecycle.test.ts", + "tests/integration/sphere-instance-scoping.test.ts" ] }, { diff --git a/tests/unit/token-engine/worker-verification.test.ts b/tests/unit/token-engine/worker-verification.test.ts index 9130786a..913c2d3f 100644 --- a/tests/unit/token-engine/worker-verification.test.ts +++ b/tests/unit/token-engine/worker-verification.test.ts @@ -355,9 +355,20 @@ describe('dispose() during an in-flight verification (#770 item 4)', () => { // A 0-transfer token never reaches the pool, so no other guard can notice the // teardown: without the `disposed` gate on verify(), a torn-down engine keeps // handing out verdicts as if it were live. - const outcome = await settleWithin(engine.verify(mintedOnly), 3000); - expect(outcome.state).toBe('rejected'); - expect(outcome).toMatchObject({ reason: { code: 'MODULE_DESTROYED' } }); + // + // Asserting only the rejection is NOT enough any more: the post-registration + // re-check would reject too, which made this guard's mutation probe SURVIVE. The + // gate's own job is to not ENTER the torn-down SDK verifier at all, so that is + // what is asserted — through the base method, the one seam that can tell them apart. + const base = vi.spyOn(WorkerTokenVerifier.prototype, 'verify'); + try { + const outcome = await settleWithin(engine.verify(mintedOnly), 3000); + expect(outcome.state).toBe('rejected'); + expect(outcome).toMatchObject({ reason: { code: 'MODULE_DESTROYED' } }); + expect(base).not.toHaveBeenCalled(); + } finally { + base.mockRestore(); + } expect(createWorker).not.toHaveBeenCalled(); }, 30000); From 1d33b2ab683c40a0df61ffd914f9754e17a0ba7d Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 04:39:53 +0200 Subject: [PATCH 36/43] docs(storage): say IN the provider why its tracked-write chain stays per-object MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two independent reviewers have now proposed sharing this chain across provider objects, because backingStoreId makes them look like one store. The refusal was recorded only in the contract test's unsupported: note, which nobody reading the provider sees — so it keeps being re-derived. The reason, now where it is read: this provider rewrites the WHOLE file from a per-object cache, so a sibling's unrelated set() rolls the registry back however carefully this one write is serialized. Sharing the chain would make the skipped cross-object contract case pass while leaving the provider unsafe. backingStoreId scopes TEARDOWN; the concurrent-write half is #771. --- impl/nodejs/storage/FileStorageProvider.ts | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/impl/nodejs/storage/FileStorageProvider.ts b/impl/nodejs/storage/FileStorageProvider.ts index 86319f90..2424f00d 100644 --- a/impl/nodejs/storage/FileStorageProvider.ts +++ b/impl/nodejs/storage/FileStorageProvider.ts @@ -192,6 +192,15 @@ export class FileStorageProvider implements StorageProvider { * (#766 item 5 — a lost update, reproducible on one network). Concurrent * calls are serialized on `trackedWrites` so a read can never interleave * with another call's write. + * + * Deliberately PER OBJECT, unlike the browser providers. Two objects over one + * `dataDir` share a `backingStoreId` but not this cache, and `save()` rewrites the + * WHOLE file from it — so a sibling's *unrelated* `set()` rolls the registry back + * regardless of how this one write is serialized. Sharing the chain here would make + * the cross-object contract case pass while leaving the provider unsafe. The real + * fix is #771 (refresh from disk under a per-file lock, on every write); until then + * `backingStoreId` scopes TEARDOWN only. Reviewers keep re-finding this — see the + * `unsupported:` note in tests/unit/storage/tracked-addresses-providers.test.ts. */ async saveTrackedAddresses(entries: TrackedAddressEntry[]): Promise { const run = this.trackedWrites.then(async () => { From 4ec8a271cab90a8cc4add757a2596d8a40a653f1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 3 Sep 2026 09:28:29 +0000 Subject: [PATCH 37/43] chore: release v0.16.0-dev.1 --- connect/version.ts | 2 +- package-lock.json | 4 ++-- package.json | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/connect/version.ts b/connect/version.ts index 84d1fb8c..1bd48b2a 100644 --- a/connect/version.ts +++ b/connect/version.ts @@ -1,2 +1,2 @@ // AUTO-GENERATED by scripts/gen-version.mjs — do not edit. -export const SDK_VERSION = '0.15.0'; +export const SDK_VERSION = '0.16.0-dev.1'; diff --git a/package-lock.json b/package-lock.json index 8c95d957..3e6db733 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@unicitylabs/sphere-sdk", - "version": "0.15.0", + "version": "0.16.0-dev.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@unicitylabs/sphere-sdk", - "version": "0.15.0", + "version": "0.16.0-dev.1", "license": "MIT", "dependencies": { "@noble/ciphers": "^2.2.0", diff --git a/package.json b/package.json index 091fa62f..e914785f 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@unicitylabs/sphere-sdk", - "version": "0.15.0", + "version": "0.16.0-dev.1", "description": "Modular TypeScript SDK for Unicity wallet operations", "type": "module", "main": "./dist/index.cjs", From c409401ae6d9e5a65ef92b995c3195e6b881deb2 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 13:45:05 +0200 Subject: [PATCH 38/43] fix(connect): a network change under a LIVE session must revoke it too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit checkCompatibility runs only in handleHandshake, and updateSphere returns EARLY for a live host — before the lock-edge guard, which is itself behind wasLocked. So the live rebind was the one path that re-ran no compatibility check at all: a host switching network without locking kept the approved session and served the dApp a chain it never agreed to, while sphere_getIdentity still reported the old network. Worse for the dApp, it cannot detect this itself. Keys and the DIRECT:// address are network-free (deriveDirectAddress takes no network, base path m/44'/0'/0'), so identity:changed does not fire and the address looks unchanged; networkId is surfaced only in the handshake response, so a long-lived session has no channel to learn the network moved. The comparison is hoisted into the live branch, network only — identity is deliberately not compared there, because changing it is what an address switch IS. Pinned both ways: a network change revokes and pushes wallet:disconnected (and the snapshot still moves, so the next handshake reports where the wallet is), and an address switch on the same network keeps the session. Not reachable from the Sphere frontend, which reloads the page on a switch and so rebuilds the host with no session — but Connect is a public protocol and nothing in the API stopped another host from doing it. --- connect/host/ConnectHost.ts | 14 ++++++++++++++ tests/mutation/probes.json | 10 ++++++++++ tests/unit/connect/lock.test.ts | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/connect/host/ConnectHost.ts b/connect/host/ConnectHost.ts index 485c546c..8b366290 100644 --- a/connect/host/ConnectHost.ts +++ b/connect/host/ConnectHost.ts @@ -256,6 +256,20 @@ export class ConnectHost { } if (this._walletState === 'live') { + // THIS rebind never re-runs checkCompatibility: the lock-edge guard below sits + // behind `wasLocked`. Unchecked, a host switching network without locking keeps + // the session and serves a chain the dApp never agreed to. Identity is NOT + // compared here — changing it is what an address switch IS. + if (this.session?.active && (this.snapshot.networkId ?? null) !== (next.networkId ?? null)) { + logger.warn( + 'ConnectHost', + `Network changed under a live session — revoking instead of rebinding (origin=${this.config.origin ?? 'unverified'})`, + ); + this.sphere = next; + this.snapshot = buildWalletSnapshot(next); + this.revokeSession(); + return; + } // Address switch on a live host — today's behaviour, verbatim. this.sphere = next; this.snapshot = buildWalletSnapshot(next); diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 62f81a67..accf8c4c 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -1164,5 +1164,15 @@ "tests/unit/impl/backing-store-id.test.ts", "tests/unit/storage/tracked-addresses-providers.test.ts" ] + }, + { + "name": "connect-live-rebind-skips-network-check", + "note": "#772: updateSphere returns early for a LIVE host and the lock-edge guard sits behind wasLocked, so this is the one rebind that re-runs no compatibility check. Unguarded, a host switching network without locking keeps the approved session and serves the dApp a chain it never agreed to", + "file": "connect/host/ConnectHost.ts", + "find": " if (this.session?.active && (this.snapshot.networkId ?? null) !== (next.networkId ?? null)) {", + "replace": " if (false) {", + "tests": [ + "tests/unit/connect/lock.test.ts" + ] } ] diff --git a/tests/unit/connect/lock.test.ts b/tests/unit/connect/lock.test.ts index 77963d4d..7b220877 100644 --- a/tests/unit/connect/lock.test.ts +++ b/tests/unit/connect/lock.test.ts @@ -1337,6 +1337,38 @@ describe('updateSphere() re-arm after a lock', () => { expect(h.host.getSession()).toBeNull(); }); + it('REVOKES when the network changes under a LIVE session, without any lock', async () => { + // The lock-edge guard sits behind `wasLocked`, and updateSphere returns early for a + // live host — so this rebind is the ONE that re-runs no compatibility check at all. + // A host that switches network without locking would otherwise keep the approved + // session and serve the dApp a chain it never agreed to, while sphere_getIdentity + // still reported the old network. + const h = await connectHarness(); + expect(h.host.walletState).toBe('live'); + expect(h.host.getSession()).not.toBeNull(); + + h.host.updateSphere(createMockSphere({ networkId: 7 })); + + expect(eventsOfType(h.pair.hostSent, WALLET_EVENTS.DISCONNECTED)).toHaveLength(1); + expect(h.host.getSession()).toBeNull(); + expect(h.host.walletState).toBe('live'); + // The snapshot still moves: the next handshake must report where the wallet IS. + // eslint-disable-next-line @typescript-eslint/no-explicit-any + expect((h.host as any).snapshot.networkId).toBe(7); + }); + + it('keeps a LIVE session across an address switch on the same network', async () => { + // The companion to the case above: identity changing is what an address switch IS, + // so the network guard must not turn every switch into a disconnect. + const h = await connectHarness(); + + h.host.updateSphere(createMockSphere({ chainPubkey: '02anotheraddress' })); + + expect(eventsOfType(h.pair.hostSent, WALLET_EVENTS.DISCONNECTED)).toHaveLength(0); + expect(h.host.getSession()).not.toBeNull(); + expect(eventsOfType(h.pair.hostSent, WALLET_EVENTS.IDENTITY_CHANGED).length).toBeGreaterThan(0); + }); + it('re-arms silently when the locked host had no session', () => { const pair = createMockTransportPair(); const host = makeHost(pair, { sphere: null, initialWalletState: 'locked' }); From 5225f443410a79876b7ab45f02a3f9c461ca830a Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 13:57:02 +0200 Subject: [PATCH 39/43] feat(constants)!: there is one testnet, so call it Testnet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v1 goggregator testnet is gone — 'testnet' has pointed at the v2 network since #765, and 'testnet2' is an alias of the same configuration. Displaying 'Testnet2' asks every user to reason about a version distinction that no longer exists. DISPLAY ONLY. NETWORKS[].name has zero non-display consumers (both entries feed labels: the badge, the switcher rows, the modals). The identifier is untouched: 'testnet2' remains the exact string wallet-api matches on, the scope in pv2g2:{network}:{pubkey}: and the file name of the token registry, and SPHERE_NETWORKS[].name — the lowercase 'testnet2' that rides the Connect handshake — is a different field and did not move. Marked breaking because it is user-visible and any consumer asserting the string sees it change; nothing functional depends on it. Collapsing the two IDENTIFIERS into one is a separate, much larger question: the id is a wire string shared with wallet-api and the scope of every wallet's local KV, so renaming it would orphan stored state and needs a coordinated release. --- constants.ts | 4 ++-- tests/unit/impl/shared/resolvers.test.ts | 5 ++++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/constants.ts b/constants.ts index f00da557..69f5eadc 100644 --- a/constants.ts +++ b/constants.ts @@ -312,7 +312,7 @@ export const NETWORKS = { // old goggregator testnet spoke the removed v1 protocol — a v2 engine cannot // run against it. 'testnet2' stays as an alias of the same configuration. testnet: { - name: 'Testnet2', + name: 'Testnet', networkId: 4, // v2 state-transition gateway (networkId 4 comes from the trust base). apiKey is env-injected. aggregatorUrl: 'https://gateway.testnet2.unicity.network', @@ -322,7 +322,7 @@ export const NETWORKS = { 'https://raw.githubusercontent.com/unicitynetwork/unicity-ids/refs/heads/main/unicity-ids.testnet2.json', }, testnet2: { - name: 'Testnet2', + name: 'Testnet', networkId: 4, // v2 state-transition gateway (networkId 4 comes from the trust base). apiKey is env-injected. aggregatorUrl: 'https://gateway.testnet2.unicity.network', diff --git a/tests/unit/impl/shared/resolvers.test.ts b/tests/unit/impl/shared/resolvers.test.ts index ca8b2460..b828b190 100644 --- a/tests/unit/impl/shared/resolvers.test.ts +++ b/tests/unit/impl/shared/resolvers.test.ts @@ -31,7 +31,10 @@ describe('getNetworkConfig', () => { it('should return testnet config when specified (alias of testnet2 since the v1 cutover)', () => { const config = getNetworkConfig('testnet'); - expect(config.name).toBe('Testnet2'); + // Display label only. The IDENTIFIER stays 'testnet2' — it is the exact string + // wallet-api matches on and the scope in pv2g2:{network}:{pubkey}: — but there is + // no v1 testnet any more, so there is only one testnet to name. + expect(config.name).toBe('Testnet'); expect(config.aggregatorUrl).toBe(NETWORKS.testnet2.aggregatorUrl); }); From a4440913cd5af211c31bbc4f31f31615e03d5171 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 3 Sep 2026 12:15:05 +0000 Subject: [PATCH 40/43] chore: release v0.16.0-dev.2 --- connect/version.ts | 2 +- package-lock.json | 4 ++-- package.json | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/connect/version.ts b/connect/version.ts index 1bd48b2a..58331fa9 100644 --- a/connect/version.ts +++ b/connect/version.ts @@ -1,2 +1,2 @@ // AUTO-GENERATED by scripts/gen-version.mjs — do not edit. -export const SDK_VERSION = '0.16.0-dev.1'; +export const SDK_VERSION = '0.16.0-dev.2'; diff --git a/package-lock.json b/package-lock.json index 3e6db733..abc27fa0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@unicitylabs/sphere-sdk", - "version": "0.16.0-dev.1", + "version": "0.16.0-dev.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@unicitylabs/sphere-sdk", - "version": "0.16.0-dev.1", + "version": "0.16.0-dev.2", "license": "MIT", "dependencies": { "@noble/ciphers": "^2.2.0", diff --git a/package.json b/package.json index e914785f..f033ea92 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@unicitylabs/sphere-sdk", - "version": "0.16.0-dev.1", + "version": "0.16.0-dev.2", "description": "Modular TypeScript SDK for Unicity wallet operations", "type": "module", "main": "./dist/index.cjs", From f6e1a847cd873bc4e845fa8a4fc6dcd7e6875990 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 17:29:40 +0200 Subject: [PATCH 41/43] fix(storage): backingStoreId must name the unit of ERASURE (Codex on #772) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two ways the id disagreed with what clear() actually destroys — both the #766 bug re-entering along a dimension the original fix did not consider. P1, IndexedDB. The id was dbName + PREFIX, but clear() with no prefix calls idbClear(), which empties the whole kv object store — every prefix in it. Two prefixed wallets in one database were therefore in separate liveness buckets while sharing one erasure fate: clearing either wiped the other's data and left its Sphere isReady over an emptied store. The id is now the database alone. Chose that over scoping clear() to the prefix: clear() is documented and used as a whole-wallet wipe (Sphere.clear passes no prefix, deliberately, so the mnemonic goes with it), and narrowing a destructive operation is the riskier half of the fix. Aligning the id is fail-safe — the bucket becomes wider than strictly necessary, and nobody is left live over erased data. Two tests pinned the old, wrong invariant ('differs on the key prefix', 'cannot be forged across the dbName/prefix boundary') and are rewritten to pin the corrected one. P2, files. path.resolve is purely LEXICAL, so a dataDir reached through a symlink and the same directory reached directly produced different ids for ONE wallet.json — and Sphere.clear() through one alias missed the Sphere registered through the other. Now canonicalised: the deepest EXISTING ancestor is passed through realpathSync and the not-yet-created tail appended lexically, so two providers agree whether or not the directory has been created yet. realpathSync alone would have thrown on a fresh dataDir. Both falsified by restoring the previous expression; both probed. The existing file-provider probe went stale with the change and is re-pointed rather than deleted. --- .../storage/IndexedDBStorageProvider.ts | 9 ++- impl/nodejs/storage/FileStorageProvider.ts | 35 ++++++++- tests/mutation/probes.json | 22 +++++- tests/unit/impl/backing-store-id.test.ts | 77 +++++++++++++++++-- 4 files changed, 130 insertions(+), 13 deletions(-) diff --git a/impl/browser/storage/IndexedDBStorageProvider.ts b/impl/browser/storage/IndexedDBStorageProvider.ts index 75bee2aa..6e4720cd 100644 --- a/impl/browser/storage/IndexedDBStorageProvider.ts +++ b/impl/browser/storage/IndexedDBStorageProvider.ts @@ -48,7 +48,7 @@ export class IndexedDBStorageProvider implements StorageProvider { readonly name = 'IndexedDB Storage'; readonly type = 'local' as const; readonly description = 'Browser IndexedDB for large-capacity persistence'; - /** The database + prefix pair — two providers over one pair share erasure (#766). */ + /** The DATABASE — the unit clear() erases, so the unit that shares a fate (#766). */ readonly backingStoreId: string; private prefix: string; @@ -66,8 +66,11 @@ export class IndexedDBStorageProvider implements StorageProvider { this.dbName = config?.dbName ?? DB_NAME; this.network = config?.network; this.debug = config?.debug ?? false; - this.backingStoreId = - `indexeddb:${encodeURIComponent(this.dbName)}:${encodeURIComponent(this.prefix)}`; + // The DATABASE, not the prefix: backingStoreId names the unit of ERASURE, and + // clear() with no prefix calls idbClear(), which empties the whole object + // store. Splitting prefixes into separate liveness buckets let a clear wipe + // another wallet's data and leave its Sphere isReady over the remains. + this.backingStoreId = `indexeddb:${encodeURIComponent(this.dbName)}`; } // =========================================================================== diff --git a/impl/nodejs/storage/FileStorageProvider.ts b/impl/nodejs/storage/FileStorageProvider.ts index 2424f00d..0e112bdd 100644 --- a/impl/nodejs/storage/FileStorageProvider.ts +++ b/impl/nodejs/storage/FileStorageProvider.ts @@ -26,6 +26,38 @@ export interface FileStorageProviderConfig { network?: NetworkType; } +/** + * The canonical path of `p`, for identity rather than for I/O. + * + * `path.resolve` is purely lexical, so a directory reached through a symlink and + * the same directory reached directly produce DIFFERENT strings for ONE file. + * Two providers would then report different `backingStoreId`s, and + * `Sphere.clear()` through one alias would miss the Sphere registered through + * the other — wiping its wallet.json and leaving it isReady over the remains. + * + * `realpathSync` needs the path to exist, and a fresh `dataDir` legitimately + * does not yet. So the deepest EXISTING ancestor is canonicalised and the + * not-yet-created tail appended lexically: two providers agree whether or not + * the directory has been created, as long as the symlinked part of the path + * exists — which is the case that causes the aliasing in the first place. + */ +function canonicalPath(p: string): string { + const resolved = path.resolve(p); + const tail: string[] = []; + let cursor = resolved; + while (!fs.existsSync(cursor)) { + const parent = path.dirname(cursor); + if (parent === cursor) return resolved; // hit the root without finding anything + tail.unshift(path.basename(cursor)); + cursor = parent; + } + try { + return path.join(fs.realpathSync(cursor), ...tail); + } catch { + return resolved; // unreadable ancestor — lexical is the honest fallback + } +} + export class FileStorageProvider implements StorageProvider { readonly id = 'file-storage'; readonly name = 'File Storage'; @@ -55,7 +87,8 @@ export class FileStorageProvider implements StorageProvider { this.network = config.network; } this.isTxtMode = this.filePath.endsWith('.txt'); - this.backingStoreId = `file:${this.filePath}`; + // Canonical, not merely resolved: aliases of one file must share an id. + this.backingStoreId = `file:${canonicalPath(this.filePath)}`; } setIdentity(identity: FullIdentity): void { diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index accf8c4c..1c8e4079 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -835,7 +835,7 @@ "name": "file-backing-store-id-is-a-class-constant", "note": "#766 review: backingStoreId identifies the STORE. A constant (what `id` is) makes every FileStorageProvider in the process one store, so clear() on one wallet tears down every live Sphere - the exact cross-wallet kill #766 removed.", "file": "impl/nodejs/storage/FileStorageProvider.ts", - "find": " this.backingStoreId = `file:${this.filePath}`;", + "find": " this.backingStoreId = `file:${canonicalPath(this.filePath)}`;", "replace": " this.backingStoreId = 'file:'; // mutant: a class constant, every wallet collides", "tests": [ "tests/unit/impl/backing-store-id.test.ts", @@ -1174,5 +1174,25 @@ "tests": [ "tests/unit/connect/lock.test.ts" ] + }, + { + "name": "idb-store-id-splits-on-prefix", + "note": "#772 codex P1: backingStoreId must name the unit of ERASURE. clear() with no prefix empties the whole object store, so two prefixed wallets in one database share a fate \u2014 splitting them lets a clear wipe one wallet's data and leave the other's Sphere isReady over the remains", + "file": "impl/browser/storage/IndexedDBStorageProvider.ts", + "find": " this.backingStoreId = `indexeddb:${encodeURIComponent(this.dbName)}`;", + "replace": " this.backingStoreId = `indexeddb:${encodeURIComponent(this.dbName)}:${encodeURIComponent(this.prefix)}`;", + "tests": [ + "tests/unit/impl/backing-store-id.test.ts" + ] + }, + { + "name": "file-store-id-not-canonical", + "note": "#772 codex P2: path.resolve is lexical, so a symlinked dataDir and the real one give different ids for ONE wallet.json \u2014 clear() through one alias then misses the Sphere registered through the other", + "file": "impl/nodejs/storage/FileStorageProvider.ts", + "find": " this.backingStoreId = `file:${canonicalPath(this.filePath)}`;", + "replace": " this.backingStoreId = `file:${this.filePath}`;", + "tests": [ + "tests/unit/impl/backing-store-id.test.ts" + ] } ] diff --git a/tests/unit/impl/backing-store-id.test.ts b/tests/unit/impl/backing-store-id.test.ts index 8f744630..8c5dbb2d 100644 --- a/tests/unit/impl/backing-store-id.test.ts +++ b/tests/unit/impl/backing-store-id.test.ts @@ -10,6 +10,7 @@ */ import { describe, expect, it, vi } from 'vitest'; +import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; @@ -70,20 +71,80 @@ describe('IndexedDBStorageProvider.backingStoreId', () => { expect(a.backingStoreId, 'those ARE the defaults').toBe(b.backingStoreId); }); - it('differs on the database name and on the key prefix independently', () => { + it('differs on the database name', () => { const base = new IndexedDBStorageProvider({ dbName: 'db-a', prefix: 'p_' }); const otherDb = new IndexedDBStorageProvider({ dbName: 'db-b', prefix: 'p_' }); - const otherPrefix = new IndexedDBStorageProvider({ dbName: 'db-a', prefix: 'q_' }); expect(base.backingStoreId).not.toBe(otherDb.backingStoreId); - expect(base.backingStoreId).not.toBe(otherPrefix.backingStoreId); }); - it('cannot be forged across the dbName/prefix boundary', () => { - // Unencoded concatenation would make ('a:b','c') and ('a','b:c') one store. - const left = new IndexedDBStorageProvider({ dbName: 'a:b', prefix: 'c' }); - const right = new IndexedDBStorageProvider({ dbName: 'a', prefix: 'b:c' }); - expect(left.backingStoreId).not.toBe(right.backingStoreId); + it('IGNORES the key prefix — clear() erases the whole database', () => { + // This used to assert the opposite, and that was the bug: clear() with no + // prefix calls idbClear(), which empties the entire `kv` object store. Two + // prefixed wallets in one database therefore share an ERASURE fate, and + // backingStoreId names the unit of erasure. Split them into separate + // liveness buckets and clearing either wipes the other's data while leaving + // its Sphere isReady over an emptied store — #766, one dimension over. + const p = new IndexedDBStorageProvider({ dbName: 'db-a', prefix: 'p_' }); + const q = new IndexedDBStorageProvider({ dbName: 'db-a', prefix: 'q_' }); + + expect(p.backingStoreId).toBe(q.backingStoreId); + }); + + it('still encodes the database name rather than concatenating it raw', () => { + // Narrower than before (there is one field now), but a dbName carrying the + // scheme delimiter must not be able to impersonate another store. + const odd = new IndexedDBStorageProvider({ dbName: 'a:b' }); + const plain = new IndexedDBStorageProvider({ dbName: 'a' }); + expect(odd.backingStoreId).not.toBe(plain.backingStoreId); + expect(odd.backingStoreId).not.toContain('a:b'); + }); +}); + +describe('FileStorageProvider.backingStoreId — aliases of one file', () => { + it('is equal through a symlinked directory and the real one', () => { + // path.resolve is LEXICAL: it leaves a symlink alias and the real path as + // different strings for ONE wallet.json. Split, Sphere.clear() through one + // alias misses the Sphere registered through the other — wiping its file + // and leaving it isReady over the remains. + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'bsid-')); + const real = path.join(root, 'real'); + const link = path.join(root, 'link'); + fs.mkdirSync(real); + try { + fs.symlinkSync(real, link, 'dir'); + } catch { + return; // no symlink privilege (Windows CI) — nothing to assert + } + + const viaReal = new FileStorageProvider({ dataDir: real }); + const viaLink = new FileStorageProvider({ dataDir: link }); + + expect(viaReal.backingStoreId).toBe(viaLink.backingStoreId); + fs.rmSync(root, { recursive: true, force: true }); + }); + + it('still separates genuinely different directories', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'bsid-')); + const a = new FileStorageProvider({ dataDir: path.join(root, 'a') }); + const b = new FileStorageProvider({ dataDir: path.join(root, 'b') }); + + expect(a.backingStoreId).not.toBe(b.backingStoreId); + fs.rmSync(root, { recursive: true, force: true }); + }); + + it('agrees before the directory exists and after it is created', () => { + // The canonicalisation walks up to the deepest EXISTING ancestor, so a + // provider built against a not-yet-created dataDir must not disagree with + // one built after connect() made it. + const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'bsid-'))); + const dir = path.join(root, 'not-yet'); + const before = new FileStorageProvider({ dataDir: dir }); + fs.mkdirSync(dir); + const after = new FileStorageProvider({ dataDir: dir }); + + expect(before.backingStoreId).toBe(after.backingStoreId); + fs.rmSync(root, { recursive: true, force: true }); }); }); From ddd56c251ee929a75ff7f84878cca9b2c6030073 Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 17:54:20 +0200 Subject: [PATCH 42/43] fix(storage): the widened bucket is for erasure, not for every decision (Codex round 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I asked Codex to challenge the trade in f6e1a847 — widening the IndexedDB backing-store id to the whole database — and it found both places the widening overreached. Both are consequences of that commit, not of the original bug. 1. Sphere.import() decided `needsClear` on the liveness bucket. That bucket now names the unit of ERASURE, which for IndexedDB is the whole database, so it is deliberately WIDER than the EXISTENCE scope: a sibling prefix's live Sphere lands in it. Importing into an UNUSED prefix therefore called the unprefixed clear(), wiped the database and destroyed a live wallet under another prefix — even though Sphere.exists() on the target was false and the import could simply have written into the empty prefix. The decision is now the target's own wallet, plus a `_storage ===` identity check that keeps the case the bucket was covering: a Sphere built on this very provider object whose data may already be gone. Erasure keeps the wide bucket, which is right — when a clear does happen, everything it destroys must go down with it. 2. canonicalPath() resolved the FINAL component too. A directory symlink is a true alias, but a file-name symlink is not: save() writes `${filePath}.tmp` and renames it OVER filePath, which REPLACES the link rather than writing through it. Two such paths diverge on the first save, so sharing a bucket would have let a clear through the link destroy the target's Sphere while the target file stayed intact. The directory is canonicalised and the final entry kept. The sibling-prefix case needed a real IndexedDB to express — it is the only shape where the bucket is wider than existence — so the scoping suite gains a fake-indexeddb case. Both fixes falsified by restoring the previous expression; both probed; the two file-provider probes whose find strings moved were re-pointed rather than dropped. --- core/Sphere.ts | 10 ++-- impl/nodejs/storage/FileStorageProvider.ts | 8 +++- .../sphere-instance-scoping.test.ts | 47 +++++++++++++++++++ tests/mutation/probes.json | 28 +++++++++-- tests/unit/impl/backing-store-id.test.ts | 22 +++++++++ 5 files changed, 105 insertions(+), 10 deletions(-) diff --git a/core/Sphere.ts b/core/Sphere.ts index db8fe576..abc42e6e 100644 --- a/core/Sphere.ts +++ b/core/Sphere.ts @@ -1136,10 +1136,12 @@ export class Sphere { logger.debug('Sphere', 'Starting import...'); - // Clear existing wallet if any. Skip if no active instance and wallet - // doesn't exist — avoids a redundant IndexedDB delete/reopen that can race - // with a subsequent initialize(). - const needsClear = Sphere.liveOn(options.storage).length > 0 || (await Sphere.exists(options.storage)); + // Clear THIS storage's wallet if it has one — not the liveness bucket's, which + // names the unit of ERASURE (an IndexedDB clear() empties the whole database). + // A sibling prefix's live Sphere lands in that bucket, so deciding on it made + // an import into an UNUSED prefix wipe a wallet nobody asked to touch. + const liveHere = Sphere.liveOn(options.storage).some((s) => s._storage === options.storage); + const needsClear = liveHere || (await Sphere.exists(options.storage)); if (needsClear) { progress?.({ step: 'clearing', message: 'Clearing previous wallet data...' }); logger.debug('Sphere', 'Clearing existing wallet data...'); diff --git a/impl/nodejs/storage/FileStorageProvider.ts b/impl/nodejs/storage/FileStorageProvider.ts index 0e112bdd..09585fad 100644 --- a/impl/nodejs/storage/FileStorageProvider.ts +++ b/impl/nodejs/storage/FileStorageProvider.ts @@ -87,8 +87,12 @@ export class FileStorageProvider implements StorageProvider { this.network = config.network; } this.isTxtMode = this.filePath.endsWith('.txt'); - // Canonical, not merely resolved: aliases of one file must share an id. - this.backingStoreId = `file:${canonicalPath(this.filePath)}`; + // Canonicalise the DIRECTORY, keep the final entry: a directory symlink is a + // true alias, but save() renames a .tmp OVER filePath, which REPLACES a + // final-component symlink rather than writing through it — so those two + // paths diverge on the first save and must not share a bucket. + this.backingStoreId = + `file:${path.join(canonicalPath(this.dataDir), path.basename(this.filePath))}`; } setIdentity(identity: FullIdentity): void { diff --git a/tests/integration/sphere-instance-scoping.test.ts b/tests/integration/sphere-instance-scoping.test.ts index c17e6a7d..cde2ae65 100644 --- a/tests/integration/sphere-instance-scoping.test.ts +++ b/tests/integration/sphere-instance-scoping.test.ts @@ -25,9 +25,11 @@ import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import 'fake-indexeddb/auto'; import { Sphere } from '../../core/Sphere'; import { FileStorageProvider } from '../../impl/nodejs/storage/FileStorageProvider'; +import { IndexedDBStorageProvider } from '../../impl/browser/storage/IndexedDBStorageProvider'; import type { TransportProvider } from '../../transport'; import type { OracleProvider } from '../../oracle'; import type { StorageProvider } from '../../storage'; @@ -621,4 +623,49 @@ describe('the legacy-import entry points return the Sphere on the SUPPLIED stora b.sphere = result.sphere; expect(result.sphere!.identity!.chainPubkey).toBe(backup.chainPubkey); }); + + it('import() into an UNUSED prefix does not wipe a live wallet sharing the database', async () => { + // backingStoreId names the unit of ERASURE, so an IndexedDB database is ONE + // bucket however many prefixes it holds — clear() empties the whole object + // store. That makes the bucket wider than the EXISTENCE scope, and deciding + // "does this import need to clear?" on the bucket destroyed a live wallet + // under a sibling prefix that nobody asked to touch. + const dbName = `scope-idb-${Date.now()}`; + const liveStorage = new IndexedDBStorageProvider({ dbName, prefix: 'q_' }); + const targetStorage = new IndexedDBStorageProvider({ dbName, prefix: 'p_' }); + const liveWorld = makePv2World(NET); + const targetWorld = makePv2World(NET); + + const { sphere: liveSphere } = await Sphere.init({ + storage: liveStorage, + transport: createMockTransport(), + oracle: createEngineOracle(), + walletApi: liveWorld.walletApi, + network: NET, + mnemonic: MNEMONIC_A, + }); + + // Same database, so ONE bucket — and the target prefix holds no wallet. + expect(targetStorage.backingStoreId).toBe(liveStorage.backingStoreId); + expect(await Sphere.exists(targetStorage)).toBe(false); + + const imported = await Sphere.import({ + storage: targetStorage, + transport: createMockTransport(), + oracle: createEngineOracle(), + walletApi: targetWorld.walletApi, + network: NET, + mnemonic: MNEMONIC_C, + }); + + // The untouched wallet is still live AND still has its data. + expect(liveSphere.isReady).toBe(true); + expect(liveSphere.identity?.chainPubkey).toBeDefined(); + expect(await Sphere.exists(liveStorage)).toBe(true); + + await imported.destroy(); + await liveSphere.destroy(); + await targetStorage.disconnect(); + await liveStorage.disconnect(); + }, 30_000); }); diff --git a/tests/mutation/probes.json b/tests/mutation/probes.json index 1c8e4079..6e28da6c 100644 --- a/tests/mutation/probes.json +++ b/tests/mutation/probes.json @@ -835,8 +835,8 @@ "name": "file-backing-store-id-is-a-class-constant", "note": "#766 review: backingStoreId identifies the STORE. A constant (what `id` is) makes every FileStorageProvider in the process one store, so clear() on one wallet tears down every live Sphere - the exact cross-wallet kill #766 removed.", "file": "impl/nodejs/storage/FileStorageProvider.ts", - "find": " this.backingStoreId = `file:${canonicalPath(this.filePath)}`;", - "replace": " this.backingStoreId = 'file:'; // mutant: a class constant, every wallet collides", + "find": " `file:${path.join(canonicalPath(this.dataDir), path.basename(this.filePath))}`;", + "replace": " 'file:'; // mutant: a class constant, every wallet collides", "tests": [ "tests/unit/impl/backing-store-id.test.ts", "tests/integration/sphere-instance-scoping.test.ts" @@ -1189,8 +1189,28 @@ "name": "file-store-id-not-canonical", "note": "#772 codex P2: path.resolve is lexical, so a symlinked dataDir and the real one give different ids for ONE wallet.json \u2014 clear() through one alias then misses the Sphere registered through the other", "file": "impl/nodejs/storage/FileStorageProvider.ts", - "find": " this.backingStoreId = `file:${canonicalPath(this.filePath)}`;", - "replace": " this.backingStoreId = `file:${this.filePath}`;", + "find": " `file:${path.join(canonicalPath(this.dataDir), path.basename(this.filePath))}`;", + "replace": " `file:${this.filePath}`;", + "tests": [ + "tests/unit/impl/backing-store-id.test.ts" + ] + }, + { + "name": "import-clears-on-the-widened-bucket", + "note": "#772 codex round 2: the liveness bucket names the unit of ERASURE and is therefore wider than the EXISTENCE scope. Deciding needsClear on it made an import into an unused IndexedDB prefix wipe the whole database and destroy a live wallet under a sibling prefix", + "file": "core/Sphere.ts", + "find": " const liveHere = Sphere.liveOn(options.storage).some((s) => s._storage === options.storage);", + "replace": " const liveHere = Sphere.liveOn(options.storage).length > 0;", + "tests": [ + "tests/integration/sphere-instance-scoping.test.ts" + ] + }, + { + "name": "file-store-id-resolves-the-final-symlink", + "note": "#772 codex round 2: save() renames a .tmp OVER filePath, REPLACING a final-component symlink rather than writing through it \u2014 so two paths differing only in that link diverge on the first save and must not share a bucket", + "file": "impl/nodejs/storage/FileStorageProvider.ts", + "find": " `file:${path.join(canonicalPath(this.dataDir), path.basename(this.filePath))}`;", + "replace": " `file:${canonicalPath(this.filePath)}`;", "tests": [ "tests/unit/impl/backing-store-id.test.ts" ] diff --git a/tests/unit/impl/backing-store-id.test.ts b/tests/unit/impl/backing-store-id.test.ts index 8c5dbb2d..a6c18d1b 100644 --- a/tests/unit/impl/backing-store-id.test.ts +++ b/tests/unit/impl/backing-store-id.test.ts @@ -124,6 +124,28 @@ describe('FileStorageProvider.backingStoreId — aliases of one file', () => { fs.rmSync(root, { recursive: true, force: true }); }); + it('SEPARATES two paths differing only in a final-component symlink', () => { + // A directory symlink is a true alias; the FILE NAME is not. save() writes + // `${filePath}.tmp` and renames it OVER filePath, which replaces a + // final-component symlink rather than writing through it — so the two paths + // diverge on the first save and must not share a lifecycle bucket. Clearing + // through the link would otherwise destroy the target's Sphere while the + // target file stayed intact. + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'bsid-')); + fs.writeFileSync(path.join(root, 'real.json'), '{}'); + try { + fs.symlinkSync(path.join(root, 'real.json'), path.join(root, 'link.json')); + } catch { + return; // no symlink privilege + } + + const viaReal = new FileStorageProvider({ dataDir: root, fileName: 'real.json' }); + const viaLink = new FileStorageProvider({ dataDir: root, fileName: 'link.json' }); + + expect(viaReal.backingStoreId).not.toBe(viaLink.backingStoreId); + fs.rmSync(root, { recursive: true, force: true }); + }); + it('still separates genuinely different directories', () => { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'bsid-')); const a = new FileStorageProvider({ dataDir: path.join(root, 'a') }); From e321c4eec616e09bb9b55f995dbb5941b56e3fcd Mon Sep 17 00:00:00 2001 From: Pavel Grigorenko Date: Thu, 3 Sep 2026 22:14:32 +0200 Subject: [PATCH 43/43] docs(changelog): cut the 0.16.0 section Everything under Unreleased ships as 0.16.0: the lifecycle globals removed (#766), the teardown fixes (#770 items 1-6), the cross-bundle identity cell, the mainnet network, the 'dev' network removed, the health probe, the Connect network-change revoke (#774), and the backingStoreId corrections from the Codex rounds. Breaking, so a minor bump on 0.x. --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f5449e6..9a0c3913 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.16.0] - 2026-09-03 + ### Removed (BREAKING) — the Sphere lifecycle globals (#766) `Sphere.getInstance()`, `Sphere.isInitialized()` and the root export `getSphere` are gone @@ -1456,6 +1458,7 @@ consumed exclusively through the `token-engine/` port. Consequences: version tags past v0.9.x, so a tag-compare link would 404. --> [Unreleased]: https://github.com/unicity-sphere/sphere-sdk/compare/main...HEAD +[0.16.0]: https://www.npmjs.com/package/@unicitylabs/sphere-sdk/v/0.16.0 [0.15.0]: https://www.npmjs.com/package/@unicitylabs/sphere-sdk/v/0.15.0 [0.14.11]: https://www.npmjs.com/package/@unicitylabs/sphere-sdk/v/0.14.11 [0.14.10]: https://www.npmjs.com/package/@unicitylabs/sphere-sdk/v/0.14.10