Harden BASM protocol validation and Merkle path offsets - #520
Merged
Merged
Conversation
…duling Subscribe to SLAP tracker updates during an active query so later advertised hosts can join a fair, concurrency-bounded queue. Read lookup bodies incrementally with honest resource-limit accounting, keep trackers as routing sources, and leave raw query$ unverified while exposing the C02 onEvidence intake seam. Preserves 2s/5s delays, reputation/backoff, query/query$/freeform APIs, and existing CORS/public lookup request headers.
Add replica-set schema bootstrap, content-addressed payload publication, transactionally guarded references/GC, and majority-commit outcome helpers behind an optional mongodb peer. This is not an AdmissionStorage adapter and does not change Engine or Knex defaults.
Preserve per-entry script results so a rejected sibling cannot poison a shared ancestor. LocalChainTracker tokens use participating sources only and fail closed on missing identity. Reset ownership is rechecked after dispose before destructive hooks. Built-in remote ChainTracks clients advertise reorg-event capability explicitly. Services.getChainTracker publishes one coalesced wrapper before yielding.
Validate untrusted BASM peer JSON, bind remote anchors to local canonical headers, and independently check admitted-list block indices before historical admission. Keep automatic BASM sync disabled and leave durable recovery jobs for B02. Include independent BRC-136 vector checks and localhost TS client tests against the Go five-method read/serving surface.
BHServiceClient no longer caches the queried merkle root. Validity is decided from a freshly read header, so a false result cannot invert on retry and a reorg cannot reuse a stale positive.
Store discovery bounds on the SLAP host cache so a tighter-limit query cannot freeze a truncated host set for a later broader attempt. Throw the historical no-competent-hosts error from query()/queryDetailed when a deadline expires before any host is admitted; query$ still emits the deadline snapshot.
Collection validators used lexicographic $lte on unpadded outputIndex strings, which rejected legal indexes such as "9". Bound uint32 fields with $toLong on outputs, consumption edges, GASP nodes, and SHIP/SLAP.
Verify claimed admitted-list block indices against the compound Merkle path even when every remote txid is already local. Check inclusion with the chain-tracker root at the claimed height instead of MerklePath.verify, which also enforces coinbase 100-block spendability.
LookupResolver is a BRC-100 JSON boundary file, so the in-flight discovery key cannot use JSON.stringify. stringifyBRC100 preserves the same service-plus-limit tuple without changing cache cover semantics.
Drop redundant optional-undefined on lookup evidence limits, extract parseEvidence and Chaintracks header-retry helpers below S3776, and rename coordinator catch params to error_.
Extract BASM reconcile helpers so cognitive complexity stays at or below 15, prefer optional chaining on nullable anchors, and invert the advertised response-length comparison.
…e-c02 # Conflicts: # docs/reference/package-api-migrations.md # governance/package-release-notes.json
…ardening # Conflicts: # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/package-release-notes.json # governance/repository-health/baselines.json
Extract query/session and host-tracking helpers so cognitive complexity stays within Sonar's limit, and clear the remaining overlay-tools findings without changing lookup behavior.
Brand StorageUint64, extract high-complexity helpers, and apply Sonar-preferred optional chaining, regex, clone, and sort APIs without changing payload identity or uint32 $toLong validators.
# Conflicts: # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/repository-health/baselines.json
S7746 on LookupResolver: prefer throw over return Promise.reject in the facilitator start chain.
discoverOverlayCertificates called getServices(), which throws on wallets constructed without services and broke BRC-100 discoverBy* conformance. Missing chain context now yields no identities instead of throwing.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Merge from main left baselines.json on 2.4.2 while package.json is 2.5.0.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Jest's istanbul graph still resolved 3.15.1, which fails pnpm audit --audit-level=high.
Health requires overrideRemovalReview.retainedCount to match the live registry.
Health requires overrideRemovalReview.retainedCount to match the live registry.
Health requires overrideRemovalReview.retainedCount to match the live registry.
…dles The ceilings carried on this branch were authored before main grew (#534, #547), so the merged wallet bundles exceeded them and CI stopped at the first over-budget dimension. Measured from the exact packed dependency graph at 83c0670 (JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio recorded in #547): client Vite 1,759,717 raw / 416,284 gzip / 324,287 brotli client esbuild 1,372,320 raw / 378,741 gzip / 302,937 brotli mobile Metro 1,811,581 raw / 461,355 gzip / 355,871 brotli mobile Hermes 3,688,871 raw / 1,502,972 gzip / 1,166,821 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Bounded discovery, the host queue and the streaming response reader grow every bundle that embeds @bsv/sdk. Measured from the exact packed dependency graph at the top of the ordered overlay wave (max of 5eef0fc and 9077394; JS bundle sizes verified byte-identical between macOS and hosted Linux; Hermes gzip estimated at the +1.3% Linux ratio from #547): @bsv/sdk UMD 591,705 raw / 168,531 gzip / 140,485 brotli @bsv/simple Vite 752,122 raw / 182,787 gzip / 150,699 brotli wallet client Vite 1,782,018 raw / 421,917 gzip / 328,724 brotli wallet client esbuild 1,389,928 raw / 383,620 gzip / 306,783 brotli wallet mobile Metro 1,834,234 raw / 466,804 gzip / 360,591 brotli wallet mobile Hermes 3,725,120 raw / 1,520,644 gzip / 1,181,384 brotli Only the dimensions that exceeded are raised, with about 0.25% headroom (1% on the Hermes gzip estimate, 0.5% on Hermes brotli run variance). Later branches in the wave move these bundles by under 150 bytes, so they inherit these ceilings unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GASP finalization already submits with historical-tx-no-spv; its historical-tx calls are dry-run admissibility checks. The restored guard protects public Engine.submit callers, so say that in the test title. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The repository's patch-coverage gate (scripts/patch-coverage.mjs, TEST_PATH pattern /(?:^|\/)__tests(?:__)?(?:\/|$)/) treats both `__tests/` and `__tests__/` directories as test code. This package's jest collectCoverageFrom only excluded `__tests__/**`, so the non-`.test.ts` support modules this PR adds under src/__tests/ (admission/ReferenceAdmissionStorage.ts, admission/AdmissionStorageContract.ts, mongo/MongoCommitResponseProxy.ts, mongo/MongoReplicaFixture.ts, ...) were instrumented and reported to Codecov as production code, understating the patch-coverage denominator correction the gate already applies elsewhere. Add '!src/**/__tests/**' so this package's own coverage collection matches the gate's definition of test code. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Move the fake-peer fixture out of BASMReconciliation.test.ts so further fail-closed suites can drive the same engine, storage, chain tracker and fetch mock without duplicating them. No test behaviour changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each BASM read endpoint fails closed with a TypeError carrying code 'BASM_UNSUPPORTED' when the storage backend does not implement the optional method behind it, and the capability check runs before the argument-shape guard on compound Merkle paths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Exercises reconcileBASMWithPeer's not-ahead branch end to end: two empty topics match without a range request, an equal matching tip is only accepted after its local anchor is re-verified against the canonical header, a higher local tip reports divergence, and a tip whose anchor is missing, unsupported, TAC-mismatched or non-canonical fails closed. Also pins the report surface: "matched" once the page catches the local tip up, a stringified non-Error failure with no invented errorCode, and BASM_UNSUPPORTED surfacing through startBASMSync. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds the canonical-position cases the reconciler fails closed on: a phantom duplicate at the right edge of an odd-width block is accepted but the same node is rejected once the canonical width is even, a BUMP that omits an admitted leaf is rejected before any raw transaction is fetched, a non-canonical root is rejected by the chain tracker, canonical-count assurance is recorded even when nothing needs fetching, and an anchor that vanishes from the peer blocks admission. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Drives a multi-block fake peer to pin the anchor-page contract: a page truncated short of its requested target and a page that skips the height after the local tip are both rejected, a peer many pages ahead is followed one capped page at a time without re-downloading anchors that already match locally, and a page mixing canonical-count and encoded-offset blocks reports the weaker assurance for the whole attempt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Cover the cancellation and resource-limit branches of HTTPSOverlayLookupFacilitator: a pre-aborted caller signal short-circuits before any fetch, an abort racing a non-ok response reports cancellation and drains the body, JSON and octet-stream output counts are rejected above the caller's budget (including a negative varint count), atomic BEEF extraction stops once the extracted bytes outrun the response budget, and an in-flight decode observes a mid-stream cancellation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…unds Exercise the uncovered fail-closed branches of LookupResolver.query$: aggregation stopping at maxOutputs across hosts, the evidenceLimits shorthand defaulting its byte budget, the bounded candidate scan (scan window, malformed entries, maxHosts), per-source quotas for additional hosts, SLAP discovery refusing bytes that would breach maxTotalBytes, byte reports arriving after cancellation, cancellation dropping an in-flight peer answer and skipping queued hosts, idempotent double cancellation, the concurrent-query ceiling, over-budget answers being dropped without blaming the host, the maxTrackers budget, advertisement-map eviction, and a broader host cache surviving a tighter rediscovery. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds real jest tests for previously-uncovered MongoPayloadStore branches: pin re-add no-op, claimGarbage/finishGarbage edge cases (never-published digest, afterDeleteClaim hook, ownership-metadata drift), lease-stolen fencing before the ready CAS and before GridFS staging, wrong explicit txid on both the fresh-upload and already-ready paths, GridFS crash-boundary hooks, abandonUpload's guard against deleting a concurrently-published file, digest-mismatch-after-spillover cleanup, stale inline-upload recovery, an expired crashed reservation reclaiming and retiring its orphaned GridFS file, and the abort-signal fallback messages used when a signal's `reason` is unset. Also stresses a many-way concurrent first-time publish of the same digest and a corrupted-in-place staged GridFS chunk to exercise the duplicate-key race and post-upload verification failure paths. Uncovered lines for this file drop from 57 to 21 (lcov, line+branch). Remaining lines are either dead code given existing invariants (the per-chunk maxPayloadBytes check, verifyGridFs's non-byte-chunk guard, the "stream writer missing" guard) or depend on timing-sensitive races with no exposed test seam (recoverStaleUpload's current-state guard, the exact instant a GridFS write signals backpressorem). A test also surfaced a real defect, reported separately: finishGarbage's FileNotFound-swallow regex never matches the mongodb driver's actual "File not found for id X" message, so a benign already-deleted-file race during cleanup always rethrows instead of being swallowed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dges Adds real jest tests for previously-uncovered MongoTransactionRunner branches: many concurrent fresh claims on the same operation id (forces the duplicate-key insert race and the "lost the claim" observer path), a claim-row insert failure whose error code is not a duplicate key (must propagate, not be swallowed), reconcile() with no attemptId against a never-created operation (the "unlocated" fallback), reconcile() with a mismatched attemptId against a genuinely pending claim, and commit failures whose deadline elapses mid-retry-loop (the budget-aborted break, verified via a real blocked commitTransaction). Uncovered lines for this file drop from 18 to 9 (lcov, line+branch). Remaining lines depend on races or invariants not reachable from the public API: Budget's manual deadline check is shadowed by the earlier throwIfAborted() once the matching timer fires; abort()'s "already out of transaction" branch has no path that reaches it before the catch that calls it; the post-insert re-read uses a majority/primary read right after an acknowledged write, so it can't observe a miss without breaking the replica set's own consistency guarantee; and the ownership-fencing mismatches and the "operation row disappeared" branches require deleting a submission-operation row, which nothing in this class ever does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds real jest tests for previously-uncovered MongoSchema branches: concurrent bootstraps of a fresh database (races the NamespaceExists and duplicate-ledger swallow paths in createCollectionIfMissing, ensureGridFs, and ensureLedger), a concurrent re-bootstrap after dropping just the GridFS bucket collections, an existing collection whose validator matches but omits an explicit collation (the collation-compatible-by-omission branch), an index-creation failure whose error code is not one of the benign conflict codes, and a ledger insert failure whose error code is not a duplicate key (both must propagate rather than being swallowed). Uncovered lines for this file drop from 11 to 10 deterministically; the concurrent-bootstrap races additionally land on the NamespaceExists swallow branches in some runs but not every run, since they depend on genuine scheduling of concurrent createCollection calls. The remaining lines are dead code given the surrounding invariants (decodeMongoUint64's re-encode check, the unused `string()` default length and `allowAdditional` option on module-private helpers no call site ever exercises) or require racing a second writer against transactionalProbe's own read-then-CAS window, which was not attempted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # packages/sdk/browser-budget.json # packages/sdk/docs/reference/overlay-tools.md # packages/sdk/src/overlay-tools/LookupResolver.ts # packages/wallet/wallet-toolbox/client/platform-budget.json # packages/wallet/wallet-toolbox/mobile/platform-budget.json
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…Garbage finishGarbage is documented as safe to retry, but it only swallowed errors matching /FileNotFound/. The MongoDB driver reports a missing file as MongoRuntimeError "File not found for id <id>", which never matched, so a finisher that lost the race to a concurrent retry rethrew and left the payload row in 'deleting'. Match the driver's actual message. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Quaakee
added a commit
to Quaakee/ts-stack
that referenced
this pull request
Sep 19, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # docs/packages/overlays/overlay-discovery-services.md # docs/packages/overlays/overlay-express.md # docs/packages/overlays/overlay-topics.md # docs/packages/overlays/overlay.md # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/package-release-notes.json # governance/repository-health/baselines.json # packages/overlays/overlay-discovery-services/CHANGELOG.md # packages/overlays/overlay-discovery-services/package.json # packages/overlays/overlay-express/CHANGELOG.md # packages/overlays/overlay-express/package.json # packages/overlays/overlay/CHANGELOG.md # packages/overlays/overlay/package.json # packages/overlays/topics/CHANGELOG.md # packages/overlays/topics/package.json
BRC-136 defines an empty topic at a height as k = 0 with R = 32 zero
bytes, and rebuildTopicAnchorChain anchors every height that way. The
reconciler accepted such an anchor and then requested a compound Merkle
path for an empty txid list, which BASMRemote rejects ("At least one
BASM proof txid is required"). Any node behind a peer aborted at the
first height the topic admitted nothing, which is most heights.
After the root and count check, an empty admitted list has nothing to
bind or fetch: count the height as checked and skip the proof and raw
transaction round trip. It diverges only when this node admitted
transactions at that height. A zero-count anchor with a non-zero root is
still rejected.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
sirdeggen
added a commit
that referenced
this pull request
Sep 19, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> # Conflicts: # docs/packages/overlays/overlay-discovery-services.md # docs/packages/overlays/overlay-express.md # docs/packages/overlays/overlay-topics.md # docs/packages/overlays/overlay.md # docs/reference/package-api-migrations.md # docs/reference/stack-facts.md # governance/package-release-notes.json # governance/repository-health/baselines.json # packages/overlays/overlay-discovery-services/package.json # packages/overlays/overlay-express/package.json # packages/overlays/overlay/CHANGELOG.md # packages/overlays/overlay/package.json # packages/overlays/topics/package.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overlay reliability program
This change is one slice of Make Overlays Great Again.
Do not merge until the stacked TypeScript and Go overlay branches have been proven together locally. Isolated CI green is not integration acceptance. Keep this PR draft. No npm publication, live database migration, or Mongo/BASM default activation.
C05 (verified progressive API) is parked. C02 (ts-stack#517) and C04 (ts-stack#518) both edit
LookupResolver.tsand must not merge independently.Program and scope
752464dff9d91ee4d0ee4abdd1ad6997a4141912Impact
Affected packages:
@bsv/sdk(MerklePath),@bsv/overlay,@bsv/overlay-expressVerification
Security and dependencies
Release and operations
enableBASMSyncremains false