Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
645e048
Merge master (v0.23.0 release) back into develop
jdogresorg Oct 12, 2026
d9cd72c
Fix 9 review findings in xchain-decoder
jdogresorg Oct 12, 2026
0a0ad44
Collapse dispenser registrations by address id
jdogresorg Oct 12, 2026
eae5f23
Preserve pinned dispenser test titles
jdogresorg Oct 13, 2026
2aee2ae
Strengthen dispenser address resolver coverage
jdogresorg Oct 13, 2026
0f4506f
Merge: Strengthen dispenser address resolver coverage
jdogresorg Oct 13, 2026
777f0b9
Add stubbed getRawTransaction whole-hex guard test
jdogresorg Oct 13, 2026
5fbd783
Merge: Add stubbed getRawTransaction whole-hex guard test
jdogresorg Oct 13, 2026
8277bc7
Bump ip-address, axios and proxy-addr past their advisories
jdogresorg Oct 13, 2026
4202032
Route contributions to develop and guard master against non-release p…
jdogresorg Oct 13, 2026
81018f3
Add GET /ready and report readiness on /live
jdogresorg Oct 14, 2026
fb69872
Settle a lower node tip with two agreeing readings before rolling back
jdogresorg Oct 14, 2026
6cd4dc6
Merge remote-tracking branch 'origin/develop' into settle-tip-before-…
jdogresorg Oct 14, 2026
60ced95
Fix review round findings for xchain-decoder (2026-10-14)
jdogresorg Oct 14, 2026
dda336d
Refuse a stripped or plain block whose transactions do not fill it or…
jdogresorg Oct 15, 2026
6acecdb
Refuse unsafe MWEB CompactSize values and trailing transaction bytes …
jdogresorg Oct 15, 2026
5862f76
Re-vendor coin registry for the mainnet genesis re-pin
jdogresorg Oct 17, 2026
ce80716
Add dispenser parent-floor activation constant
jdogresorg Oct 17, 2026
6e4fe37
Merge: Add dispenser parent-floor activation constant
jdogresorg Oct 17, 2026
b730225
Anchor dispenser grace to parent protocol time
jdogresorg Oct 17, 2026
411e5f7
Keep dispenser grace tests within structure limits
jdogresorg Oct 17, 2026
27c772b
Tighten dispenser grace documentation
jdogresorg Oct 17, 2026
0d549d1
Preserve dispenser grace suite titles
jdogresorg Oct 17, 2026
1c3c3c1
Bound current block MTP history
jdogresorg Oct 17, 2026
4c701d3
Stabilize parent-floor regression fixture
jdogresorg Oct 17, 2026
e3fe948
Merge: Stabilize parent-floor regression fixture
jdogresorg Oct 17, 2026
c6c4139
Merge master into develop to level it before the v0.24.0 cut
jdogresorg Oct 17, 2026
a91a7d9
Release v0.24.0
jdogresorg Oct 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ DECODER_API_PORT=3002
# Optional: API rate limit in requests per minute per IP (default 100).
# DECODER_RATE_LIMIT_RPM=100

# Optional: enable AuxPoW (merged-mining) block-header parsing, required for
# Dogecoin-family chains. Set to 1/true to enable.
# Legacy, ignored: AuxPoW (merged-mining) block-header parsing is selected from
# NETWORK (on for dogecoin-*, off otherwise). Kept for backward compatibility.
# AUX_POW=

# Optional: coin node RPC timeout in ms (default 30000).
Expand Down
4 changes: 4 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
<!-- Thanks for contributing to XChain Platform. -->

## Base branch

- [ ] This pull request targets `develop`. `master` only takes release merges, and a pull request into it from a fork cannot be merged.

## What this changes

<!-- One or two sentences on what changed and why. Link an issue if there is one. -->
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
# so the merge commit a release tag points at has its own green run.
branches: [master, develop]
pull_request:
branches: [master]
branches: [master, develop]

jobs:
ci:
Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.24.0] - 2026-10-17

### Added
- Added a ready endpoint and reported readiness on the live endpoint.
- Added the dispenser parent-floor activation that anchors dispenser grace to the parent protocol time.

### Fixed
- Refused a block whose transactions do not fill it or match its merkle root, and parsed LTC MWEB-form transactions.
- Refused unsafe MWEB CompactSize values and trailing transaction bytes with a typed parse error.
- Settled a lower node tip with two agreeing readings before rolling back, and bounded the current block MTP history.

### Security
- Bumped ip-address, axios and proxy-addr past their advisories.

## [0.23.0] - 2026-10-11

### Added
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ this service's entire input surface is attacker-controlled.

Match the existing log style: a concise subject line, then a short body explaining what changed and why.

- Branch off `master` and keep history linear (rebase, don't merge).
- Fork the repository, branch off `develop`, and open your pull request against `develop`. Keep history linear (rebase, don't merge). `master` is what production runs and only ever receives release merges from `develop`; a pull request into `master` from a fork or any other branch fails the `base-branch guard` check and cannot be merged.
- One logical change per commit; don't batch unrelated work.
- **No `Co-Authored-By` trailers.** This is a project policy.
- **Never `--no-verify`.** If a hook fails, fix the cause; don't bypass it.
Expand Down
4 changes: 2 additions & 2 deletions bin/pins/identity.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
"repo": "xchain-decoder",
"what": "sha256 of every file this repo vendors from a canonical it does not own",
"coins": {
"src/coins/BTC.js": "182baf5173e3846b94a5edc1ae15913a54fb266addfc45a3cf9a013bd9ec16c6",
"src/coins/DOGE.js": "4659b45fb7a5c959ce2777ab9d5e7c193f3515f03cdd8cd23bb6b9c4d012fc11",
"src/coins/BTC.js": "6868afc2d18911ba112234c980c094fda71cfc3e744a00f493def781c7610f8d",
"src/coins/DOGE.js": "29ac02299b0c779f4e3eda579eeaa9f673c454b73cfe9978a35f03f0e9aed33c",
"src/coins/LTC.js": "6155195bfa5fdfeab77dec3802ce67b5cc47a77695e81a4a5545d980e4c421dd",
"src/coins/consensus_pin.js": "342bf7af224a19ea0a98ff9cfb2c9a70b3fd5be75de7ecef2db05936f82baa63",
"src/coins/index.js": "08b678d934fa647d3c7d14dc5ccdaeb86722ee32a3662de69e4a31cd7b09234c"
Expand Down
3 changes: 3 additions & 0 deletions bin/pins/suite-title-renames.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
"test/unit/db_queries.test/04_delete_block_by_index.test.js": {
"Database#deleteBlockByIndex() executes 4 DELETE queries and returns true on success": "Database#deleteBlockByIndex() executes 5 DELETE queries and returns true on success"
},
"test/unit/db_queries.test/20_error_path_transaction_connection_branches.test.js": {
"Database error-path transactionConnection branches createAddress: swallows INSERT error even with active transactionConnection": "Database error-path transactionConnection branches createAddress: rethrows an INSERT error inside an active transaction so the block rolls back"
},
"test/unit/fee_destination.test.js": {
"resolveFeeDestination @unit honors an env override on regtest ONLY": "resolveFeeDestination @unit ignores a bare env override on regtest too, so capture and indexer validation resolve one address"
},
Expand Down
36 changes: 20 additions & 16 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "xchain-decoder",
"description": "xchain-decoder decodes XChain platform transactions from a given blockchain and populates a database with the decoded data.",
"version": "0.23.0",
"version": "0.24.0",
"license": "AGPL-3.0-or-later",
"repository": {
"type": "git",
Expand All @@ -20,7 +20,7 @@
"express-json-rpc-router": "^1.4.0",
"express-rate-limit": "^8.5.2",
"helmet": "^8.2.0",
"mariadb": "3.5.3",
"mariadb": "3.5.4",
"tiny-secp256k1": "2.2.4"
},
"scripts": {
Expand Down Expand Up @@ -93,7 +93,7 @@
"diff": "^8.0.3",
"tmp": "^0.2.7",
"fast-uri": "^3.1.5",
"ip-address": "^10.3.1",
"ip-address": "^10.5.1",
"tough-cookie": "^6.0.2",
"uuid": "^11.1.1",
"request": {
Expand Down
4 changes: 4 additions & 0 deletions src/XChainDecoder.js
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,10 @@ function initializeDecoderMode(decoder) {

decoder.rpcErrors = 0
decoder.parseErrors = 0
decoder.quarantinedTxs = 0
decoder.lastQuarantinedBlock = null
decoder.lastQuarantinedTxHash = null
decoder.lastQuarantinedAt = null
decoder.dispenserPurgeFailures = 0
}

Expand Down
47 changes: 33 additions & 14 deletions src/XChainDecoder/block_store.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,21 @@

const { format: formatLogLine } = require('node:util')
const { isDispenserExpiryRealignActive } = require('../protocol/dispenser_expiry_realign')
const { cancelGraceFloor, purgeGraceFloor } = require('../protocol/dispenser_cancel_grace')
const {
cancelGraceFloor,
isDispenserCancelGraceParentFloorActive,
purgeGraceFloor,
} = require('../protocol/dispenser_cancel_grace')
const protocolTime = require('../protocol/protocol_time')
const { logger, SYNCED_THRESHOLD, DB_TRANSACTION_BLOCKS_QUANTITY, LOG_BLOCK_INTERVAL, DISPENSER_EXPIRE_SAFE_DEPTH } = require('./constants.js')
const { ingestTransaction, clearQuarantineState } = require('./transaction_ingest.js')

async function loadOpenDispenserAddresses(block, nextBlockHeight){
async function loadOpenDispenserAddresses(block, nextBlockHeight, parentProtocolTime){
// Load the set once per block after any legacy expiry and before the
// realigned end-of-block expiry. The grace floor preserves dispensers
// that the indexer still considers fillable during cancellation grace.
let openDispenserAddresses = await this.db.getAllOpenDispenserAddresses(
cancelGraceFloor(this.consensusNetwork, block.timestamp))
cancelGraceFloor(this.consensusNetwork, block.timestamp, parentProtocolTime))
if (openDispenserAddresses == null){
logger.error(`Could not load open dispenser addresses for block ${nextBlockHeight}; retrying block`)
await this.db.endTransaction()
Expand Down Expand Up @@ -129,13 +133,11 @@ async function finishBlock(loop, block, nextBlockHeight, nextBlockHash, openDisp
this.lastAdvanceAt = Date.now()
}

async function fetchPreviousBlockTimes(nextBlockHeight, span){
// Walk strictly backward so the result is newest first by block height and
// never wider than the median span, even when asked for more. A failed
async function fetchBlockTimes(nextBlockHeight, span){
// Walk strictly backward so the result is newest first by block height. A failed
// lookup returns no partial window, avoiding mixed reorg states.
const boundedSpan = Math.min(span, protocolTime.MEDIAN_TIME_SPAN)
let previousBlockTimes = []
for (let height = nextBlockHeight - 1; height >= 0 && previousBlockTimes.length < boundedSpan; height--){
for (let height = nextBlockHeight - 1; height >= 0 && previousBlockTimes.length < span; height--){
let previousBlock
try {
previousBlock = await this.db.getBlockByIndex(height)
Expand All @@ -149,18 +151,34 @@ async function fetchPreviousBlockTimes(nextBlockHeight, span){
return previousBlockTimes
}

async function fetchPreviousBlockTimes(nextBlockHeight, span){
return fetchBlockTimes.call(
this, nextBlockHeight, Math.min(span, protocolTime.MEDIAN_TIME_SPAN))
}

async function resolveBlockTimeContext(block, nextBlockHeight){
// MTP networks resolve against the previous MEDIAN_TIME_SPAN blocks.
const mtpActive = protocolTime.isProtocolTimeMtpActive(this.consensusNetwork)
const parentFloorActive = isDispenserCancelGraceParentFloorActive(
this.consensusNetwork, block.timestamp)
const historySpan = mtpActive
? protocolTime.MEDIAN_TIME_SPAN + (parentFloorActive ? 1 : 0)
: (parentFloorActive ? 1 : 0)
let previousBlockTimes = []
if (protocolTime.isProtocolTimeMtpActive(this.consensusNetwork)){
previousBlockTimes = await fetchPreviousBlockTimes.call(this, nextBlockHeight, protocolTime.MEDIAN_TIME_SPAN)
if (historySpan > 0){
previousBlockTimes = await fetchBlockTimes.call(this, nextBlockHeight, historySpan)
if (previousBlockTimes === null){
await this.db.endTransaction()
return null
}
}
return protocolTime.createBlockTimeContext(
this.consensusNetwork, block.timestamp, previousBlockTimes)
const current = protocolTime.createBlockTimeContext(
this.consensusNetwork, block.timestamp,
previousBlockTimes.slice(0, protocolTime.MEDIAN_TIME_SPAN))
const parentProtocolTime = previousBlockTimes.length === 0
? null
: protocolTime.protocolTime(
this.consensusNetwork, previousBlockTimes[0], previousBlockTimes.slice(1))
return Object.freeze({ ...current, parentProtocolTime })
}

async function insertBlockRecord(blockTimeContext, nextBlockHeight, nextBlockHash, previousBlockHash){
Expand Down Expand Up @@ -203,7 +221,8 @@ async function storeBlock(loop, block, nextBlockHeight, nextBlockHash, previousB
if ((await expireDispensersAtBlockStart.call(
this, block, nextBlockHeight, expireAtEnd)) === 'rollback') return 'rollback'

const openAddresses = await loadOpenDispenserAddresses.call(this, block, nextBlockHeight)
const openAddresses = await loadOpenDispenserAddresses.call(
this, block, nextBlockHeight, blockTimeContext.parentProtocolTime)
if (openAddresses === 'rollback') return 'rollback'
return await finishBlock.call(
this, loop, block, nextBlockHeight, nextBlockHash, openAddresses, expireAtEnd)
Expand Down
28 changes: 26 additions & 2 deletions src/XChainDecoder/chain_integrity.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,24 @@
********************************************************************/

const { chainGenesisMismatch, chainGenesisUnpinned } = require('../protocol/chain_identity')
const { logger, REORG_HALT_PROBE_INTERVAL_MS, AUXPOW_REASSEMBLE_AFTER } = require('./constants.js')
const { logger, REORG_HALT_PROBE_INTERVAL_MS, AUXPOW_REASSEMBLE_AFTER, TIP_SETTLE_DELAY_MS, TIP_SETTLE_MAX_READS } = require('./constants.js')

// Read the durable REORG_HALT marker through whichever probe the db exposes, or
// null when it exposes none (or there is no db) and the cached state must stand.
// A probe that exists but answers with nothing throws instead, so checkReorgHalt
// logs the fault and keeps the cached state: an empty answer is never a clear.
async function readReorgHaltMarker(db){
if (!db) return null
if (typeof db.getReorgHaltMarker === 'function'){
return (await db.getReorgHaltMarker()) || { halted: false, at: null, reason: null }
const marker = await db.getReorgHaltMarker()
if (!marker || typeof marker !== 'object') throw new Error('getReorgHaltMarker returned no marker')
return marker
}
if (typeof db.isReorgHalted === 'function'){
// Older/minimal db shapes (and the mocks in the verifyReorg suites)
// expose only the boolean probe.
const halted = await db.isReorgHalted()
if (halted == null) throw new Error('isReorgHalted returned no answer')
return { halted: !!(halted && halted.halted !== undefined ? halted.halted : halted), at: null, reason: null }
}
return null
Expand Down Expand Up @@ -236,5 +241,24 @@ module.exports = {
// refresh the timestamp, so the refusal is re-proved on every retry.
if (!mismatch) this.chainGenesisCheckedAt = Date.now()
return mismatch
},

// Re-read with readOnce() until two readings TIP_SETTLE_DELAY_MS apart agree, starting
// from `first`, the reading that raised the question. readOnce() returns null when the
// node cannot be read or a gate refuses the reply. Returns the settled reading, or null
// when a re-read fails or the reading is still moving after TIP_SETTLE_MAX_READS, so
// the caller ends its pass with nothing deleted and the next pass reads afresh.
async settleNodeReading(first, readOnce, what){
let previous = first
for (let read = 0; read < TIP_SETTLE_MAX_READS; read++){
await this.sleep(TIP_SETTLE_DELAY_MS)
const next = await readOnce()
if (next === null) return null
if (next === previous) return next
previous = next
}
logger.warn('The ' + what + ' is still moving after ' + TIP_SETTLE_MAX_READS + ' readings (last ' + previous
+ '); not rolling back on a moving reading, re-checking next pass.')
return null
}
}
10 changes: 10 additions & 0 deletions src/XChainDecoder/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ const lenientTextDecoder = new TextDecoder('utf-8')
const CHECK_BLOCK_DELAY_MS = 1000 //1 second to continously ask for new block when all has been parsed
const BLOCKCHAIN_INFO_REFRESH_MS = 30000 //Re-poll the node tip at least this often during catch-up so reported lag stays accurate
const MEMPOOL_INTERVAL = 60000 //60 seconds between mempool checks
// A node tip below the stored tip, or a different block at the stored height, is acted
// on only once two readings TIP_SETTLE_DELAY_MS apart agree; a reading that keeps
// changing for TIP_SETTLE_MAX_READS re-reads ends the pass with nothing deleted. One
// stale reply (an RPC failover onto a lagging endpoint) otherwise rolls back valid
// blocks and writes REORG markers for a reorg that never happened. Same values as
// xchain-utxo-tracker, which settles its tip the same way before a rollback.
const TIP_SETTLE_DELAY_MS = 2000
const TIP_SETTLE_MAX_READS = 5
// How often a health surface may re-probe the durable REORG_HALT marker. The marker
// changes at most once in a decoder's life, so a slow TTL is ample; the point of the
// cache is that an unauthenticated health endpoint must not turn into one DB query
Expand Down Expand Up @@ -164,6 +172,8 @@ module.exports = {
CHECK_BLOCK_DELAY_MS,
BLOCKCHAIN_INFO_REFRESH_MS,
MEMPOOL_INTERVAL,
TIP_SETTLE_DELAY_MS,
TIP_SETTLE_MAX_READS,
REORG_HALT_PROBE_INTERVAL_MS,
REORG_HALT_PARK_TICK_MS,
STALL_ALERT_MS,
Expand Down
Loading
Loading