Context
Chain-derived TAO/rao amounts (stake, emission, transfer/stake-swap amounts) are stored as SQLite REAL (IEEE-754 double) across several D1 tables. A double is only exact up to 2^53 - 1, which at 1 TAO = 1e9 rao corresponds to ~9,007,199 TAO. Values above that lose low-order-bit precision silently.
A full audit (2026-07-03) found this is two distinct problems, not one:
- Ingestion-time loss (the worse one). Several Python ingestion scripts call the Bittensor SDK's
Balance.__float__(), which itself performs the rao→TAO division in double precision — the exact rao integer is discarded before this codebase's own code ever runs. This is retroactively unfixable for already-written rows without re-querying the chain at the historical block height.
- Explicit-division loss (fixable going forward). Two scripts (
fetch-events.py, backfill-neuron-history.py) hold the exact SCALE-decoded integer one line before dividing by 1e9 — an easy fix for new data, though existing rows are still already-lossy.
Critically: this codebase already has one proven, merged fix for this exact bug class — src/account-balance.mjs's toRao() helper (issue #2070), which sums in BigInt rao-space and only converts to a JS Number after the final division. But that pattern cannot fix this issue — #2070's fix worked because the RPC response was read fresh in the same Node process. Every writer here is a separate, already-completed Python process by the time any JS/D1 code sees the number; there's nothing left to recover downstream. The fix has to move into the Python ingestion layer itself.
Reality check: today's actual per-neuron/per-subnet magnitudes are nowhere near the ~9M TAO ceiling. This is a correctness/consistency bug now (different writers can already disagree in low-order bits for the same conceptual quantity), not an active emergency — but it compounds over time as stake/emission accumulate.
Scope (sub-issues, work through in order)
Explicit non-goals for this roadmap
- No D1 schema/column-type change as part of Phases 1-2. The fix changes how the float is computed inside the writer, not what type is written to the
REAL column.
- No OpenAPI/contract/client-SDK change. Every field stays a plain JSON
number; the wire format is unaffected by Phases 1-2 (see Phase 4 for the one theoretical future exception).
- No bundling of the risky backfill (Phase 3) with the safe ingestion fix (Phase 1) in the same PR or deploy.
Why this matters now
Given a live incident earlier in this project's timeline from an under-verified schema assumption on a large production table, this roadmap is explicit about which phases are safe to ship immediately (1, 2) and which require real verification/staging before touching production data (3).
maintainer-only (owner)
Context
Chain-derived TAO/rao amounts (stake, emission, transfer/stake-swap amounts) are stored as SQLite
REAL(IEEE-754 double) across several D1 tables. A double is only exact up to2^53 - 1, which at 1 TAO = 1e9 rao corresponds to ~9,007,199 TAO. Values above that lose low-order-bit precision silently.A full audit (2026-07-03) found this is two distinct problems, not one:
Balance.__float__(), which itself performs the rao→TAO division in double precision — the exact rao integer is discarded before this codebase's own code ever runs. This is retroactively unfixable for already-written rows without re-querying the chain at the historical block height.fetch-events.py,backfill-neuron-history.py) hold the exact SCALE-decoded integer one line before dividing by1e9— an easy fix for new data, though existing rows are still already-lossy.Critically: this codebase already has one proven, merged fix for this exact bug class —
src/account-balance.mjs'stoRao()helper (issue #2070), which sums in BigInt rao-space and only converts to a JSNumberafter the final division. But that pattern cannot fix this issue — #2070's fix worked because the RPC response was read fresh in the same Node process. Every writer here is a separate, already-completed Python process by the time any JS/D1 code sees the number; there's nothing left to recover downstream. The fix has to move into the Python ingestion layer itself.Reality check: today's actual per-neuron/per-subnet magnitudes are nowhere near the ~9M TAO ceiling. This is a correctness/consistency bug now (different writers can already disagree in low-order bits for the same conceptual quantity), not an active emergency — but it compounds over time as stake/emission accumulate.
Scope (sub-issues, work through in order)
Explicit non-goals for this roadmap
REALcolumn.number; the wire format is unaffected by Phases 1-2 (see Phase 4 for the one theoretical future exception).Why this matters now
Given a live incident earlier in this project's timeline from an under-verified schema assumption on a large production table, this roadmap is explicit about which phases are safe to ship immediately (1, 2) and which require real verification/staging before touching production data (3).
maintainer-only (owner)