Skip to content

Resolve Txo spending post-traverse #43

Merged
VJalili merged 11 commits into
B1AAB:mainfrom
VJalili:spent-utxo
Apr 12, 2026
Merged

Resolve Txo spending post-traverse #43
VJalili merged 11 commits into
B1AAB:mainfrom
VJalili:spent-utxo

Conversation

@VJalili

@VJalili VJalili commented Apr 12, 2026

Copy link
Copy Markdown
Collaborator

Given a Txo, Bitcoin Core does not provide details about its spending (only an indirect method to determine whether it is spent or unspent). However, for many quant-related features, we need details such as the block height where a Txo is spent (see #41 for details).

There are multiple ways to track Txo spending. In earlier versions of EBA, we used PostgreSQL to track created and spent Txos, which were populated as part of the traverse process. However, using this database had a significant performance penalty at traverse time, so that feature was dropped. As part of #41 we tried relying on the graph imported in Neo4j, and used Neo4j to track Txo spending. The graph schema provides an efficient way to track the spending; however, for many quant-related features (e.g., NUPL) we need to update the graph with spending information, or else we will need to track all Txo spending for feature computation. And, the drawback of this method lies at the updating step, where updating tens of billions of edges takes a considerable amount of time.

This PR adds a method that runs post-traverse and pre-graph-import. Hence, it does not add a performance penalty to traverse, and does not need a mass database update post-import. This method creates a per-batch file that contains all the Txos spent in that batch by reading all the *_edges_Script_Redeems_Tx.csv.gz files, then updates all the edges in the *_edges_Tx_Credits_Script.csv.gz files. Since the Txo spending is recorded per-batch, it can perform an update in memory, without a need for a database-based solution.

@VJalili
VJalili merged commit bd3af9f into B1AAB:main Apr 12, 2026
@VJalili
VJalili deleted the spent-utxo branch April 12, 2026 01:13
VJalili added a commit to VJalili/EBA that referenced this pull request Apr 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant