diff --git a/website/docs/bitcoin/etl/s2-traverse-bitcoin.md b/website/docs/bitcoin/etl/s2-traverse-bitcoin.md index c04cf63f..73a2b44e 100644 --- a/website/docs/bitcoin/etl/s2-traverse-bitcoin.md +++ b/website/docs/bitcoin/etl/s2-traverse-bitcoin.md @@ -165,4 +165,23 @@ node files. ```shell .\aab.eba.exe bitcoin post-traverse --batches-filename batches.json -``` \ No newline at end of file +``` + +For each `*_nodes_Block` and `*_edges_Tx_Credits_Script` file, +the above command creates a new file that contains the updated information. +To ensure the downstream steps of the pipeline correctly refer to the files +containing the updated information, we can run the following script, +which creates a backup folder named `original` and moves the unmodified data files into it. +It then takes the newly updated files and removes the extra text from their filenames. + + +```shell +mkdir -p original && \ +mv *_nodes_Block.csv.gz *_edges_Tx_Credits_Script.csv.gz original/ 2>/dev/null; \ +for f in *_edges_Tx_Credits_Script_with_txo_spent_height_set.csv.gz; do \ + [ -e "$f" ] && mv "$f" "${f/_with_txo_spent_height_set/}"; \ +done; \ +for f in *_nodes_Block_supply_updated.csv.gz; do \ + [ -e "$f" ] && mv "$f" "${f/_supply_updated/}"; \ +done +``` diff --git a/website/docs/bitcoin/etl/s3-off-chain.md b/website/docs/bitcoin/etl/s3-off-chain.md index 964247d3..ec4360f4 100644 --- a/website/docs/bitcoin/etl/s3-off-chain.md +++ b/website/docs/bitcoin/etl/s3-off-chain.md @@ -43,7 +43,13 @@ You may run the following command for this step. Run the following command to add market-related features to the graph. ```shell -.\aab.eba.exe bitcoin augment --ohlcv-filename mapped-block-ohlcv.tsv +.\aab.eba.exe bitcoin augment --ohlcv-filename mapped-block-ohlcv.tsv --batches-filename batches.json ``` -Note that this process may take a considerable amount of time to complete. +```shell +mkdir -p pre_augment && \ +mv *_nodes_Block.csv.gz pre_augment/ 2>/dev/null; \ +for f in *_nodes_Block_with_economic_indicators.csv.gz; do \ + [ -e "$f" ] && mv "$f" "${f/_with_economic_indicators/}"; \ +done +``` diff --git a/website/docs/bitcoin/etl/s4a-import.md b/website/docs/bitcoin/etl/s4a-import.md index 3cc60586..5214ed31 100644 --- a/website/docs/bitcoin/etl/s4a-import.md +++ b/website/docs/bitcoin/etl/s4a-import.md @@ -225,12 +225,12 @@ meaning it does not support incremental updates to an existing graph. & "$env:NDIR\bin\neo4j-admin.bat" database import full ` --overwrite-destination neo4j ` --nodes="$env:GDIR\Coinbase.csv.gz" ` - --nodes="$env:GDIR\header_nodes_Block.csv.gz,$env:GDIR\[0-9].*_nodes_Block_supply_updated.csv.gz" ` + --nodes="$env:GDIR\header_nodes_Block.csv.gz,$env:GDIR\[0-9].*_nodes_Block.csv.gz" ` --nodes="$env:GDIR\header_nodes_Script.csv.gz,$env:GDIR\unique_nodes_Script.csv.gz" ` --nodes="$env:GDIR\header_nodes_Tx.csv.gz,$env:GDIR\unique_nodes_Tx.csv.gz" ` --relationships="$env:GDIR\header_edges_Block_Confirms_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Block_Confirms_Tx.csv.gz" ` --relationships="$env:GDIR\header_edges_Coinbase_Mints_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Coinbase_Mints_Tx.csv.gz" ` - --relationships="$env:GDIR\header_edges_Tx_Credits_Script.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Credits_Script_with_txo_spent_height_set.csv.gz" ` + --relationships="$env:GDIR\header_edges_Tx_Credits_Script.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Credits_Script.csv.gz" ` --relationships="$env:GDIR\header_edges_Script_Redeems_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Script_Redeems_Tx.csv.gz" ` --relationships="$env:GDIR\header_edges_Tx_Fee_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Fee_Tx.csv.gz" ` --relationships="$env:GDIR\header_edges_Tx_Transfers_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Transfers_Tx.csv.gz" `