Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion website/docs/bitcoin/etl/s2-traverse-bitcoin.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,4 +165,23 @@ node files.

```shell
.\aab.eba.exe bitcoin post-traverse --batches-filename batches.json
```
```

For each `*_nodes_Block` and `*_edges_Tx_Credits_Script` file,
the above command creates a new file that contains the updated information.
To ensure the downstream steps of the pipeline correctly refer to the files
containing the updated information, we can run the following script,
which creates a backup folder named `original` and moves the unmodified data files into it.
It then takes the newly updated files and removes the extra text from their filenames.


```shell
mkdir -p original && \
mv *_nodes_Block.csv.gz *_edges_Tx_Credits_Script.csv.gz original/ 2>/dev/null; \
for f in *_edges_Tx_Credits_Script_with_txo_spent_height_set.csv.gz; do \
[ -e "$f" ] && mv "$f" "${f/_with_txo_spent_height_set/}"; \
done; \
for f in *_nodes_Block_supply_updated.csv.gz; do \
[ -e "$f" ] && mv "$f" "${f/_supply_updated/}"; \
done
```
10 changes: 8 additions & 2 deletions website/docs/bitcoin/etl/s3-off-chain.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,13 @@ You may run the following command for this step.
Run the following command to add market-related features to the graph.

```shell
.\aab.eba.exe bitcoin augment --ohlcv-filename mapped-block-ohlcv.tsv
.\aab.eba.exe bitcoin augment --ohlcv-filename mapped-block-ohlcv.tsv --batches-filename batches.json
```

Note that this process may take a considerable amount of time to complete.
```shell
mkdir -p pre_augment && \
mv *_nodes_Block.csv.gz pre_augment/ 2>/dev/null; \
for f in *_nodes_Block_with_economic_indicators.csv.gz; do \
[ -e "$f" ] && mv "$f" "${f/_with_economic_indicators/}"; \
done
```
4 changes: 2 additions & 2 deletions website/docs/bitcoin/etl/s4a-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,12 +225,12 @@ meaning it does not support incremental updates to an existing graph.
& "$env:NDIR\bin\neo4j-admin.bat" database import full `
--overwrite-destination neo4j `
--nodes="$env:GDIR\Coinbase.csv.gz" `
--nodes="$env:GDIR\header_nodes_Block.csv.gz,$env:GDIR\[0-9].*_nodes_Block_supply_updated.csv.gz" `
--nodes="$env:GDIR\header_nodes_Block.csv.gz,$env:GDIR\[0-9].*_nodes_Block.csv.gz" `
--nodes="$env:GDIR\header_nodes_Script.csv.gz,$env:GDIR\unique_nodes_Script.csv.gz" `
--nodes="$env:GDIR\header_nodes_Tx.csv.gz,$env:GDIR\unique_nodes_Tx.csv.gz" `
--relationships="$env:GDIR\header_edges_Block_Confirms_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Block_Confirms_Tx.csv.gz" `
--relationships="$env:GDIR\header_edges_Coinbase_Mints_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Coinbase_Mints_Tx.csv.gz" `
--relationships="$env:GDIR\header_edges_Tx_Credits_Script.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Credits_Script_with_txo_spent_height_set.csv.gz" `
--relationships="$env:GDIR\header_edges_Tx_Credits_Script.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Credits_Script.csv.gz" `
--relationships="$env:GDIR\header_edges_Script_Redeems_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Script_Redeems_Tx.csv.gz" `
--relationships="$env:GDIR\header_edges_Tx_Fee_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Fee_Tx.csv.gz" `
--relationships="$env:GDIR\header_edges_Tx_Transfers_Tx.csv.gz,$env:GDIR\[0-9].*_edges_Tx_Transfers_Tx.csv.gz" `
Expand Down
Loading