Difficulty plot helpers - original PR - #1967
ImplOfAnImpl wants to merge 4 commits into
Conversation
|
|
||
| # The cache for docker container dependency | ||
| .cargo | ||
| .cargo/* |
There was a problem hiding this comment.
FYI: I have no idea what "The cache for docker container dependency" means and suspect that this line is not actually needed. But for now I've just made it consistent with .gitignore
…ock difficulty target
…postgres storage; fix some inconsistencies between postgres and in-memory storages
e83e0dd to
9cdb184
Compare
| genesis_info = self.get("chain/genesis", {}) | ||
|
|
||
| block_infos_by_height = {} | ||
| starting_height = 1 |
There was a problem hiding this comment.
It would also be nice if we can update an existing file instead of always fetching the entire chain, i.e. we can check the output file if it exists and is valid, get the last height (as it is not stored, maybe number of lines I guess) and start from there, and append to it only the new blocks.
There was a problem hiding this comment.
It would also be nice if we can update an existing file instead of always fetching the entire chain, i.e. we can check the output file if it exists and is valid, get the last height (as it is not stored, maybe number of lines I guess) and start from there, and append to it only the new blocks.
Well, even in this PR the whole chain retrieval was taking under 20s (at least when the API server was being run locally), so the improvement wouldn't be that useful IMO.
And with the new approach (using the chainstate dumper) it takes just a few seconds.
Edit: this has been superseded by #1974 (the difficulty plot helpers themselves) and #1971 (minor improvements and cleanup).
The original description goes below.
Here I add a couple of python scripts to plot block targets and block time differences -
collect_data.pycollects the data from the API server andshow_plots.pymakes the plots usingmatplotlib.(previously we would collect the data from node's logs and use Mathematica to make the plots, which was inconvenient).
The plots can be improved further I guess, but even in the current state they are good enough to observe the chain performance.
In order to be able to obtain block targets and timestamps from the API server "in bulk", I had to add a new endpoint "/chain" which returns an array of "block infos" containing block's id, height, timestamp and target.
For consistency, I also changed the "/chain/tip" endpoint, so that it also returns the timestamp and the target.
P.S. actually now I'm not sure if using the API server to collect the data was a good idea. Probably it'd be better if
collect_data.pywas accessing the local node's chainstatedb directly instead, this way we'd be able to add more interesting visualizations in the future (e.g. visualize the block tree stored inside the chainstatedb).But I guess the changes in the API server are not completely useless on their own, so I'd keep it this way for now.
P.S.2 I also had to "cargo update"
tracing-subscriberto address the vulnerability https://rustsec.org/advisories/RUSTSEC-2025-0055P.S.3 I've modified
set_mainchain_blockin the postgres storage so that it always updates all columns on block id conflict (the reason being that it's a low-level function that shouldn't make assumptions about the data). Also I fixed some inconsistencies between the postgres and in-memory storage implementations.