Massive refactoring to use local JSONRPC interface. - #31
Conversation
Remove all CLI-based scraping code and use JSON/RPC to interact with the validator programmatically, instead. Also remove all queries to the Helium API; they can be satisfied locally and likely more accurately.
Clean up the mechanism used for passing parameters to JSONRPC functions. Do it the Python way, using keyword arguments. Then unpack them into a JSON dictionary at the appropriate time. Also, for the time being, don't ask the validator to enumerate all validators recorded in the ledger. This is an expensive operation and doesn't help much in the long run. Instead, ask for this validator's stats, only.
| UPDATE_PERIOD = int(os.environ.get('UPDATE_PERIOD', 30)) | ||
| VALIDATOR_CONTAINER_NAME = os.environ.get('VALIDATOR_CONTAINER_NAME', 'validator') | ||
| # for testnet, https://testnet-api.helium.wtf/v1 | ||
| API_BASE_URL = os.environ.get('API_BASE_URL', 'https://api.helium.io/v1') |
There was a problem hiding this comment.
Should this also be deleted since the public API is no longer used?
|
|
||
| if __name__ == '__main__': | ||
| prometheus_client.start_http_server(9825) # 9-VAL on your phone | ||
| miner = MinerJSONRPC('http://localhost:4467/') |
There was a problem hiding this comment.
Does localhost work for docker? I thought someone determined when JSONRPC came out that docker won't pick up that endpoint. Or did this PR fix this issues? helium/miner#884
|
I also tried to spin this up on one of validators to test it out. I'm able to get the service running and active while the miner is stopped however once I start the miner the service crashes with the following status. ● validator_exporter.service - Validator statistics for Prometheus
Loaded: loaded (/etc/systemd/system/validator_exporter.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2021-07-25 16:28:09 UTC; 6s ago
Process: 43513 ExecStart=/home/helium/validator_exporter/run (code=exited, status=1/FAILURE)
Main PID: 43513 (code=exited, status=1/FAILURE)
Jul 25 16:28:09 ip-10-0-1-161 systemd[1]: validator_exporter.service: Scheduled restart job, restart counter is at 5.
Jul 25 16:28:09 ip-10-0-1-161 systemd[1]: Stopped Validator statistics for Prometheus.
Jul 25 16:28:09 ip-10-0-1-161 systemd[1]: validator_exporter.service: Start request repeated too quickly.
Jul 25 16:28:09 ip-10-0-1-161 systemd[1]: validator_exporter.service: Failed with result 'exit-code'.
Jul 25 16:28:09 ip-10-0-1-161 systemd[1]: Failed to start Validator statistics for Prometheus.I'm building from source but the validator is running in a directory |
|
This code depends on a change to the hbbft perf call that still isn't merged. That's likely what is crashing. |
|
Ahh well fair enough, I thought maybe I messed up the install... I'm super excited to see this in action though! |
|
I thought I heard through the grape vine that hbbft_perf is now supported via JSONRPC.. do you think this will work now? |
|
It could be! Let's see if they actually merged my PR. |
|
Nope. PR is still open =( helium/miner#936 |
| log.error("in consensus fetch failure") | ||
|
|
||
| this_validator = None | ||
| try: |
There was a problem hiding this comment.
With release 1.2.2 the PR you were waiting on has been merged! After spinning this up the following error continues to occur "validator fetch failure" so maybe the address filter isn't working on ledger validators?
|
@ke6jjj Hey, sorry to bother again. It appears that there's some missing information from the scrapper it could be related to the error I noted above. But my python is absolutely terrible and my understanding of the JSONRPC is even less... It appears that it's not able to get the ledger details for the validator..
Let me know if I can help at all besides annoying you with this issues... |
|
No problem! I'll have to take a look and see how things have diverged. I haven't been running this branch recently. |
2 participants
Use the new JSONRPC interface to the validator/miner when retrieving stats, removing
the need to scrape the CLI output and removing the need to query the Helium blockchain
API.