The nightly snippet run failed on c1cb04dfa41e8dbca73d0069ffc5a94070936d92.
Run log: https://github.com/allora-network/docs/actions/runs/31239309367
Snippet run report
- snippets discovered: 15
- executed: 13
- failed: 4
snippets/forecaster.py
exited with code 1
Traceback (most recent call last):
File "/tmp/allora-snippets-CCJhnn/runs/forecaster.py/forecaster.py", line 38, in <module>
asyncio.run(main())
File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/tmp/allora-snippets-CCJhnn/runs/forecaster.py/forecaster.py", line 31, in main
async for result in worker.run():
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/allora_sdk/worker/worker.py", line 498, in run
await self._ensure_initialized()
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/allora_sdk/worker/worker.py", line 332, in _ensure_initialized
self._chain_id = await self.client.raise_for_chain_id_mismatch()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/allora_sdk/rpc_client/client.py", line 211, in raise_for_chain_id_mismatch
chain_id = await self.chain_id()
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/allora_sdk/rpc_client/client.py", line 218, in chain_id
node_info_resp = await self.tendermint.query.get_node_info(tendermint_v1beta1.GetNodeInfoRequest())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/allora_sdk/rpc_client/protos/cosmos/base/tendermint/v1beta1/__init__.py", line 619, in get_node_info
return await self._unary_unary(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/betterproto2/grpc/grpclib_client.py", line 69, in _unary_unary
response = await stream.recv_message()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/grpclib/client.py", line 426, in recv_message
await self.recv_initial_metadata()
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/grpclib/client.py", line 394, in recv_initial_metadata
self._raise_for_grpc_status(status, message, details)
File "/tmp/allora-snippets-CCJhnn/venv/lib/python3.11/site-packages/grpclib/client.py", line 346, in _raise_for_grpc_status
raise GRPCError(status, message, details)
grpclib.exceptions.GRPCError: (<Status.UNAVAILABLE: 14>, 'upstream connect error or disconnect/reset before headers. reset reason: connection termination', None)
snippets/migrate_worker.py
printed an error it swallowed (/Inference worker error:/)
_ _ _ ___ ____ _
/ \ | | | | / _ \| _ \ / \
/ _ \ | | | | | | | | |_) | / _ \
/ ___ \| |___| |__| |_| | _ < / ___ \ Chain: allora-testnet-1
/_/ \_\_____|_____\___/|_| \_\/_/ \_\ Topic: PLAYGROUND: 1 day BTC/USD Price Prediction (ID: 69)
__ _____ ____ _ _______ ____ Address: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe
\ \ / / _ \| _ \| |/ / ____| _ \ Role: INFERER
\ \ /\ / / | | | |_) | ' /| _| | |_) |
\ V V /| |_| | _ <| . \| |___| _ <
\_/\_/ \___/|_| \_\_|\_\_____|_| \_\
2026-08-08 04:23:58,986 INF Worker wallet: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe || Balance: 100002.395366489480469174 ALLO
2026-08-08 04:23:59,011 INF 🔄 Starting polling worker
2026-08-08 04:23:59,035 INF Topic 69: unfulfilled nonces: {10418255}
2026-08-08 04:23:59,036 INF Our unfulfilled nonces: {10418255}
2026-08-08 04:23:59,049 INF 👉 Found new nonce 10418255 for topic 69, submitting... account_seq=40
2026-08-08 04:23:59,073 WRN ⚠️⚠️⚠️ SANITY CHECK WARNING: Your prediction (123.450000) is 328.4 standard deviations from the network consensus (mean: 64332.131497, std: 195.490681). Please verify you're predicting the correct target variable and using the right units.
2026-08-08 04:24:09,251 INF ⚠️ Already submitted for this epoch: topic_id=69 nonce=10418255 code=1
2026-08-08 04:24:09,275 INF Worker wallet: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe || Balance: 100002.395366489466969174 ALLO
Inference worker error: TxError: codespace=undefined code=1 tx_hash=A5C12FEE328A53D97D064CD5F59AF132FC6E733A4EC4648E32FE9EB2FD3DB9BB failed to execute message; message index: 0: Error planning inference admission: inference already submitted
2026-08-08 04:24:09,286 INF Received signal 15, initiating graceful shutdown
snippets/sdk_py_worker.py
printed an error it swallowed (/Inference worker error:/)
_ _ _ ___ ____ _
/ \ | | | | / _ \| _ \ / \
/ _ \ | | | | | | | | |_) | / _ \
/ ___ \| |___| |__| |_| | _ < / ___ \ Chain: allora-testnet-1
/_/ \_\_____|_____\___/|_| \_\/_/ \_\ Topic: PLAYGROUND: 1 day BTC/USD Price Prediction (ID: 69)
__ _____ ____ _ _______ ____ Address: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe
\ \ / / _ \| _ \| |/ / ____| _ \ Role: INFERER
\ \ /\ / / | | | |_) | ' /| _| | |_) |
\ V V /| |_| | _ <| . \| |___| _ <
\_/\_/ \___/|_| \_\_|\_\_____|_| \_\
2026-08-08 04:27:13,473 INF Worker wallet: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe || Balance: 100002.395366489453469164 ALLO
2026-08-08 04:27:13,499 INF 🔄 Starting polling worker
2026-08-08 04:27:13,524 INF Topic 69: unfulfilled nonces: {10418309}
2026-08-08 04:27:13,524 INF Our unfulfilled nonces: {10418309}
2026-08-08 04:27:13,549 INF 👉 Found new nonce 10418309 for topic 69, submitting... account_seq=42
2026-08-08 04:27:15,691 INF ⚠️ Already submitted for this epoch: topic_id=69 nonce=10418309 code=1
2026-08-08 04:27:15,714 INF Worker wallet: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe || Balance: 100002.395366489439969164 ALLO
Inference worker error: TxError: codespace=undefined code=1 tx_hash=F75780859D3CBBCC37B364E201BA156A8ED31CDD5047298EBFF7B47E2FE67629 failed to execute message; message index: 0: Error planning inference admission: inference already submitted
2026-08-08 04:27:15,726 INF Received signal 15, initiating graceful shutdown
snippets/worker_minimal.py
printed an error it swallowed (/Inference worker error:/)
_ _ _ ___ ____ _
/ \ | | | | / _ \| _ \ / \
/ _ \ | | | | | | | | |_) | / _ \
/ ___ \| |___| |__| |_| | _ < / ___ \ Chain: allora-testnet-1
/_/ \_\_____|_____\___/|_| \_\/_/ \_\ Topic: PLAYGROUND: 1 day BTC/USD Price Prediction (ID: 69)
__ _____ ____ _ _______ ____ Address: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe
\ \ / / _ \| _ \| |/ / ____| _ \ Role: INFERER
\ \ /\ / / | | | |_) | ' /| _| | |_) |
\ V V /| |_| | _ <| . \| |___| _ <
\_/\_/ \___/|_| \_\_|\_\_____|_| \_\
2026-08-08 04:27:20,913 INF Worker wallet: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe || Balance: 100002.395366489439969164 ALLO
2026-08-08 04:27:20,942 INF 🔄 Starting polling worker
2026-08-08 04:27:20,968 INF Topic 69: unfulfilled nonces: {10418309}
2026-08-08 04:27:20,968 INF Our unfulfilled nonces: {10418309}
2026-08-08 04:27:20,994 INF 👉 Found new nonce 10418309 for topic 69, submitting... account_seq=43
2026-08-08 04:27:39,246 INF ⚠️ Already submitted for this epoch: topic_id=69 nonce=10418309 code=1
2026-08-08 04:27:39,270 INF Worker wallet: allo12vaa2z44uyse3sagxvrya8mavahguamqfqxefe || Balance: 100002.395366489426469164 ALLO
Inference worker error: TxError: codespace=undefined code=1 tx_hash=25701F2DCC58D983A48C97DB0A85A967445232DC2C0963AF309A81336F98AC27 failed to execute message; message index: 0: Error planning inference admission: inference already submitted
2026-08-08 04:27:39,283 INF Received signal 15, initiating graceful shutdown
The nightly snippet run failed on
c1cb04dfa41e8dbca73d0069ffc5a94070936d92.Run log: https://github.com/allora-network/docs/actions/runs/31239309367
Snippet run report
snippets/forecaster.pyexited with code 1
snippets/migrate_worker.pyprinted an error it swallowed (/Inference worker error:/)
snippets/sdk_py_worker.pyprinted an error it swallowed (/Inference worker error:/)
snippets/worker_minimal.pyprinted an error it swallowed (/Inference worker error:/)