antd: /health reports ok when the node has no peers and cannot write (no peer-count or write-readiness signal)

[nkoteskey]

Summary

antd's GET /health reports status: ok even when the node has no usable DHT peers and cannot store a single chunk. There is no endpoint that exposes peer count or write-readiness, so a client cannot distinguish "healthy" from "reachable but unable to write" without attempting a paid upload and interpreting the failure.

What we hit

A long-running mainnet antd (v0.11.0, ~34h uptime) silently lost its routing table. Its own log showed:

Auto re-bootstrap: routing table size (0) below threshold (3)
[NETWORK] Found 1 closest nodes: [...]
single-node upload: 3 chunk(s) in 1 wave(s) (continue-on-partial)
POST /v1/files/public -> 502 Bad Gateway

Throughout this:

• GET /health returned {"status":"ok", ...}
• GET /v1/wallet/balance returned a funded wallet
• reads kept succeeding, served from the local cache — so content that had been uploaded earlier fetched fine and fast

Every health signal available to us said the node was fine. Three publish attempts failed before we read the daemon log. Restarting antd repopulated the routing table (5 peers) and the identical upload succeeded immediately.

Why the current surface isn't enough

/health carries status, network, version, evm_network, uptime_seconds, build_commit, and the payment addresses. None of that changes when the routing table empties. We also probed /v1/network/status, /v1/peers, /v1/network, /v1/status, /v1/node/info, and /v1/network/peers — none exist in 0.11.0.

That leaves clients inferring peer loss from a 502 on a write, which is (a) reactive, only observable after a user-visible failure, and (b) inference rather than measurement — a 502 is strong evidence but not proof, so any UI built on it risks asserting something it cannot actually know.

The cached-read behaviour is what makes this genuinely dangerous: a node in this state looks more healthy than a cold one, because reads are fast.

What would help

Something on /health (or a sibling endpoint) that reflects network participation. In rough order of usefulness to us:

1. A write-readiness boolean — can this node currently store data? That's the question clients actually have, and it lets the daemon answer it however it knows best.
2. Peer / routing-table count, plus the bootstrap threshold it's compared against, so a client can show "0 of 3 peers" rather than guessing.
3. Last successful store timestamp, which would also distinguish "never connected" from "was fine, degraded".

Any one of these turns inference into measurement. (1) alone would be enough for us.

Related thought

Given the daemon already detects this — it logs Auto re-bootstrap: routing table size (0) below threshold (3) — it may also be worth surfacing whether auto-rebootstrap is failing repeatedly, or escalating after N failed attempts. In our case the auto-rebootstrap ran but never recovered until a manual restart.

Happy to test any of this against our setup — we run a mainnet antd continuously and hit this in normal use.