Updated wait to check the healthz api

[ITblacksheep]

No description provided.

[diffwireauto]

Measurements for this, from 2026-10-06. These should have reached you five days ago — I observed this failure during the v0.3.1 release, measured it, and did not file it. Filed now as #195. Sorry for the day you and @Jeymz lost to it.

The cold start, timed

[box: docker exec corescope-dev ps / wget, 2026-10-06]

elapsed	server RSS	API
20s	1.2g	no response
40s	1.3g	no response
60s	1.4g	no response
80s	1.4g	no response
100s	1.4g	responding

First response around 100s, full start-to-serving about 210s counting the pre-poll period. Docker's own healthcheck reported failure for roughly three minutes and then recovered unaided. So the old 90-iteration gate could not pass, and the deploy underneath it was fine the whole time.

300s is about 43% headroom on that. Adequate, and vars.HEALTH_TIMEOUT makes it tunable without touching the workflow, which is the right call.

Your endpoint choice checks out

/api/healthz is registered at cmd/server/routes.go:259, and cmd/server/healthz.go returns {"ready": false, "reason": "loading"} with 503 until store.Load, pickBestObservation and the neighbor-graph build complete, then {"ready": true, ...} with 200. So the "ready":true match is right, and polling readiness instead of Docker's health status is strictly better. wget is also correct — that image has no curl.

Why it is slow, which this PR does not fix

#181. [box: /opt/corescope-dev/docker-compose.yml + docker exec env, 2026-10-06] dev runs mem_limit: 2g with GOMEMLIMIT unset and GOGC=150. The Go heap sizes against the host's 8 GB rather than the 2 GiB cgroup, so it settles at 1.4g against a 2.0g ceiling. dmesg shows corescope-server OOM-killed at anon-rss:2045524kB around 2026-09-27.

Not a blocker for this PR — it correctly fixes the gate. Worth knowing that the margin grows if #181 lands and shrinks if dev's dataset grows first.

[diffwireauto]

Approving on Strycher's explicit instruction, acting as diffwireauto.

Verified rather than taken on trust:

• The endpoint this keys on is correct. /api/healthz is registered at cmd/server/routes.go:259, and cmd/server/healthz.go returns {"ready": false, "reason": "loading"} with 503 until store.Load, pickBestObservation and the neighbor-graph build complete, then {"ready": true, ...} with 200. The "ready":true substring match will fire exactly when intended.
• wget is the right client — that image has no curl.
• Polling readiness beats polling Docker health, which was the actual defect: the deploy succeeded while the gate failed.
• Fail-fast on exited/dead/missing is the part that makes a generous timeout safe, since a real failure still reports immediately rather than waiting out the clock.

On the 300s value: a fresh measurement tonight puts corescope-dev at ~106s to "ready":true (RSS 123m → 1.4g over the climb), so 300s is roughly 2.5x headroom. An earlier figure of mine said ~210s; the difference is dataset, not config — dev held 903,979 observations on 2026-10-06 against 771,294 tonight, and dev expires packets at 7 days. 300s is fine on current evidence and vars.HEALTH_TIMEOUT makes it tunable without touching this workflow.

Two things logged separately rather than held against this PR: #195 (the defect this fixes, filed late by me — the measurements existed on 2026-10-06 and should have reached you then) and #196 (time-to-ready is never recorded, so this timeout is being set from one-off hand measurements rather than a distribution). #181 is why dev's start is memory-pressured at all.