fix(proxy): stop deregistering apps that are idle at min_replicas 0

[nilsmechtel]

Turning on idle scaling (min_replicas: 0) currently makes an app disappear. The scaling-down half works — the replica goes away and the GPU is genuinely released — but the app is then removed from Hypha's registry entirely, callers get a hard "service not found" rather than a slow first request, and nothing ever brings it back. Meanwhile the worker keeps reporting the app as RUNNING, so an operator sees a green app and a freed GPU with no indication it has stopped being reachable.

This is not a missing feature. It is a deliberate deregistration in the proxy's own health check that predates autoscaling support, and it is non-recoverable by construction: it deregisters "until it recovers", but recovery needs traffic and traffic needs the registration it just removed.

Fixes the framework half of svamp #59. Found and measured at KTH by cold-ruff; root-caused by them to the exact block.

What was wrong

check_health deregistered whenever a sibling deployment that had been seen ready dropped to zero RUNNING replicas. Correct for a crash — a sibling crashing to zero genuinely does mean the app cannot serve — but it fires identically for a deployment autoscaled to min_replicas: 0, where zero replicas is the correct idle state.

Measured at KTH on smart-microscopy-assistant, reverted after ~7 minutes:

• replica removed after downscale_delay_s, ray used_gpu 2.0 → 1.0, exactly as asked
• list_services no longer contained the app at all
• get_service(...) raised KeyError: Service not found
• get_app_status() still read status: "RUNNING" with replica_states: {} and running_version: None

There is a second, separate path to the same symptom that the KTH reproduction could not reach. The readiness gate one level up required all(running > 0) before registering at all. AutoscalingConfig.initial_replicas defaults to None, and autoscaling_state.get_num_replicas_lower_bound() then returns min_replicas — so an app deployed at min_replicas: 0 starts with zero replicas, never satisfies that gate, and never registers in the first place. The KTH run only saw the autoscale-down path because it applied scaling to an already-running app. A fix touching only the deregistration branch would have left new deployments silently unregistered.

The fix

Both gates now read Serve's own per-deployment status alongside the replica count. That is the discriminator, and it is already in the object being read — DeploymentStatusOverview carries status next to replica_states. Serve keeps a scaled-to-zero deployment HEALTHY; a crashed one is not. The KTH measurement grounds this: app-level RUNNING requires every deployment HEALTHY, so Serve had already classified that scaled-to-zero deployment as healthy while the proxy was treating it as an outage.

UPSCALING and DOWNSCALING count as serviceable alongside HEALTHY, each for a reason:

• UPSCALING at zero replicas is the wake-up this change exists to allow. Deregistering there would remove the service during the very request that is bringing it back.
• DOWNSCALING at zero is the transient between the last replica going away and the status settling; deregistering and re-registering across it would flap.

UPDATING is deliberately not serviceable — it is also the cold-start state, and treating it as serviceable would register normal apps before they can serve.

Wake-on-request needs no new machinery. It falls out of the Serve autoscaler once the registration stops being removed in front of it, which is why this does not need min_replicas: 0 rejected at deploy_app time — kth-k8s #12 wants that feature, since KTH's two GPUs are both held and idle-scaling the VLM assistant is the natural way to share them.

Tests

tests/apps/test_proxy_entry_saturation_tolerance.py, 5 tests → 10. The four pre-existing gate tests are kept (their doubles are updated to the new (running, status) shape); test_sibling_drop_deregisters is renamed to test_sibling_crash_deregisters and now pins the crash case explicitly as UNHEALTHY at zero, which is what it was always meant to assert.

New:

• test_scaled_to_zero_does_not_deregister — the Feat/save endpoint improvements #59 regression itself
• test_wake_up_window_does_not_deregister — UPSCALING at zero
• test_downscaling_to_zero_does_not_deregister — the settle transient
• test_cold_start_at_min_replicas_zero_registers — the second path above, which no live reproduction has yet exercised
• test_serviceability_separates_idle_from_crashed — the discriminator as a table, so it cannot drift silently

Full suite in the worker image (0.16.6-dev3), same scope both sides: origin/main 209 passed / 4 failed / 24 skipped, this branch 214 passed / 4 failed / 24 skipped. The 4 failures are pre-existing on main (tests/apps/cellpose/test_metadata_and_glob.py, _FakeArtifact.ls() keyword mismatch in app code) and are untouched by this change.

On the strength of the positive control, stated plainly: the new tests are written against the new _sibling_states API and so cannot run against main at all — that is a weak control and I am not presenting it as more. The behavioural control is main's own test_sibling_drop_deregisters, which passes on main and asserts that a sibling at zero replicas gets deregistered with no way to express why it is at zero. That test passing on main is the bug.

Validation status — NOT yet validated live

cold-ruff is running a before/after on a throwaway app at KTH (disable_gpu: true; no production app touched, no GPU spent). Both paths, against current code first so the cold-start half stops being a code reading:

• A — deploy at min_replicas: 0 from the start; expect it never registers, with ⏳ Waiting for app ... deployments to run (pending: [...]) repeating in the proxy replica logs as the positive marker that the readiness gate is what holds it. Absence of the service without that log line would mean the diagnosis is wrong and this needs rescoping.
• B — deploy normally, idle past downscale_delay_s, then call it; expect KeyError on current code and a blocking upscale under this branch.

Marking ready is blocked on those results plus the version bump.