feat(agent): scale-to-zero wake-on-traffic datapath (Phase 3) - #29
Merged
Conversation
Implements the full auto-suspend/auto-resume loop for desiredState=ScaleToZero, landing Phases 3+4+5 together so there is no roach-motel intermediate state. Cluster/hardware validation is deferred (see UNVALIDATED note); the feature is gated off by default behind the agent's IMP_SCALE_TO_ZERO opt-in. Core (internal/agent/scaletozero.go, platform-neutral + fully unit-tested): - wakeRegistry: tracks suspended VMs by IP, fires a reconcile via a source.Channel GenericEvent on the first matching packet, with dedup and no-lost-wake-on-full-channel semantics. - ScaleToZero.observe: TAP byte-counter idle detector; any traffic resets the clock, so a freshly-resumed VM always gets a full idleTimeout grace. - The two host-only syscalls are injected behind seams (linkStatsFunc, PacketSource) with fakes in tests; real netlink + AF_PACKET impls live in scaletozero_linux.go behind the swappable PacketSource (tc-BPF later). Reconciler wiring: - handleRunning: ScaleToZero VMs poll TAP traffic and auto-suspend when idle. - handleSuspending (Phase 4): keep the VTEP for ScaleToZero so the overlay still delivers the wake packet; register the VM's IP with the activator. - handleSuspended: resume only once a wake packet is observed. - handleResuming: clear wake state. - SetupWithManager: run the activator as a per-node runnable + wire the wake channel. Webhook: emit an experimental admission warning for desiredState=ScaleToZero. Tests: registry + idle math run with -race on the host; envtest state-machine cases (idle->suspend, wake->resume, busy stays running) run in CI on linux.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3 — Scale-to-Zero wake-on-traffic datapath
Adds the
ScaleToZerodesiredState mode (not a Running↔Suspended flip, which would flap): a ScaleToZero VM auto-suspends when its TAP goes idle and auto-resumes when a packet arrives for its IP. Lands the idle-detector, VTEP-retention, and activator/resume together so a VM can never suspend into a state it can't wake from ("roach motel").How it works
netlinklink stats). Any traffic resets the idle clock; a freshly-resumed VM always gets a fullidleTimeoutof grace (anti-thrash hysteresis). Idle past the timeout →Suspending.AF_PACKETraw socket observes inbound overlay frames; awakeRegistrymatches the destination IP to a suspended VM and enqueues an in-process reconcile (no spec write, no new RBAC).Suspended+ pending wake →Resuming.Test strategy (host-only; hardware validation deferred)
Host-untestable syscalls (netlink stats, AF_PACKET) sit behind
linkStatsFunc/PacketSourceseams with fakes, so all logic is host-unit-testable:-race): wake registry match/dedup/clear/full-channel-retry/re-register, idle observe hysteresis.Deferred (needs a live cluster / Firecracker host): confirming the
AF_PACKEThook actually observes the first frame destined to a TAP-less suspended VM. MarkedUNVALIDATEDin-code; thePacketSourceinterface exists precisely so the hook can be swapped to tc-BPF without touching any logic. Cold-ARP-cache and cross-resume packet buffering are named follow-ups.Gating
Off by default. Enable per-node with
IMP_SCALE_TO_ZERO=true; the webhook emits an experimental admission warning on ScaleToZero VMs.