Skip to content

fix: sweep-cleanup lingering sandboxes after each benchmark mode#32

Open
HeyGarrison wants to merge 1 commit intomasterfrom
fix/sandbox-cleanup-sweep
Open

fix: sweep-cleanup lingering sandboxes after each benchmark mode#32
HeyGarrison wants to merge 1 commit intomasterfrom
fix/sandbox-cleanup-sweep

Conversation

@HeyGarrison
Copy link
Contributor

Summary

  • After each provider finishes a benchmark mode (sequential, staggered, burst), call compute.sandbox.list() and destroy any sandboxes still alive
  • Logs how many lingering sandboxes were found and whether destroys succeeded
  • Prevents resource accumulation across modes — daytona was hitting quota exceeded on burst (51/100 failures) because sandboxes from earlier modes were still alive

Problem

We run 3 modes per provider: sequential (1 sandbox), staggered (100 sandboxes), burst (100 sandboxes) = ~201 total. The per-iteration finally blocks call destroy(), but:

  • Destroy can time out (15s) and we move on
  • The provider's delete API may be eventually consistent
  • @computesdk/daytona silently drops autoStopInterval/autoDeleteInterval from sandbox options, so sandboxes aren't ephemeral

By burst mode, dozens of sandboxes from earlier modes are still lingering, eating quota.

Fix

Added cleanupAllSandboxes(compute, providerName) that runs after each mode per provider:

  1. Calls compute.sandbox.list() (30s timeout)
  2. Destroys everything found via Promise.allSettled (15s timeout each)
  3. Logs results so we have visibility into what's being cleaned up

After each provider finishes a benchmark mode (sequential, staggered,
burst), call compute.sandbox.list() and destroy any sandboxes still
alive. This prevents resource accumulation across modes — e.g. sandboxes
from sequential/staggered modes eating quota before burst starts.

Previously we only relied on per-iteration finally blocks, but those
can miss sandboxes when destroy times out, fails silently, or the
provider's delete API is eventually consistent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant