Repository navigation
perf(demo,enterprise,containers): lite is the floor, and it is enough - #46
Merged
Merged
Conversation
Memory and disk bill on what is *provisioned*, for every awake second;
only CPU bills on what is used. An oversized instance therefore costs
full price to sit idle, which is most of what a gateway does. Both
gateways were `basic` — 1 GiB and 4 GB — against a real footprint a
fraction of that.
Measured under exactly lite's limits (--memory 256m --cpus 0.0625), on
these configs rather than a sample one:
demo boots 1.65s, idles 9.7 MiB, peaks 12.1 MiB across five
concurrent federated sessions — 4.7% of the 256 MiB cap
enterprise boots 1.75s, idles 11.0 MiB with auth, tenants, policy
and audit configured
demo-tasks idles 42.3 MiB, already lite, unchanged
So `lite` is not a squeeze, it is roughly 20x headroom. It is also the
floor: a custom instance type requires a minimum of 1 vCPU and 3 GiB per
vCPU, which makes the smallest configurable one *larger* than `basic`.
Under 1 vCPU the choice is lite or basic and nothing else. And because
CPU bills on utilization, the two cost the same to compute with — the
1/16 vCPU is a ceiling on bursts, not a saving. That ceiling is what to
watch if fan-out slows with many upstreams; going back is one line.
Verified after deploy: both gateways healthy, tools/list returns all 8
tools across three namespaces with no partialFailure, and jobs__start_job
still mints through the affinity pin.
Images were already minimal and are untouched — the gateways are the
unmodified distroless release (47.2 MB), which is the claim the demo
page makes, and demo-tasks is already multi-stage distroless static at
4.08 MB.
Also corrects a comment that had gone false: sleepAfter was documented as
a backstop because the uptime monitor pinged every 5 minutes and kept
these warm. That monitor is undeployed, so the timeout alone now decides
how long an idle container stays on the bill. Left at 1h; shortening it
is the other half of this bill and a separate call.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
blakebauman
added a commit
that referenced
this pull request
Aug 26, 2026
Memory and disk bill on what is provisioned, for every awake second, so with the uptime monitor undeployed this timeout is the only thing bounding what an idle container costs. All three drop to 15m. demo 1h -> 15m enterprise 1h -> 15m demo-tasks 2h -> 15m demo-tasks had the longest tail and the least reason for it. The 2h predates the teardown and carries no stated rationale; the board is ephemeral by design, demo jobs run for seconds, and the gateway forwards every tasks call — so the upstream stays awake exactly as long as anyone is actually using it, and there is no state here worth paying to keep warm. A poll that does land on a cold start gets the 503-and-retry the Worker already returns. The cost is a ~1.7s cold start for the first visitor after a quiet quarter hour, measured on lite in #46. That is a fair trade for a demo people arrive at one at a time rather than in a steady stream, and the enterprise gateway has more reason still: it sits behind a sign-in, so it is idle more of the time than the demo is. Verified after deploy: all three healthy, tools/list returns 8 tools across three namespaces with no partialFailure, and jobs__start_job still mints through the affinity pin. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both gateways move from
basictolite. Images are already minimal and are untouched — that was not where the money was.Why the instance type is the lever
Memory and disk bill on what is provisioned, for every awake second. Only CPU bills on what is used (Nov 2025 CPU pricing change). An oversized instance therefore costs full price to sit idle, which is most of what a gateway does.
litebasicDropping to
litecuts provisioned memory 4x and disk 2x per awake second. Because CPU bills on utilization, the two cost the same to compute with — the 1/16 vCPU is a ceiling on bursts, not a saving.Measured, not assumed
Run locally under exactly lite's limits (
--memory 256m --memory-swap 256m --cpus 0.0625), against the realFOLD_CONFIGfrom each app rather than a sample config:lite, unchanged)Roughly 20x headroom. This is not a squeeze.
liteis the floor, not a step toward oneA custom instance type requires a minimum of 1 vCPU and at least 3 GiB of memory per vCPU (limits), so the smallest configurable custom instance is larger than
basic. Under 1 vCPU the choice isliteorbasicand nothing else. There is no dial in between and nothing below.If fan-out ever gets slow with many upstreams, the 1/16 vCPU ceiling is the thing to suspect, and going back to
basicis the one line this PR changes.Images left alone, deliberately
ghcr.io/fold-run/fold:v1.13.0, 47.2 MB, distroless — the unmodified release binary, which is precisely the claim/#demoand the walkthrough make. Rebuilding it smaller would forfeit that.-trimpath -ldflags="-s -w", 4.08 MB.Verified after deploy
Both gateways are live on
lite:demo,enterprise,tasks/healthall 200tools/listreturns 8 tools acrosscfdocs(2),git(5),jobs(1), norun.fold/partialFailurejobs__start_jobstill mints through the affinity pin (demo-job-7,run.fold/upstream: demo-tasks)One comment corrected
sleepAfterwas documented as a mere backstop because the uptime monitor pinged every 5 minutes and kept these warm. That monitor is undeployed (see #45), so the timeout alone now decides how long an idle container stays on the bill. Left at1h— shortening it trades a ~1.7s cold start against the idle tail, and that is a separate call, not a silent one.🤖 Generated with Claude Code