Skip to content

perf(demo,enterprise,containers): lite is the floor, and it is enough - #46

Merged
blakebauman merged 1 commit into
mainfrom
containers-lite
Aug 26, 2026
Merged

blakebauman merged 1 commit into
mainfrom
containers-lite

Conversation

@blakebauman

Copy link
Copy Markdown
Member

Both gateways move from basic to lite. Images are already minimal and are untouched — that was not where the money was.

Why the instance type is the lever

Memory and disk bill on what is provisioned, for every awake second. Only CPU bills on what is used (Nov 2025 CPU pricing change). An oversized instance therefore costs full price to sit idle, which is most of what a gateway does.

vCPU Memory Disk
lite 1/16 256 MiB 2 GB
basic 1/4 1 GiB 4 GB

Dropping to lite cuts provisioned memory 4x and disk 2x per awake second. Because CPU bills on utilization, the two cost the same to compute with — the 1/16 vCPU is a ceiling on bursts, not a saving.

Measured, not assumed

Run locally under exactly lite's limits (--memory 256m --memory-swap 256m --cpus 0.0625), against the real FOLD_CONFIG from each app rather than a sample config:

Container Boot (1/16 vCPU) Idle Under load % of 256 MiB cap
demo 1.65s 9.7 MiB 12.1 MiB across 5 concurrent federated sessions 4.7%
enterprise 1.75s 11.0 MiB (auth, tenants, policy, audit) — 4.3%
demo-tasks — 42.3 MiB — 16.5% (already lite, unchanged)

Roughly 20x headroom. This is not a squeeze.

lite is the floor, not a step toward one

A custom instance type requires a minimum of 1 vCPU and at least 3 GiB of memory per vCPU (limits), so the smallest configurable custom instance is larger than basic. Under 1 vCPU the choice is lite or basic and nothing else. There is no dial in between and nothing below.

If fan-out ever gets slow with many upstreams, the 1/16 vCPU ceiling is the thing to suspect, and going back to basic is the one line this PR changes.

Images left alone, deliberately

  • demo / enterprise: ghcr.io/fold-run/fold:v1.13.0, 47.2 MB, distroless — the unmodified release binary, which is precisely the claim /#demo and the walkthrough make. Rebuilding it smaller would forfeit that.
  • demo-tasks: already multi-stage distroless static with -trimpath -ldflags="-s -w", 4.08 MB.

Verified after deploy

Both gateways are live on lite:

  • demo, enterprise, tasks /health all 200
  • tools/list returns 8 tools across cfdocs (2), git (5), jobs (1), no run.fold/partialFailure
  • jobs__start_job still mints through the affinity pin (demo-job-7, run.fold/upstream: demo-tasks)

One comment corrected

sleepAfter was documented as a mere backstop because the uptime monitor pinged every 5 minutes and kept these warm. That monitor is undeployed (see #45), so the timeout alone now decides how long an idle container stays on the bill. Left at 1h — shortening it trades a ~1.7s cold start against the idle tail, and that is a separate call, not a silent one.

🤖 Generated with Claude Code

Memory and disk bill on what is *provisioned*, for every awake second;
only CPU bills on what is used. An oversized instance therefore costs
full price to sit idle, which is most of what a gateway does. Both
gateways were `basic` — 1 GiB and 4 GB — against a real footprint a
fraction of that.

Measured under exactly lite's limits (--memory 256m --cpus 0.0625), on
these configs rather than a sample one:

  demo        boots 1.65s, idles 9.7 MiB, peaks 12.1 MiB across five
              concurrent federated sessions — 4.7% of the 256 MiB cap
  enterprise  boots 1.75s, idles 11.0 MiB with auth, tenants, policy
              and audit configured
  demo-tasks  idles 42.3 MiB, already lite, unchanged

So `lite` is not a squeeze, it is roughly 20x headroom. It is also the
floor: a custom instance type requires a minimum of 1 vCPU and 3 GiB per
vCPU, which makes the smallest configurable one *larger* than `basic`.
Under 1 vCPU the choice is lite or basic and nothing else. And because
CPU bills on utilization, the two cost the same to compute with — the
1/16 vCPU is a ceiling on bursts, not a saving. That ceiling is what to
watch if fan-out slows with many upstreams; going back is one line.

Verified after deploy: both gateways healthy, tools/list returns all 8
tools across three namespaces with no partialFailure, and jobs__start_job
still mints through the affinity pin.

Images were already minimal and are untouched — the gateways are the
unmodified distroless release (47.2 MB), which is the claim the demo
page makes, and demo-tasks is already multi-stage distroless static at
4.08 MB.

Also corrects a comment that had gone false: sleepAfter was documented as
a backstop because the uptime monitor pinged every 5 minutes and kept
these warm. That monitor is undeployed, so the timeout alone now decides
how long an idle container stays on the bill. Left at 1h; shortening it
is the other half of this bill and a separate call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@blakebauman
blakebauman merged commit c00fae4 into main Aug 26, 2026
1 check passed
blakebauman added a commit that referenced this pull request Aug 26, 2026
Memory and disk bill on what is provisioned, for every awake second, so
with the uptime monitor undeployed this timeout is the only thing
bounding what an idle container costs. All three drop to 15m.

  demo         1h -> 15m
  enterprise   1h -> 15m
  demo-tasks   2h -> 15m

demo-tasks had the longest tail and the least reason for it. The 2h
predates the teardown and carries no stated rationale; the board is
ephemeral by design, demo jobs run for seconds, and the gateway forwards
every tasks call — so the upstream stays awake exactly as long as anyone
is actually using it, and there is no state here worth paying to keep
warm. A poll that does land on a cold start gets the 503-and-retry the
Worker already returns.

The cost is a ~1.7s cold start for the first visitor after a quiet
quarter hour, measured on lite in #46. That is a fair trade for a demo
people arrive at one at a time rather than in a steady stream, and the
enterprise gateway has more reason still: it sits behind a sign-in, so
it is idle more of the time than the demo is.

Verified after deploy: all three healthy, tools/list returns 8 tools
across three namespaces with no partialFailure, and jobs__start_job
still mints through the affinity pin.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant