Skip to content

fix(buffer): take the spool caps from config or the volume limit - #170

Merged
catinspace-au merged 1 commit into
mainfrom
fix/archiver-spool-caps-from-volume
Nov 11, 2026
Merged

catinspace-au merged 1 commit into
mainfrom
fix/archiver-spool-caps-from-volume

Conversation

@catinspace-au

Copy link
Copy Markdown
Contributor

The archiver's staged-bytes cap, startup free-space floor and quarantine cap were literals sized for the contract's 10Gi spool volume. Raising the volume in the chart left the caps at 8, 1 and 1 GiB.

  • New keys: buffer.staged_bytes_cap, buffer.min_free_spool_bytes, buffer.quarantine_bytes_cap, each in bytes.
  • Unset, they are 8/9, 1/9 and 1/9 of the spool budget. The budget is 90% of the size limit in SCALO_WRITABLE_PATH_LIMITS for the volume holding spool_dir.
  • 10Gi gives the old 8/1/1 GiB. 20Gi gives 16/2/2.
  • With no limit declared (docker, a bare host) they stay 8/1/1 GiB. The caps take no share of the host disk because statvfs on an emptyDir reads the node.
  • Validation rejects a zero staged_bytes_cap.
  • Docs: README, internals.md and the config schema files.

Tests cover env resolution, the no-limit fallback, config overrides, volume-path matching, rejection of a zero cap, and the quarantine cap from Staging::open. The keys and argument did not exist before this change, so these tests cannot fail against the old code.

The archiver's chart may not set SCALO_WRITABLE_PATH_LIMITS. The scalo-service chart sets it from scalo 2.14.23. The archiver's library pin is 2.14.20, so it may be unset until the pin moves.

The staged-bytes cap, startup free-space floor and quarantine cap were fixed at 8, 1 and 1 GiB. Only a comment tied them to the contract's 10Gi spool volume. Bigger volumes kept those values.

Each is now a buffer key: staged_bytes_cap, min_free_spool_bytes and quarantine_bytes_cap. Unset, they default to 8/9, 1/9 and 1/9 of 90% of the volume size limit in SCALO_WRITABLE_PATH_LIMITS. A 10Gi limit still gives 8/1/1 GiB. With no declared limit the defaults stay 8/1/1 GiB.

A zero staged_bytes_cap is refused. Staging::open takes the quarantine cap.
@catinspace-au
catinspace-au marked this pull request as ready for review November 11, 2026 06:39
@catinspace-au
catinspace-au merged commit 10bcb8f into main Nov 11, 2026
23 checks passed
@catinspace-au
catinspace-au deleted the fix/archiver-spool-caps-from-volume branch November 11, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant