Skip to content

feat(config): add remote config layer fetched from S3 - #328

Open
zizou0x wants to merge 1 commit into
zz/eng-6176-config-filefrom
zz/eng-6176-s3-config-layer
Open

feat(config): add remote config layer fetched from S3#328
zizou0x wants to merge 1 commit into
zz/eng-6176-config-filefrom
zz/eng-6176-s3-config-layer

Conversation

@zizou0x

@zizou0x zizou0x commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #320 (config centralization).

What

Adds a remote config layer to the resolution stack. Per-chain tuned values are
pulled from S3 at startup and applied right above the embedded defaults:

CLI flags > local config file (fynd.toml) > remote config (S3) > embedded default

Default URL (per chain, overridable via --remote-config-url / REMOTE_CONFIG_URL;
--no-remote-config disables):

https://s3.eu-central-1.amazonaws.com/repo.propellerheads-propellerheads/fynd/presets/{chain}/latest.toml

The payload is a PartialConfig in the same schema as fynd.toml. Empty payloads
are published for all six supported chains (verified end to end against the live
bucket, including a tuned-value round trip).

Fail-safe behavior

The remote layer can never take a solver down:

  • Fetch errors and timeouts log a warning and resolve without the remote layer;
    the caller bounds the fetch with a timeout (binary: 2 s)
  • Bounded retries with backoff for transient failures only (connect errors, 5xx)
  • Response size capped at 256 KiB before parsing
  • Non-2xx statuses outside 4xx/5xx are rejected explicitly (S3 answers an
    unfollowable 301 on region mismatches)
  • Unknown fields in the payload are ignored (forward compatibility); a payload
    whose pools reference an unknown algorithm is rejected wholesale so outdated
    binaries fall back instead of failing at solver build
  • No panic paths reachable from network input
  • The final resolved config is validated as a whole, regardless of source

Library API

  • Config::apply_remote(url, timeout) — async layer application; returns self
    unchanged on any fetch problem
  • config::get_default(chain) — embedded + remote with default URL and built-in
    2 s budget, in one call
  • config::remote::fetch_remote_config(url) — raw fetch for custom error handling

Covered by mock-server tests: valid payload, 5xx retry, 4xx no-retry, unfollowable
redirect, oversized body, garbage TOML, unknown algorithm, and fallback-on-timeout.

🤖 Generated with Claude Code

Per-chain tuned values are pulled at startup and applied between the
embedded defaults and the local config file. Fetching is fail-safe:
bounded retries, size cap, explicit redirect handling, and any error
falls back to the lower layers with a warning. Lib users get
config::get_default(chain) for embedded + remote in one call.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@brunoguerios brunoguerios left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one!
Fits in nicely on top of the previous PR 👏
LGTM ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants