diff --git a/.claude/CODEBASE.md b/.claude/CODEBASE.md index b315c4714..4b4f8712c 100644 --- a/.claude/CODEBASE.md +++ b/.claude/CODEBASE.md @@ -25,7 +25,7 @@ Key properties: | Crate | Location | Description | |---|---|---| | `fynd` | root (`src/`) | CLI binary and library crate: parses args, sets up observability, runs `FyndRPCBuilder`. `lib.rs` re-exports `fynd_core` and `fynd_rpc` as a single dependency | -| [`fynd-core`](../fynd-core/CLAUDE.md) | `fynd-core/` | Pure solving logic: algorithms, worker pools, graph, feed, derived data, encoding. No HTTP deps | +| [`fynd-core`](../fynd-core/CLAUDE.md) | `fynd-core/` | Pure solving logic: algorithms, worker pools, graph, feed, derived data, encoding, layered config. No HTTP deps | | [`fynd-rpc`](../fynd-rpc/CLAUDE.md) | `fynd-rpc/` | HTTP RPC server builder (Actix Web): API handlers, middleware, `FyndRPCBuilder` | | [`fynd-rpc-types`](../fynd-rpc-types/CLAUDE.md) | `fynd-rpc-types/` | Shared DTO types for the RPC API (request/response wire format) | | `fynd-test-fixtures` | `test-fixtures/` | Shared types for recorded-market test fixtures: `MarketRecording`, expected outputs, test scenarios. Not published | @@ -102,7 +102,8 @@ See `docs/ARCHITECTURE.md` for the full architecture diagram and detailed compon | `TYCHO_URL` | Tycho endpoint (chain-specific default) | | `HTTP_HOST` | HTTP bind address (default: `0.0.0.0`) | | `HTTP_PORT` | API port (default: `3000`) | -| `WORKER_POOLS_CONFIG` | Worker pools config file (default: `worker_pools.toml`) | +| `CONFIG_FILE` | Solver config file (default: `fynd.toml` if present) | +| `WORKER_POOLS_CONFIG` | Deprecated legacy pools-only config file; its pools override the config file's | | `BLOCKLIST_CONFIG` | Blocklist config file | | `RUST_LOG` | Tracing filter (e.g. `info,fynd=debug`) | | `METRICS_PORT` | Prometheus metrics server port (default: `9898`, requires `metrics` feature) | @@ -111,7 +112,7 @@ See `docs/ARCHITECTURE.md` for the full architecture diagram and detailed compon | Command | Purpose | |---|---| -| `serve` | Run the solver: Tycho feed + HTTP RPC server. Notable flags: `--enable-price-guard` (default `false`), `--partial-blocks` (enable flashblock/partial-block updates from Tycho stream) | +| `serve` | Run the solver: Tycho feed + HTTP RPC server. Notable flags: `--config-file` (default `fynd.toml`), `--enable-price-guard` (default `false`), `--partial-blocks` (enable flashblock/partial-block updates from Tycho stream) | | `openapi` | Print the OpenAPI spec JSON to stdout | | `derive-connector-tokens` | Derive and print connector token lists for configured protocols | @@ -119,7 +120,8 @@ See `docs/ARCHITECTURE.md` for the full architecture diagram and detailed compon | File | Purpose | |---|---| -| `worker_pools.toml` | Worker pool definitions: algorithm, num_workers, hop limits, timeout. Optional — binary falls back to embedded defaults if not found | +| `fynd.toml` | Full solver config: any subset of tuning fields + `[pools]`. Resolved field-by-field: CLI > file > embedded default (`fynd-core/src/config/default_config.toml`) | +| `worker_pools.toml` | Deprecated legacy pools-only file, still honored (its pools override the config file's) | | `blocklist.toml` | Component IDs to exclude from the Tycho stream. Optional — falls back to tycho-simulation defaults if not found | ## Testing diff --git a/docker-compose.yml b/docker-compose.yml index 8c83609fe..f032cc229 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -7,13 +7,13 @@ services: - "3000:3000" - "9898:9898" volumes: - - ./worker_pools.toml:/etc/fynd/worker_pools.toml:ro + - ./fynd.toml:/etc/fynd/fynd.toml:ro - ./blocklist.toml:/etc/fynd/blocklist.toml:ro environment: - RPC_URL=${RPC_URL} - TYCHO_API_KEY=${TYCHO_API_KEY:-} - TYCHO_URL=${TYCHO_URL:-tycho-fynd-ethereum.propellerheads.xyz} - - WORKER_POOLS_CONFIG=/etc/fynd/worker_pools.toml + - CONFIG_FILE=/etc/fynd/fynd.toml - BLOCKLIST_CONFIG=/etc/fynd/blocklist.toml - RUST_LOG=fynd=info - OTEL_EXPORTER_OTLP_ENDPOINT=http://tempo:4317 diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 1f507c325..679e86f83 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -172,7 +172,7 @@ Manages dedicated OS threads for CPU-bound route finding. Each pool has: * A bounded `TaskQueue` (via `async_channel`) * N `SolverWorker` instances on separate threads -Pools can use either a built-in algorithm by name (e.g., `"most_liquid"`) or a custom `Algorithm` implementation via `WorkerPoolBuilder::with_algorithm`. Pools are configured via `worker_pools.toml` for built-in algorithms, or programmatically via the builder for custom algorithms. Multiple pools can use the same algorithm with different parameters (e.g., fast 2-hop vs deep 3-hop). +Pools can use either a built-in algorithm by name (e.g., `"most_liquid"`) or a custom `Algorithm` implementation via `WorkerPoolBuilder::with_algorithm`. Pools are configured via the `[pools]` section of the config file (`fynd.toml`) for built-in algorithms, or programmatically via the builder for custom algorithms. Multiple pools can use the same algorithm with different parameters (e.g., fast 2-hop vs deep 3-hop). *** diff --git a/docs/algorithms/README.md b/docs/algorithms/README.md index 6a987b4c9..3a509f8bc 100644 --- a/docs/algorithms/README.md +++ b/docs/algorithms/README.md @@ -33,5 +33,5 @@ See [Architecture](../ARCHITECTURE.md) for the full system design and [Custom Al | **Approach** | Enumerate paths, score by heuristic, simulate top-N | Simulate every reachable edge, keep best amounts | Bellman-Ford based multi-path discovery + Frank-Wolfe split optimisation | | **Strengths** | Fast; good at common, high-liquidity pairs | Finds non-obvious routes; no heuristic blind spots | Reduces price impact by splitting flow across parallel paths | | **Weaknesses** | Path count explodes at high hop counts; heuristic can misjudge | Single path only; suboptimal for large trades | More simulation work per request; overkill for small trades | -| **Default config** | _(not in default `worker_pools.toml`)_ | 2 hops, 3 workers (see `worker_pools.toml`) | _(not in default `worker_pools.toml`)_ | +| **Default config** | _(not in default `fynd.toml`)_ | 2 hops, 3 workers (see `fynd.toml`) | _(not in default `fynd.toml`)_ | | **Derived data needs** | Spot prices + pool depths (scoring), token gas prices (gas ranking) | Token gas prices (optional, for gas-aware mode) | Token gas prices + spot prices (price impact, probe amount, gas cost) | diff --git a/docs/algorithms/bellman-ford.md b/docs/algorithms/bellman-ford.md index bf6d5f9a2..769d4eeb8 100644 --- a/docs/algorithms/bellman-ford.md +++ b/docs/algorithms/bellman-ford.md @@ -349,4 +349,4 @@ The Bellman-Ford routing approach in Fynd was inspired by the work of [János Ta | `fynd-core/src/algorithm/mod.rs` | `Algorithm` trait definition | | `fynd-core/src/graph/petgraph.rs` | Graph implementation (petgraph::StableDiGraph) | | `fynd-core/src/worker_pool/registry.rs` | Maps `"bellman_ford"` to `BellmanFordAlgorithm` | -| `worker_pools.toml` | Worker pool configuration | +| `fynd.toml` | Worker pool configuration | diff --git a/docs/algorithms/most-liquid.md b/docs/algorithms/most-liquid.md index 7a9e246b1..a0613a6c9 100644 --- a/docs/algorithms/most-liquid.md +++ b/docs/algorithms/most-liquid.md @@ -86,4 +86,4 @@ The path with the highest `net_output` wins. | `fynd-core/src/algorithm/mod.rs` | `Algorithm` trait definition | | `fynd-core/src/graph/petgraph.rs` | Graph implementation (petgraph::StableDiGraph) | | `fynd-core/src/worker_pool/registry.rs` | Maps `"most_liquid"` to `MostLiquidAlgorithm` | -| `worker_pools.toml` | Worker pool configuration | +| `fynd.toml` | Worker pool configuration | diff --git a/docs/algorithms/path-frank-wolfe.md b/docs/algorithms/path-frank-wolfe.md index 6428fa7fb..b300667ae 100644 --- a/docs/algorithms/path-frank-wolfe.md +++ b/docs/algorithms/path-frank-wolfe.md @@ -111,7 +111,7 @@ max_hops = 3 timeout_ms = 500 ``` -The PFW-specific tuning parameters are not currently exposed in `worker_pools.toml`; they use defaults: +The PFW-specific tuning parameters are not currently exposed in `fynd.toml`; they use defaults: | Parameter | Default | Description | | --- | --- | --- | @@ -149,4 +149,4 @@ The Frank-Wolfe loop checks elapsed time at the start of each iteration. If the | `fynd-core/src/algorithm/bellman_ford.rs` | Inner BF solver used for path discovery | | `fynd-core/src/algorithm/mod.rs` | `Algorithm` trait definition | | `fynd-core/src/worker_pool/registry.rs` | Maps `"path_frank_wolfe"` to `PathFrankWolfeAlgorithm` | -| `worker_pools.toml` | Worker pool configuration (add a `path_frank_wolfe` pool to enable) | +| `fynd.toml` | Worker pool configuration (add a `path_frank_wolfe` pool to enable) | diff --git a/docs/guides/server-configuration.md b/docs/guides/server-configuration.md index 72c4f4a60..475ecc8ec 100644 --- a/docs/guides/server-configuration.md +++ b/docs/guides/server-configuration.md @@ -86,7 +86,8 @@ Run `fynd serve --help` for the full list. | `--traded-n-days-ago` | — | `3` | Only include tokens traded within this many days. | | `--worker-router-timeout-ms` | — | `100` | Default solve timeout (ms) | | `--worker-router-min-responses` | — | `0` | Early return threshold (0 = wait for all pools) | -| `-w, --worker-pools-config` | `WORKER_POOLS_CONFIG` | `worker_pools.toml` | Worker pools config file path | +| `--config-file` | `CONFIG_FILE` | `fynd.toml` (if present) | TOML config file overriding the embedded defaults (see [Config file](#config-file-fyndtoml)). | +| `-w, --worker-pools-config` | `WORKER_POOLS_CONFIG` | `worker_pools.toml` (if present) | **Deprecated** — legacy pools-only config file; move the `[pools]` section into `fynd.toml`. Still honored: its pools override the config file's. | | `--blocklist-config` | `BLOCKLIST_CONFIG` | [tycho-simulation default](https://github.com/propeller-heads/tycho-simulation/blob/main/blocklist.toml) | Path to blocklist TOML config file. Components listed here are excluded from the Tycho stream. | | `--disable-tls` | — | `false` | Disable TLS for Tycho connection | | `--min-token-quality` | — | `100` | Minimum [token quality](https://docs.propellerheads.xyz/tycho/overview/concepts#token) filter | @@ -101,12 +102,17 @@ Run `fynd serve --help` for the full list. | `--price-guard-fail-on-token-price-not-found`| — | `false` | Reject quotes when no provider lists the token. | | `--metrics-port` | `METRICS_PORT` | `9898` | Port for the Prometheus metrics HTTP server. Requires the `metrics` feature (enabled by default). | -## Worker pools (`worker_pools.toml`) +## Config file (`fynd.toml`) + +Every solver-tuning flag above resolves field-by-field through three layers, highest priority +first: **CLI flags > config file > embedded defaults**. The config file may set any subset of +the fields (same names as the flags) plus the worker pools; `./fynd.toml` is picked up +automatically, or pass `--config-file`. Worker pools control solver thread count and routing strategies. The default config ships with one pool: ```toml -# worker_pools.toml +# fynd.toml [pools.bellman_ford_2_hops] algorithm = "bellman_ford" num_workers = 3 @@ -164,7 +170,7 @@ The command scores every token by pool count and outputs a ready-to-paste TOML s To use a custom config file: ```bash -fynd serve -w my_worker_pools.toml +fynd serve --config-file my_fynd.toml ``` ## Blocklist config diff --git a/fynd-core/CLAUDE.md b/fynd-core/CLAUDE.md index 5bc78b2e7..28c0177bb 100644 --- a/fynd-core/CLAUDE.md +++ b/fynd-core/CLAUDE.md @@ -9,6 +9,7 @@ applications. |-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `algorithm/` | `Algorithm` trait + built-in `MostLiquidAlgorithm`, `BellmanFordAlgorithm`, `PathFrankWolfeAlgorithm`. Pluggable via associated graph types. `AlgorithmConfig` shared by all | | `solver.rs` | `FyndBuilder` assembles the full pipeline (feed + gas + computations + pools + encoder + router). `Solver` runs it | +| `config/` | Layered solver config: `embedded_default()` parses `default_config.toml` (single source of truth for all tuning defaults) into a complete `Config`; `PartialConfig` layers overlay via `Config::apply` (CLI/file > embedded); `Config::validate` range-checks the result | | `worker_pool/` | `WorkerPool` manages dedicated OS threads. `SolverWorker` runs a prioritized select loop (shutdown > market events > derived events > tasks). `TaskQueue` is `async_channel`-based | | `worker_pool_router/` | `WorkerPoolRouter` fans out orders to all pools, ranks candidates by `amount_out_net_gas` descending; price guard (if enabled) validates in rank order; optionally encodes | | `feed/` | `TychoFeed` (WebSocket → MarketState), `GasPriceFetcher`, `MarketEvent` broadcasting, `ProtocolRegistry` | @@ -55,6 +56,10 @@ pub trait EdgeWeightUpdaterWithDerived { **`FyndBuilder`** (`solver.rs`): Assembles feed + gas + computations + pools + encoder + router. Returns a `Solver` that can `quote()` directly. For standalone (non-HTTP) use. +Defaults come from the embedded default config (`config/default_config.toml`); apply a resolved +`config::Config` in one call with `apply_config(&config)` (validates internally). Exception: the +worker router timeout defaults to a generous 10s standalone value. + Price guard methods: `price_guard_enabled(bool)`, `register_price_provider(Box)`, `add_default_price_providers()` (registers Binance WS + Hyperliquid providers). diff --git a/fynd-core/Cargo.toml b/fynd-core/Cargo.toml index 7e80c0c39..820a7fe0b 100644 --- a/fynd-core/Cargo.toml +++ b/fynd-core/Cargo.toml @@ -36,6 +36,7 @@ typetag.workspace = true tycho-execution.workspace = true reqwest.workspace = true tokio-tungstenite.workspace = true +toml.workspace = true alloy = { workspace = true, features = ["sol-types"] } [features] @@ -48,6 +49,5 @@ experimental = [] [dev-dependencies] rstest.workspace = true tempfile.workspace = true -toml.workspace = true fynd-test-fixtures.workspace = true tracing-subscriber.workspace = true diff --git a/fynd-core/src/config/default_config.toml b/fynd-core/src/config/default_config.toml new file mode 100644 index 000000000..9d94ce87f --- /dev/null +++ b/fynd-core/src/config/default_config.toml @@ -0,0 +1,27 @@ +# Embedded default solver configuration, compiled into the binary. +# +# This file is the single source of truth for all solver-tuning defaults: +# `FyndBuilder::new` seeds its initial values from it, and the fynd binary uses it as the +# base layer of config resolution (CLI flags > local config file > this file). +# +# `min_tvl` is intentionally absent: it falls back to a chain-specific default TVL +# threshold when no layer sets it. Every other field must be defined here — unit tests +# (`config::tests::test_embedded_default_is_complete_and_valid`) enforce completeness. + +tvl_buffer_ratio = 1.1 +min_token_quality = 100 +traded_n_days_ago = 3 +gas_refresh_interval_secs = 30 +reconnect_delay_secs = 5 +worker_router_timeout_ms = 100 +worker_router_min_responses = 0 +partial_blocks = false +protocols = ["all_onchain"] + +[pools.bellman_ford_2_hops] +algorithm = "bellman_ford" +num_workers = 3 +task_queue_capacity = 1000 +min_hops = 1 +max_hops = 2 +timeout_ms = 500 diff --git a/fynd-core/src/config/mod.rs b/fynd-core/src/config/mod.rs new file mode 100644 index 000000000..2ec8a1b9a --- /dev/null +++ b/fynd-core/src/config/mod.rs @@ -0,0 +1,464 @@ +//! Layered solver configuration. +//! +//! Every solver-tuning field resolves independently through three layers, highest priority +//! first: +//! +//! 1. **Explicit overrides** — lib builder setters or CLI flags +//! 2. **Local config file** — any subset of the fields, same schema as the embedded default +//! 3. **Embedded default** — `default_config.toml`, compiled into the binary +//! +//! The embedded default deserializes directly into a complete [`Config`] — every field +//! (except the chain-specific `min_tvl`) is required, so a gap between the struct and +//! `default_config.toml` cannot go unnoticed. The other layers each produce a +//! [`PartialConfig`] applied on top with [`Config::apply`], field by field, in ascending +//! priority order. The final result is validated as a whole ([`Config::validate`]), not +//! per layer; [`FyndBuilder::apply_config`](crate::solver::FyndBuilder::apply_config) +//! validates automatically. +//! +//! Collections (`protocols`, `pools`) merge atomically: a layer either sets the whole +//! list/map or nothing. +//! +//! # Example +//! +//! ```ignore +//! use fynd_core::config::{embedded_default, PartialConfig}; +//! +//! let overrides = PartialConfig { worker_router_timeout_ms: Some(50), ..Default::default() }; +//! let config = embedded_default() +//! .clone() +//! .apply(&PartialConfig::from_file("fynd.toml")?) +//! .apply(&overrides); +//! let builder = FyndBuilder::new(chain, tycho_url, rpc_url, config.protocols.clone(), min_tvl) +//! .apply_config(&config)?; // validates the config +//! ``` + +use std::{collections::HashMap, path::Path, sync::LazyLock}; + +use serde::Deserialize; +use tycho_simulation::tycho_common::models::{Chain, TvlThresholdTier}; + +use crate::solver::PoolConfig; + +/// The embedded default configuration, compiled into the binary. +const EMBEDDED_DEFAULT_TOML: &str = include_str!("default_config.toml"); + +/// A fully resolved solver configuration — no optional fields, ready for the engine. +/// +/// Start from [`embedded_default`], overlay [`PartialConfig`] layers with [`Config::apply`], +/// and consume via [`FyndBuilder::apply_config`](crate::solver::FyndBuilder::apply_config). +/// +/// Deserialization is only used for the embedded default and requires every field (except +/// `min_tvl`): adding a field here without updating `default_config.toml` fails parsing, +/// which unit tests catch. +#[derive(Debug, Clone, PartialEq, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Config { + /// Minimum TVL threshold in native token units (e.g. ETH). Components below this are + /// excluded from routing. `None` means the chain's default TVL threshold — the one + /// chain-specific value, which is why the embedded default leaves it unset. Resolve it + /// with [`Config::min_tvl_or_chain_default`]. + #[serde(default)] + pub min_tvl: Option, + /// Multiplier defining the lower hysteresis bound of the TVL filter (must be >= 1.0). + pub tvl_buffer_ratio: f64, + /// Minimum token quality score required for a token to be included in routing. + pub min_token_quality: i32, + /// Only include tokens traded within this many days. + pub traded_n_days_ago: u64, + /// How often the gas price is refreshed from the RPC node, in seconds. + pub gas_refresh_interval_secs: u64, + /// Delay before reconnecting to the Tycho feed after a disconnect, in seconds. + pub reconnect_delay_secs: u64, + /// Worker router timeout in milliseconds. + pub worker_router_timeout_ms: u64, + /// Minimum solver responses before early return (`0` = wait for all pools). + pub worker_router_min_responses: usize, + /// Enable partial block (flashblock) updates from the Tycho stream. + pub partial_blocks: bool, + /// Protocols to index (e.g. `"uniswap_v2"`). The fynd binary also accepts the + /// `"all_onchain"` placeholder, expanded to every on-chain protocol available from + /// Tycho RPC (the embedded default uses it). + pub protocols: Vec, + /// Worker pool definitions, keyed by pool name. + pub pools: HashMap, +} + +/// One layer's contribution to the configuration: every field is optional. +/// +/// All layers — builder/CLI overrides, the local config file, and the embedded default — +/// deserialize into this same type. Unknown keys are ignored, so a config file written for +/// a newer binary still parses on an older one, which only applies the fields it knows. +#[derive(Debug, Clone, Default, PartialEq, Deserialize)] +pub struct PartialConfig { + /// See [`Config::min_tvl`]. + pub min_tvl: Option, + /// See [`Config::tvl_buffer_ratio`]. + pub tvl_buffer_ratio: Option, + /// See [`Config::min_token_quality`]. + pub min_token_quality: Option, + /// See [`Config::traded_n_days_ago`]. + pub traded_n_days_ago: Option, + /// See [`Config::gas_refresh_interval_secs`]. + pub gas_refresh_interval_secs: Option, + /// See [`Config::reconnect_delay_secs`]. + pub reconnect_delay_secs: Option, + /// See [`Config::worker_router_timeout_ms`]. + pub worker_router_timeout_ms: Option, + /// See [`Config::worker_router_min_responses`]. + pub worker_router_min_responses: Option, + /// See [`Config::partial_blocks`]. + pub partial_blocks: Option, + /// See [`Config::protocols`]. Merged atomically: a layer either sets the whole list or + /// nothing. + pub protocols: Option>, + /// See [`Config::pools`]. Merged atomically: a layer either sets the whole map or + /// nothing. + pub pools: Option>, +} + +impl PartialConfig { + /// Parses a config layer from a TOML string. + /// + /// `source` names the origin (e.g. a file path) for error messages. Unknown fields are + /// ignored (see the type-level docs). + pub fn from_toml_str(raw: &str, source: &str) -> Result { + toml::from_str(raw) + .map_err(|e| ConfigError::Parse { context: source.to_string(), source: e }) + } + + /// Reads and parses a config layer from a TOML file. See [`Self::from_toml_str`]. + pub fn from_file(path: impl AsRef) -> Result { + let path = path.as_ref(); + let raw = std::fs::read_to_string(path) + .map_err(|e| ConfigError::ReadFile { path: path.display().to_string(), source: e })?; + Self::from_toml_str(&raw, &path.display().to_string()) + } +} + +/// The embedded default configuration, parsed once on first use. +static EMBEDDED_DEFAULT: LazyLock = LazyLock::new(|| { + toml::from_str(EMBEDDED_DEFAULT_TOML) + .expect("embedded default_config.toml is a complete, valid Config; checked by unit tests") +}); + +/// Returns the embedded default configuration (`default_config.toml`), parsed once and +/// cached. +/// +/// This is the single source of truth for all solver-tuning defaults. The TOML +/// deserializes directly into a complete [`Config`]; every field is required except the +/// chain-specific `min_tvl` (see [`Config::min_tvl`]). +pub fn embedded_default() -> &'static Config { + &EMBEDDED_DEFAULT +} + +impl Config { + /// Returns `min_tvl`, falling back to `chain`'s default TVL threshold when unset. + pub fn min_tvl_or_chain_default(&self, chain: Chain) -> f64 { + self.min_tvl + .unwrap_or_else(|| chain.default_tvl_threshold(TvlThresholdTier::Low)) + } + + /// Applies a partial layer on top of this config: fields the layer sets replace the + /// current values; collections (`protocols`, `pools`) are replaced atomically. + #[must_use] + pub fn apply(mut self, partial: &PartialConfig) -> Self { + // Exhaustive destructuring (no `..`): adding a field to `PartialConfig` without + // handling it here is a compile error. + let PartialConfig { + min_tvl, + tvl_buffer_ratio, + min_token_quality, + traded_n_days_ago, + gas_refresh_interval_secs, + reconnect_delay_secs, + worker_router_timeout_ms, + worker_router_min_responses, + partial_blocks, + protocols, + pools, + } = partial; + if let Some(value) = min_tvl { + self.min_tvl = Some(*value); + } + if let Some(value) = tvl_buffer_ratio { + self.tvl_buffer_ratio = *value; + } + if let Some(value) = min_token_quality { + self.min_token_quality = *value; + } + if let Some(value) = traded_n_days_ago { + self.traded_n_days_ago = *value; + } + if let Some(value) = gas_refresh_interval_secs { + self.gas_refresh_interval_secs = *value; + } + if let Some(value) = reconnect_delay_secs { + self.reconnect_delay_secs = *value; + } + if let Some(value) = worker_router_timeout_ms { + self.worker_router_timeout_ms = *value; + } + if let Some(value) = worker_router_min_responses { + self.worker_router_min_responses = *value; + } + if let Some(value) = partial_blocks { + self.partial_blocks = *value; + } + if let Some(value) = protocols { + self.protocols = value.clone(); + } + if let Some(value) = pools { + self.pools = value.clone(); + } + self + } +} + +/// Errors from parsing, resolving, or validating the layered configuration. +#[derive(Debug, thiserror::Error)] +pub enum ConfigError { + /// A config file could not be read from disk. + #[error("failed to read config file {path}: {source}")] + ReadFile { + /// Path that could not be read. + path: String, + /// Underlying I/O error. + #[source] + source: std::io::Error, + }, + /// A layer failed to deserialize (malformed TOML). + #[error("failed to parse {context}: {source}")] + Parse { + /// Origin of the payload (file path or "embedded"). + context: String, + /// Underlying TOML error. + #[source] + source: toml::de::Error, + }, + /// The final resolved configuration failed validation. + #[error("invalid resolved config: {reasons}")] + Validation { + /// All validation failures, joined with "; ". + reasons: String, + }, +} + +impl Config { + /// Range-checks the fully resolved configuration. + /// + /// Call this after the final [`apply`](Self::apply) — validation is meant for the + /// resolved result, not individual layers, so a layer may set a value that only + /// becomes invalid (or valid) in combination with lower layers. + /// [`FyndBuilder::apply_config`](crate::solver::FyndBuilder::apply_config) calls it + /// automatically. + /// + /// # Errors + /// + /// Returns [`ConfigError::Validation`] listing every violated range check. + pub fn validate(&self) -> Result<(), ConfigError> { + let issues = validate(self); + if issues.is_empty() { + return Ok(()); + } + Err(ConfigError::Validation { reasons: issues.join("; ") }) + } +} + +/// Range-checks the final resolved configuration. Returns one message per violation. +fn validate(config: &Config) -> Vec { + let mut issues = Vec::new(); + if let Some(min_tvl) = config.min_tvl { + if !min_tvl.is_finite() || min_tvl < 0.0 { + issues.push(format!("min_tvl must be a non-negative finite number, got {min_tvl}")); + } + } + if !config.tvl_buffer_ratio.is_finite() || config.tvl_buffer_ratio < 1.0 { + issues.push(format!("tvl_buffer_ratio must be >= 1.0, got {}", config.tvl_buffer_ratio)); + } + if config.gas_refresh_interval_secs == 0 { + issues.push("gas_refresh_interval_secs must be > 0".to_string()); + } + if config.reconnect_delay_secs == 0 { + issues.push("reconnect_delay_secs must be > 0".to_string()); + } + if config.worker_router_timeout_ms == 0 { + issues.push("worker_router_timeout_ms must be > 0".to_string()); + } + if config.pools.is_empty() { + issues.push("at least one worker pool must be configured".to_string()); + } + for (name, pool) in &config.pools { + if pool.num_workers() == 0 { + issues.push(format!("pool '{name}': num_workers must be > 0")); + } + if pool.task_queue_capacity() == 0 { + issues.push(format!("pool '{name}': task_queue_capacity must be > 0")); + } + if pool.min_hops() == 0 { + issues.push(format!("pool '{name}': min_hops must be >= 1")); + } + if pool.min_hops() > pool.max_hops() { + issues.push(format!( + "pool '{name}': min_hops ({}) must not exceed max_hops ({})", + pool.min_hops(), + pool.max_hops() + )); + } + if pool.timeout_ms() == 0 { + issues.push(format!("pool '{name}': timeout_ms must be > 0")); + } + } + issues +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_embedded_default_is_complete_and_valid() { + let config = embedded_default(); + config + .validate() + .expect("embedded default failed validation"); + assert!(config + .pools + .contains_key("bellman_ford_2_hops")); + // min_tvl is the one chain-specific field: unset until resolved against a chain. + assert_eq!(config.min_tvl, None); + assert_eq!( + config.min_tvl_or_chain_default(Chain::Ethereum), + Chain::Ethereum.default_tvl_threshold(TvlThresholdTier::Low) + ); + } + + #[test] + fn test_resolve_precedence() { + let overrides = PartialConfig { + worker_router_timeout_ms: Some(50), + min_token_quality: Some(90), + ..PartialConfig::default() + }; + let local_file = PartialConfig { + min_token_quality: Some(80), + traded_n_days_ago: Some(7), + ..PartialConfig::default() + }; + // Ascending priority: embedded default, then local config file, then overrides. + let config = embedded_default() + .clone() + .apply(&local_file) + .apply(&overrides); + config + .validate() + .expect("validation errored"); + + assert_eq!(config.worker_router_timeout_ms, 50); + assert_eq!(config.min_token_quality, 90); + assert_eq!(config.traded_n_days_ago, 7); + assert_eq!(config.tvl_buffer_ratio, 1.1); + } + + #[test] + fn test_resolve_collections_merge_atomically() { + let mut pools = HashMap::new(); + pools.insert("custom".to_string(), PoolConfig::new("most_liquid")); + let local_file = PartialConfig { + pools: Some(pools), + protocols: Some(vec!["curve".to_string()]), + ..PartialConfig::default() + }; + let config = embedded_default() + .clone() + .apply(&local_file); + // The whole map/list is replaced, not merged entry by entry. + assert_eq!(config.pools.len(), 1); + assert_eq!(config.pools["custom"].algorithm(), "most_liquid"); + assert_eq!(config.protocols, vec!["curve"]); + } + + #[test] + fn test_min_tvl_override() { + let overrides = PartialConfig { min_tvl: Some(42.0), ..PartialConfig::default() }; + let config = embedded_default() + .clone() + .apply(&overrides); + assert_eq!(config.min_tvl, Some(42.0)); + } + + #[test] + fn test_partial_from_toml_ignores_unknown_fields() { + // Permissive parsing: config files written for newer binaries (with fields this + // version doesn't know) must still parse, applying only the known fields. + let partial = + PartialConfig::from_toml_str("min_tvl = 5.0\nfield_from_the_future = 1", "test.toml") + .expect("parse errored"); + assert_eq!(partial.min_tvl, Some(5.0)); + assert_eq!(partial.min_token_quality, None); + } + + #[test] + fn test_partial_from_toml_parses_fields_and_pools() { + let partial = PartialConfig::from_toml_str( + r#" + min_tvl = 20.0 + worker_router_timeout_ms = 150 + + [pools.quick] + algorithm = "most_liquid" + max_hops = 2 + "#, + "test.toml", + ) + .expect("parse errored"); + assert_eq!(partial.min_tvl, Some(20.0)); + assert_eq!(partial.worker_router_timeout_ms, Some(150)); + let pools = partial.pools.expect("pools missing"); + assert_eq!(pools["quick"].algorithm(), "most_liquid"); + assert_eq!(pools["quick"].max_hops(), 2); + } + + #[test] + fn test_partial_from_toml_rejects_malformed_toml() { + let error = PartialConfig::from_toml_str("min_tvl = [", "broken.toml").unwrap_err(); + assert!(matches!(error, ConfigError::Parse { .. })); + assert!(error + .to_string() + .contains("broken.toml")); + } + + #[test] + fn test_validation_runs_on_resolved_config() { + let overrides = PartialConfig { tvl_buffer_ratio: Some(0.5), ..PartialConfig::default() }; + let error = embedded_default() + .clone() + .apply(&overrides) + .validate() + .unwrap_err(); + assert!(matches!(error, ConfigError::Validation { .. })); + assert!(error + .to_string() + .contains("tvl_buffer_ratio")); + } + + #[test] + fn test_validation_pool_checks() { + let mut pools = HashMap::new(); + pools.insert( + "bad".to_string(), + PoolConfig::new("bellman_ford") + .with_num_workers(0) + .with_min_hops(3) + .with_max_hops(2), + ); + let overrides = PartialConfig { pools: Some(pools), ..PartialConfig::default() }; + let error = embedded_default() + .clone() + .apply(&overrides) + .validate() + .unwrap_err(); + let message = error.to_string(); + assert!(message.contains("num_workers")); + assert!(message.contains("min_hops")); + } +} diff --git a/fynd-core/src/lib.rs b/fynd-core/src/lib.rs index 4bf9af7fc..5078d4136 100644 --- a/fynd-core/src/lib.rs +++ b/fynd-core/src/lib.rs @@ -24,6 +24,7 @@ /// [`algorithm::BellmanFordAlgorithm`], [`PathFrankWolfeAlgorithm`], and the /// pluggable [`Algorithm`] trait. pub mod algorithm; +pub mod config; /// Derived data computations: spot prices, pool depths, and gas prices. pub mod derived; /// Encodes solved routes into ABI-encoded on-chain calldata via Tycho's router contracts. diff --git a/fynd-core/src/solver.rs b/fynd-core/src/solver.rs index 750223721..57cdbafc4 100644 --- a/fynd-core/src/solver.rs +++ b/fynd-core/src/solver.rs @@ -51,29 +51,17 @@ use crate::{ Algorithm, Quote, QuoteRequest, SolveError, }; -/// Default values for [`FyndBuilder`] configuration and [`PoolConfig`] deserialization. +/// Default values for [`PoolConfig`] deserialization and non-config-managed intervals. /// -/// These are the single source of truth for all tunable defaults. Downstream -/// crates (e.g. `fynd-rpc`) should re-export or reference these rather than -/// redeclaring their own copies. +/// Solver-tuning defaults (TVL filters, refresh intervals, router responses, …) are NOT +/// defined here: their single source of truth is the embedded default config +/// (`config/default_config.toml`), which [`FyndBuilder::new`] resolves for its initial +/// values. pub mod defaults { use std::time::Duration; - /// Minimum token quality score required for a token to be included in routing. - pub const MIN_TOKEN_QUALITY: i32 = 100; - /// Maximum age (in days) of trading history required for a token to be considered liquid. - pub const TRADED_N_DAYS_AGO: u64 = 3; - /// Multiplier applied to a pool's TVL when estimating available liquidity. - pub const TVL_BUFFER_RATIO: f64 = 1.1; - /// How often the gas price is refreshed from the RPC node. - pub const GAS_REFRESH_INTERVAL: Duration = Duration::from_secs(30); /// How often router fees are refreshed from the on-chain FeeCalculator contract. pub const ROUTER_FEE_REFRESH_INTERVAL: Duration = Duration::from_secs(300); - /// Delay before reconnecting to the Tycho feed after a disconnect. - pub const RECONNECT_DELAY: Duration = Duration::from_secs(5); - /// Minimum number of solver pool responses required before returning a quote (`0` = wait for - /// all). - pub const ROUTER_MIN_RESPONSES: usize = 0; /// Capacity of the task queue for each worker pool. pub const POOL_TASK_QUEUE_CAPACITY: usize = 1000; /// Minimum number of hops allowed in a route. @@ -127,7 +115,7 @@ fn parse_connector_tokens( /// Per-pool configuration for [`FyndBuilder::add_pool`]. #[must_use] -#[derive(Debug, Clone, Serialize, Deserialize)] +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct PoolConfig { /// Algorithm name for this pool (e.g., `"most_liquid"`). algorithm: String, @@ -283,6 +271,9 @@ pub enum SolverBuildError { /// A pool referenced an algorithm name that is not registered. #[error(transparent)] UnknownAlgorithm(#[from] UnknownAlgorithmError), + /// The configuration passed to [`FyndBuilder::apply_config`] failed validation. + #[error(transparent)] + Config(#[from] crate::config::ConfigError), /// No native gas token is defined for the requested chain. #[error("gas token not configured for chain")] GasToken, @@ -391,6 +382,11 @@ pub struct FyndBuilder { impl FyndBuilder { /// Creates a new builder with the required parameters. + /// + /// Solver-tuning defaults come from the embedded default config + /// (`config/default_config.toml`) — the single source of truth. Exception: the worker + /// router timeout defaults to a generous standalone value rather than the embedded + /// HTTP-service-oriented one. pub fn new( chain: Chain, tycho_url: impl Into, @@ -398,6 +394,7 @@ impl FyndBuilder { protocols: Vec, min_tvl: f64, ) -> Self { + let embedded = crate::config::embedded_default(); Self { chain, tycho_url: tycho_url.into(), @@ -406,15 +403,15 @@ impl FyndBuilder { min_tvl, tycho_api_key: None, tycho_use_tls: DEFAULT_TYCHO_USE_TLS, - min_token_quality: defaults::MIN_TOKEN_QUALITY, - traded_n_days_ago: defaults::TRADED_N_DAYS_AGO, - tvl_buffer_ratio: defaults::TVL_BUFFER_RATIO, - gas_refresh_interval: defaults::GAS_REFRESH_INTERVAL, - reconnect_delay: defaults::RECONNECT_DELAY, + min_token_quality: embedded.min_token_quality, + traded_n_days_ago: embedded.traded_n_days_ago, + tvl_buffer_ratio: embedded.tvl_buffer_ratio, + gas_refresh_interval: Duration::from_secs(embedded.gas_refresh_interval_secs), + reconnect_delay: Duration::from_secs(embedded.reconnect_delay_secs), blocklisted_components: HashSet::new(), - partial_blocks: false, + partial_blocks: embedded.partial_blocks, router_timeout: DEFAULT_ROUTER_TIMEOUT, - router_min_responses: defaults::ROUTER_MIN_RESPONSES, + router_min_responses: embedded.worker_router_min_responses, encoder: None, pools: Vec::new(), price_guard_enabled: false, @@ -423,6 +420,41 @@ impl FyndBuilder { } } + /// Applies a resolved [`Config`](crate::config::Config) to this builder, validating it + /// first. + /// + /// Sets every solver-tuning field (including the protocol list) and adds the config's + /// worker pools (in addition to any pools already registered). Setter calls after this + /// override the config's values. + /// + /// # Errors + /// + /// Returns [`SolverBuildError::Config`] if the config fails validation, or + /// [`SolverBuildError::AlgorithmConfig`] if a pool's `connector_tokens` contains a + /// malformed hex address. + pub fn apply_config( + mut self, + config: &crate::config::Config, + ) -> Result { + config.validate()?; + let chain = self.chain; + self = self + .min_tvl(config.min_tvl_or_chain_default(chain)) + .tvl_buffer_ratio(config.tvl_buffer_ratio) + .min_token_quality(config.min_token_quality) + .traded_n_days_ago(config.traded_n_days_ago) + .gas_refresh_interval(Duration::from_secs(config.gas_refresh_interval_secs)) + .reconnect_delay(Duration::from_secs(config.reconnect_delay_secs)) + .worker_router_timeout(Duration::from_millis(config.worker_router_timeout_ms)) + .worker_router_min_responses(config.worker_router_min_responses) + .partial_blocks(config.partial_blocks) + .protocols(config.protocols.clone()); + for (name, pool_config) in &config.pools { + self = self.add_pool(name, pool_config)?; + } + Ok(self) + } + /// The blockchain this builder is configured for. pub fn chain(&self) -> Chain { self.chain @@ -440,6 +472,12 @@ impl FyndBuilder { self } + /// Overrides the protocol list set in [`FyndBuilder::new`]. + pub fn protocols(mut self, protocols: Vec) -> Self { + self.protocols = protocols; + self + } + /// Enables or disables TLS for the Tycho WebSocket connection (default: `true`). pub fn tycho_use_tls(mut self, use_tls: bool) -> Self { self.tycho_use_tls = use_tls; @@ -1216,7 +1254,8 @@ impl Solver { )); let router_config = WorkerPoolRouterConfig::default() .with_timeout(Duration::from_millis(max_timeout_ms.max(5000))) - .with_min_responses(defaults::ROUTER_MIN_RESPONSES); + // Replay waits for every pool before answering (deterministic tests). + .with_min_responses(0); let router = WorkerPoolRouter::new(solver_pool_handles, router_config, encoder); // Trigger derived data computation diff --git a/fynd-core/tests/integration/README.md b/fynd-core/tests/integration/README.md index 30a1990c8..268e0feca 100644 --- a/fynd-core/tests/integration/README.md +++ b/fynd-core/tests/integration/README.md @@ -39,7 +39,7 @@ Same recording + same code = same derived data. A mismatch means code changed ho derived data is computed. Re-record if the change is intentional. **Timing violations**: -Timing depends on hardware. The threshold derives from `worker_pools.toml` max timeout. +Timing depends on hardware. The threshold derives from the `fynd.toml` pools' max timeout. CI runners may be slower — timing tests use a 3x multiplier. ## Fixtures diff --git a/fynd-core/tests/integration/harness.rs b/fynd-core/tests/integration/harness.rs index d48a049af..d6b3698f2 100644 --- a/fynd-core/tests/integration/harness.rs +++ b/fynd-core/tests/integration/harness.rs @@ -70,6 +70,6 @@ impl TestHarness { } fn load_pools() -> HashMap { - let toml_content = include_str!("../../../worker_pools.toml"); - fynd_test_fixtures::parse_pools_toml(toml_content).expect("failed to parse worker_pools.toml") + let toml_content = include_str!("../../../fynd.toml"); + fynd_test_fixtures::parse_pools_toml(toml_content).expect("failed to parse fynd.toml pools") } diff --git a/fynd-core/tests/integration/timing_tests.rs b/fynd-core/tests/integration/timing_tests.rs index 6fbea322d..81d1ef369 100644 --- a/fynd-core/tests/integration/timing_tests.rs +++ b/fynd-core/tests/integration/timing_tests.rs @@ -7,9 +7,9 @@ fn expected_path() -> std::path::PathBuf { } fn max_pool_timeout_ms() -> u64 { - let toml_content = include_str!("../../../worker_pools.toml"); + let toml_content = include_str!("../../../fynd.toml"); let pools = fynd_test_fixtures::parse_pools_toml(toml_content) - .expect("failed to parse worker_pools.toml"); + .expect("failed to parse fynd.toml pools"); pools .values() .map(|p| p.timeout_ms()) diff --git a/fynd-rpc/CLAUDE.md b/fynd-rpc/CLAUDE.md index 55c3ba30f..82b58a8d8 100644 --- a/fynd-rpc/CLAUDE.md +++ b/fynd-rpc/CLAUDE.md @@ -49,8 +49,9 @@ The builder calls `FyndBuilder::build()` → `Solver::into_parts()` → wraps th ## Defaults -The `config::defaults` module re-exports `fynd-core::solver::defaults::*` and adds HTTP-specific -constants: +Solver-tuning defaults live in fynd-core's embedded default config +(`fynd-core/src/config/default_config.toml`); `WorkerPoolsConfig::builtin_default()` returns its +pools. The `config::defaults` module holds only HTTP-specific values: - `HTTP_HOST = "0.0.0.0"`, `HTTP_PORT = 3000` - `WORKER_ROUTER_TIMEOUT_MS = 100` (tighter than fynd-core's 10s standalone default) - `default_tycho_url(chain)` maps chain names to hosted endpoints diff --git a/fynd-rpc/src/config.rs b/fynd-rpc/src/config.rs index 6a611b941..fb12d3e03 100644 --- a/fynd-rpc/src/config.rs +++ b/fynd-rpc/src/config.rs @@ -8,23 +8,6 @@ use anyhow::{Context, Result}; pub use fynd_core::PoolConfig; use serde::{Deserialize, Serialize}; -/// The default worker pools configuration embedded at compile time. -/// -/// Used as a fallback when the default `worker_pools.toml` path is not found at runtime, -/// so the binary works out-of-the-box without a config file (e.g. `cargo install`, Docker). -/// -/// Keep in sync with the repo-root `worker_pools.toml` (the user-facing example config). -/// Cannot use `include_str!` here because `cargo publish` verifies the crate in isolation, -/// and the file lives outside the `fynd-rpc` package directory. -const DEFAULT_WORKER_POOLS_TOML: &str = r#" -[pools.bellman_ford_2_hops] -algorithm = "bellman_ford" -num_workers = 3 -task_queue_capacity = 1000 -max_hops = 2 -timeout_ms = 500 -"#; - /// Worker pools configuration loaded from TOML file. #[derive(Debug, Clone, Serialize, Deserialize)] pub struct WorkerPoolsConfig { @@ -48,11 +31,15 @@ impl WorkerPoolsConfig { self.pools } - /// Returns the built-in default configuration embedded in the binary. - /// - /// This is the repo-root `worker_pools.toml` baked in at compile time. + /// Returns the built-in default configuration embedded in the binary: the pools of the + /// embedded default config (`fynd-core`'s `default_config.toml`), the single source of + /// truth for solver-tuning defaults. pub fn builtin_default() -> Self { - toml::from_str(DEFAULT_WORKER_POOLS_TOML).expect("built-in worker_pools.toml is valid TOML") + Self { + pools: fynd_core::config::embedded_default() + .pools + .clone(), + } } /// Load worker pools configuration from a TOML file. @@ -155,13 +142,10 @@ mod tests { } /// Default values for all `fynd-rpc` configuration parameters. +/// +/// Solver-tuning defaults live in `fynd-core`'s embedded default config +/// (`config/default_config.toml`); this module only holds HTTP-specific values. pub mod defaults { - // Re-export shared defaults from fynd-core as the single source of truth. - pub use fynd_core::solver::defaults::{ - GAS_REFRESH_INTERVAL, MIN_TOKEN_QUALITY, RECONNECT_DELAY, ROUTER_MIN_RESPONSES, - TRADED_N_DAYS_AGO, TVL_BUFFER_RATIO, - }; - /// Default HTTP bind host (`"0.0.0.0"` — all interfaces). pub const HTTP_HOST: &str = "0.0.0.0"; /// Default HTTP port (`3000`). diff --git a/worker_pools.toml b/fynd.toml similarity index 64% rename from worker_pools.toml rename to fynd.toml index d3f90c091..b24018920 100644 --- a/worker_pools.toml +++ b/fynd.toml @@ -1,10 +1,32 @@ -# Worker pools configuration file +# Fynd solver configuration file (user-facing example). # -# NOTE: This file is also inlined as the built-in default in fynd-rpc/src/config.rs -# (DEFAULT_WORKER_POOLS_TOML). That constant is used as a fallback when this file is -# not found at runtime (e.g. `cargo install`, Docker). Keep the two in sync. +# Loaded automatically from ./fynd.toml, or from the path given via --config-file / +# CONFIG_FILE. Any subset of the fields may be set; every field resolves independently: +# CLI flags > this file > the embedded default config +# (fynd-core/src/config/default_config.toml — the single source of truth for defaults). +# +# The legacy pools-only worker_pools.toml is still honored for backward compatibility +# (via --worker-pools-config / WORKER_POOLS_CONFIG, or ./worker_pools.toml when no other +# layer sets pools). + +# --- Solver tuning (defaults shown; uncomment to override) --- +# +# Minimum TVL threshold in native token units (e.g. ETH); defaults to a chain-specific +# value when unset. +#min_tvl = 10.0 +#tvl_buffer_ratio = 1.1 +#min_token_quality = 100 +#traded_n_days_ago = 3 +#gas_refresh_interval_secs = 30 +#reconnect_delay_secs = 5 +#worker_router_timeout_ms = 100 +#worker_router_min_responses = 0 +#partial_blocks = false +# "all_onchain" expands to every on-chain protocol available from Tycho RPC and can be +# combined with explicit entries, e.g. ["all_onchain", "rfq:bebop"]. +#protocols = ["all_onchain"] -# Worker pools +# --- Worker pools (setting any pool replaces the embedded default pools entirely) --- [pools.bellman_ford_2_hops] algorithm = "bellman_ford" num_workers = 3 diff --git a/src/cli.rs b/src/cli.rs index c92d66fbe..96afe8fe9 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -1,13 +1,21 @@ use std::path::PathBuf; use clap::Parser; -use fynd_rpc::config::defaults; +use fynd_core::config::{embedded_default, PartialConfig}; +use fynd_rpc::config::{defaults, WorkerPoolsConfig}; #[cfg(feature = "metrics")] pub(crate) const METRICS_PORT: u16 = 9898; use crate::commands::derive_connector_tokens::DeriveConnectorTokensArgs; +/// Builds help text for a config-layered flag, appending the real default value pulled +/// from the embedded default config (used when neither the CLI nor a config file sets the +/// field). +fn cfg_help(text: &str, default: impl std::fmt::Display) -> String { + format!("{text} [default: {default}]") +} + /// Fynd - High-performance DEX solver built on Tycho /// /// Finds optimal swap routes across multiple protocols using real-time market data. @@ -30,12 +38,22 @@ pub enum Commands { } /// Arguments for the `serve` subcommand. +/// +/// Solver-tuning flags resolve field-by-field through three layers, highest priority first: +/// explicit CLI flags, the local config file (`--config-file`, default `fynd.toml` if +/// present), and the default config embedded in the binary. #[derive(clap::Args, PartialEq, Debug)] pub struct ServeArgs { /// Target chain (e.g. Ethereum) #[arg(short, long, default_value = "Ethereum")] pub chain: String, + /// Path to a local TOML config file overriding the embedded defaults. Any subset of + /// the config fields, same schema as the embedded default config. + /// When omitted, ./fynd.toml is used if present. + #[arg(long, env)] + pub config_file: Option, + /// HTTP host (e.g. 0.0.0.0) #[arg(long, default_value = defaults::HTTP_HOST, env)] pub http_host: String, @@ -61,51 +79,64 @@ pub struct ServeArgs { pub rpc_url: Option, /// List of protocols to index (comma-separated, e.g., uniswap_v2,uniswap_v3). - /// If omitted, all on-chain protocols are fetched from Tycho RPC. - /// Use "all_onchain" to fetch all on-chain protocols and combine with explicit entries, - /// e.g., --protocols all_onchain,rfq:bebop. - #[arg(short, long, value_delimiter = ',', value_name = "PROTO1,PROTO2")] + #[arg(short, long, value_delimiter = ',', value_name = "PROTO1,PROTO2", + help = cfg_help( + "List of protocols to index (comma-separated). \"all_onchain\" expands to all \ + on-chain protocols fetched from Tycho RPC and can be combined with explicit \ + entries, e.g. all_onchain,rfq:bebop", + embedded_default().protocols.join(","), + ))] pub protocols: Vec, - /// Minimum TVL threshold in native token (e.g. ETH). Components below this threshold will be - /// removed from the market data. Defaults to a chain-specific value if not set. + /// Minimum TVL threshold in native token (e.g. ETH). Components below this threshold + /// will be removed from the market data. Defaults to a chain-specific value when no + /// config layer sets it. #[arg(long)] pub min_tvl: Option, /// TVL buffer ratio. - /// Used to avoid fluctuations caused by components hovering around a single threshold. - /// Default is 1.1 (10% buffer). For example, if the minimum TVL is 10 ETH, components are - /// added when TVL >= 10 ETH and removed when TVL drops below 10 / 1.1 ≈ 9.09 ETH. - #[arg(long, default_value_t = defaults::TVL_BUFFER_RATIO)] - pub tvl_buffer_ratio: f64, + #[arg(long, + help = cfg_help( + "TVL buffer ratio: avoids fluctuations from components hovering around a single \ + threshold. With ratio 1.1 and minimum TVL 10 ETH, components are added when \ + TVL >= 10 ETH and removed below 10 / 1.1 ≈ 9.09 ETH", + embedded_default().tvl_buffer_ratio, + ))] + pub tvl_buffer_ratio: Option, /// Minimum token quality filter. - #[arg(long, default_value_t = defaults::MIN_TOKEN_QUALITY)] - pub min_token_quality: i32, + #[arg(long, help = cfg_help("Minimum token quality filter", embedded_default().min_token_quality))] + pub min_token_quality: Option, /// Only include tokens traded within this many days. - #[arg(long, default_value_t = defaults::TRADED_N_DAYS_AGO)] - pub traded_n_days_ago: u64, - - /// Gas price refresh interval in seconds - #[arg(long, default_value_t = defaults::GAS_REFRESH_INTERVAL.as_secs())] - pub gas_refresh_interval_secs: u64, - - /// Reconnect delay on connection failure in seconds - #[arg(long, default_value_t = defaults::RECONNECT_DELAY.as_secs())] - pub reconnect_delay_secs: u64, - - /// Worker router timeout in milliseconds - #[arg(long, default_value_t = defaults::WORKER_ROUTER_TIMEOUT_MS)] - pub worker_router_timeout_ms: u64, - - /// Minimum solver responses before early return (0 = wait for all) - #[arg(long, default_value_t = defaults::ROUTER_MIN_RESPONSES)] - pub worker_router_min_responses: usize, - - /// Path to worker pools TOML config file - #[arg(short, long, env, default_value = "worker_pools.toml")] - pub worker_pools_config: PathBuf, + #[arg(long, help = cfg_help("Only include tokens traded within this many days", embedded_default().traded_n_days_ago))] + pub traded_n_days_ago: Option, + + /// Gas price refresh interval in seconds. + #[arg(long, help = cfg_help("Gas price refresh interval in seconds", embedded_default().gas_refresh_interval_secs))] + pub gas_refresh_interval_secs: Option, + + /// Reconnect delay on connection failure in seconds. + #[arg(long, help = cfg_help("Reconnect delay on connection failure in seconds", embedded_default().reconnect_delay_secs))] + pub reconnect_delay_secs: Option, + + /// Worker router timeout in milliseconds. + #[arg(long, help = cfg_help("Worker router timeout in milliseconds", embedded_default().worker_router_timeout_ms))] + pub worker_router_timeout_ms: Option, + + /// Minimum solver responses before early return (0 = wait for all). + #[arg(long, + help = cfg_help( + "Minimum solver responses before early return (0 = wait for all)", + embedded_default().worker_router_min_responses, + ))] + pub worker_router_min_responses: Option, + + /// Path to a legacy worker pools TOML config file; overrides the pools defined by + /// every other config layer. When omitted, ./worker_pools.toml is used if present and + /// no other layer sets pools. + #[arg(short, long, env)] + pub worker_pools_config: Option, /// Path to blocklist TOML config file. Components listed here are excluded from the /// Tycho stream. @@ -118,9 +149,13 @@ pub struct ServeArgs { pub gas_price_stale_threshold_secs: Option, /// Enable partial block (flashblock) updates from the Tycho stream. - /// When enabled, pool state updates arrive mid-block rather than only at finalization, - /// reducing latency. Only applies to on-chain protocols. - #[arg(long)] + #[arg(long, + help = cfg_help( + "Enable partial block (flashblock) updates from the Tycho stream: pool state \ + updates arrive mid-block rather than only at finalization, reducing latency. \ + Only applies to on-chain protocols", + embedded_default().partial_blocks, + ))] pub partial_blocks: bool, /// Enable price guard validation against external price sources. @@ -134,6 +169,40 @@ pub struct ServeArgs { pub metrics_port: u16, } +impl ServeArgs { + /// Builds the explicit-overrides config layer from the flags the user actually set; + /// unset flags stay `None` so the lower config layers supply them. + /// + /// A worker pools file passed via `--worker-pools-config` is an explicit override for + /// the whole `pools` section. + /// + /// # Errors + /// + /// Fails when the worker pools file cannot be read or parsed. + pub fn explicit_config(&self) -> anyhow::Result { + let pools = self + .worker_pools_config + .as_deref() + .map(WorkerPoolsConfig::load_from_file) + .transpose()? + .map(WorkerPoolsConfig::into_pools); + Ok(PartialConfig { + min_tvl: self.min_tvl, + tvl_buffer_ratio: self.tvl_buffer_ratio, + min_token_quality: self.min_token_quality, + traded_n_days_ago: self.traded_n_days_ago, + gas_refresh_interval_secs: self.gas_refresh_interval_secs, + reconnect_delay_secs: self.reconnect_delay_secs, + worker_router_timeout_ms: self.worker_router_timeout_ms, + worker_router_min_responses: self.worker_router_min_responses, + partial_blocks: self.partial_blocks.then_some(true), + // An empty --protocols means "not set"; the lower config layers supply the list. + protocols: (!self.protocols.is_empty()).then(|| self.protocols.clone()), + pools, + }) + } +} + #[cfg(test)] mod cli_tests { use super::*; @@ -175,7 +244,7 @@ mod cli_tests { assert_eq!(args.tycho_url, Some("wss://custom.tycho.url".to_string())); assert_eq!(args.protocols, vec!["uniswap_v2", "uniswap_v3"]); assert_eq!(args.min_tvl, Some(20.0)); - assert_eq!(args.worker_pools_config, PathBuf::from("new_worker_pools.toml")); + assert_eq!(args.worker_pools_config, Some(PathBuf::from("new_worker_pools.toml"))); assert_eq!(args.blocklist_config, None); } @@ -187,6 +256,8 @@ mod cli_tests { std::env::remove_var("TYCHO_URL"); std::env::remove_var("HTTP_HOST"); std::env::remove_var("HTTP_PORT"); + std::env::remove_var("CONFIG_FILE"); + std::env::remove_var("WORKER_POOLS_CONFIG"); let cli = Cli::try_parse_from(vec!["fynd", "serve"]).expect("parse errored"); let Commands::Serve(args) = cli.command else { @@ -199,12 +270,15 @@ mod cli_tests { assert_eq!(args.rpc_url, None); assert_eq!(args.tycho_url, None); assert!(args.protocols.is_empty()); + // Solver-tuning flags default to None: the config layers supply the values. + assert_eq!(args.config_file, None); assert_eq!(args.min_tvl, None); - assert_eq!(args.tvl_buffer_ratio, 1.1); - assert_eq!(args.gas_refresh_interval_secs, 30); - assert_eq!(args.reconnect_delay_secs, 5); - assert_eq!(args.worker_router_timeout_ms, 100); - assert_eq!(args.worker_router_min_responses, 0); + assert_eq!(args.tvl_buffer_ratio, None); + assert_eq!(args.gas_refresh_interval_secs, None); + assert_eq!(args.reconnect_delay_secs, None); + assert_eq!(args.worker_router_timeout_ms, None); + assert_eq!(args.worker_router_min_responses, None); + assert_eq!(args.worker_pools_config, None); assert_eq!(args.blocklist_config, None); assert!(!args.partial_blocks); #[cfg(feature = "metrics")] @@ -212,14 +286,29 @@ mod cli_tests { } #[test] - fn test_arg_parsing_default_worker_pools() { - let cli = Cli::try_parse_from(vec!["fynd", "serve", "--tycho-api-key", "test-key"]) - .expect("parse errored"); - + fn test_explicit_config_only_set_flags() { + let cli = Cli::try_parse_from(vec![ + "fynd", + "serve", + "--worker-router-timeout-ms", + "42", + "--partial-blocks", + "--protocols", + "uniswap_v2", + ]) + .expect("parse errored"); let Commands::Serve(args) = cli.command else { panic!("expected Serve command"); }; - assert_eq!(args.worker_pools_config, PathBuf::from("worker_pools.toml")); + let overrides = args + .explicit_config() + .expect("explicit config errored"); + assert_eq!(overrides.worker_router_timeout_ms, Some(42)); + assert_eq!(overrides.partial_blocks, Some(true)); + assert_eq!(overrides.protocols, Some(vec!["uniswap_v2".to_string()])); + assert_eq!(overrides.min_tvl, None); + assert_eq!(overrides.pools, None); + assert_eq!(overrides.tvl_buffer_ratio, None); } #[test] diff --git a/src/main.rs b/src/main.rs index 3175d80f0..83601be1f 100644 --- a/src/main.rs +++ b/src/main.rs @@ -29,12 +29,13 @@ //! //! See `fynd --help` for all available options. -use std::time::Duration; +use std::{path::Path, time::Duration}; #[cfg(feature = "metrics")] use actix_web::{web, App, HttpResponse, HttpServer, Responder}; use anyhow::anyhow; use clap::Parser; +use fynd_core::config::{embedded_default, PartialConfig}; use fynd_rpc::{ builder::FyndRPCBuilder, config::{defaults, BlocklistConfig, WorkerPoolsConfig}, @@ -56,7 +57,7 @@ use tokio::{ }; use tracing::{error, info, warn}; use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt, EnvFilter}; -use tycho_simulation::{tycho_common::models::TvlThresholdTier, utils::default_blocklist}; +use tycho_simulation::utils::default_blocklist; fn main() -> Result<(), anyhow::Error> { let cli = Cli::parse(); @@ -217,61 +218,114 @@ fn resolve_rpc_url(chain: &str, override_url: Option<&str>) -> Result Result { - // Load worker pools config, falling back to the built-in defaults when the default path is - // absent (e.g. `cargo install`, Docker). Custom paths that don't exist still fail fast. - let default_path = std::path::Path::new("worker_pools.toml"); - let pools_config = - if args.worker_pools_config == default_path && !args.worker_pools_config.exists() { - warn!( - "worker_pools.toml not found; using built-in defaults. \ - Set --worker-pools-config or WORKER_POOLS_CONFIG to use a custom config." - ); - WorkerPoolsConfig::builtin_default() - } else { - WorkerPoolsConfig::load_from_file(&args.worker_pools_config).map_err(|e| { - SolverError::SetupError(format!("failed to load worker pools config: {}", e)) +/// Default local config file probed in the working directory when `--config-file` is not +/// passed. +const DEFAULT_CONFIG_PATH: &str = "fynd.toml"; + +/// Legacy pools-only config file, still honored: it supplies the `pools` section at +/// local-file priority when no higher layer sets pools. +const LEGACY_WORKER_POOLS_PATH: &str = "worker_pools.toml"; + +/// Resolves the layered solver config: CLI flags > local config file > embedded default. +/// +/// Local files fail fast when explicitly passed or malformed — a broken local setup should +/// not start silently misconfigured. +fn resolve_solver_config(args: &cli::ServeArgs) -> Result { + if args.worker_pools_config.is_some() { + warn!( + "--worker-pools-config is deprecated and will be removed soon; move the [pools] \ + section into a config file passed via --config-file" + ); + } + let overrides = args + .explicit_config() + .map_err(|e| SolverError::SetupError(format!("failed to build config overrides: {e:#}")))?; + + let mut local_file = match &args.config_file { + Some(path) => Some( + PartialConfig::from_file(path) + .map_err(|e| SolverError::SetupError(format!("failed to load config file: {e}")))?, + ), + None => { + let default_path = Path::new(DEFAULT_CONFIG_PATH); + if default_path.exists() { + info!("{DEFAULT_CONFIG_PATH} found; using it as the local config file layer"); + Some(PartialConfig::from_file(default_path).map_err(|e| { + SolverError::SetupError(format!("failed to load config file: {e}")) + })?) + } else { + None + } + } + }; + + // The legacy worker_pools.toml keeps working and, to not break existing clients, its + // pools take priority over the config file's (only explicit CLI overrides beat it). + let legacy_pools_path = Path::new(LEGACY_WORKER_POOLS_PATH); + if overrides.pools.is_none() && legacy_pools_path.exists() { + warn!( + "{LEGACY_WORKER_POOLS_PATH} is deprecated and will be removed soon; move its \ + [pools] section into {DEFAULT_CONFIG_PATH}. For now its pools override the \ + config file's" + ); + let pools = WorkerPoolsConfig::load_from_file(legacy_pools_path) + .map_err(|e| { + SolverError::SetupError(format!("failed to load worker pools config: {e:#}")) })? - }; + .into_pools(); + local_file + .get_or_insert_with(PartialConfig::default) + .pools = Some(pools); + } - // Parse chain + // Ascending priority: embedded default, then the local config file, then CLI overrides. + let config = embedded_default() + .clone() + .apply(&local_file.unwrap_or_default()) + .apply(&overrides); + config + .validate() + .map_err(|e| SolverError::SetupError(format!("failed to resolve solver config: {e}")))?; + Ok(config) +} + +/// Sets up the solver (resolves config, parses chain, builds solver). +/// Returns setup errors if any step fails. +async fn setup_solver(args: &cli::ServeArgs) -> Result { let chain = parse_chain(&args.chain) .map_err(|e| SolverError::SetupError(format!("failed to parse chain: {}", e)))?; + let config = resolve_solver_config(args)?; + info!(?config, "solver config resolved"); + let tycho_url = resolve_tycho_url(&args.chain, args.tycho_url.as_deref())?; let rpc_url = resolve_rpc_url(&args.chain, args.rpc_url.as_deref())?; - let min_tvl = args - .min_tvl - .unwrap_or_else(|| chain.default_tvl_threshold(TvlThresholdTier::Low)); let protocols = resolve_protocols( &tycho_url, args.tycho_api_key.as_deref(), !args.disable_tls, chain, - &args.protocols, + &config.protocols, ) .await .map_err(|e| SolverError::SetupError(format!("failed to resolve protocols: {e}")))?; info!(?protocols, "starting with {} protocol(s)", protocols.len()); - // Build solver with all fields from CLI let mut builder = - FyndRPCBuilder::new(chain, pools_config.into_pools(), tycho_url, rpc_url, protocols) + FyndRPCBuilder::new(chain, config.pools.clone(), tycho_url, rpc_url, protocols) .map_err(|e| SolverError::SetupError(format!("invalid pool configuration: {e}")))? .http_host(args.http_host.clone()) .http_port(args.http_port) - .min_tvl(min_tvl) - .min_token_quality(args.min_token_quality) - .traded_n_days_ago(args.traded_n_days_ago) - .tvl_buffer_ratio(args.tvl_buffer_ratio) - .gas_refresh_interval(Duration::from_secs(args.gas_refresh_interval_secs)) - .reconnect_delay(Duration::from_secs(args.reconnect_delay_secs)) - .worker_router_timeout(Duration::from_millis(args.worker_router_timeout_ms)) - .worker_router_min_responses(args.worker_router_min_responses) + .min_tvl(config.min_tvl_or_chain_default(chain)) + .min_token_quality(config.min_token_quality) + .traded_n_days_ago(config.traded_n_days_ago) + .tvl_buffer_ratio(config.tvl_buffer_ratio) + .gas_refresh_interval(Duration::from_secs(config.gas_refresh_interval_secs)) + .reconnect_delay(Duration::from_secs(config.reconnect_delay_secs)) + .worker_router_timeout(Duration::from_millis(config.worker_router_timeout_ms)) + .worker_router_min_responses(config.worker_router_min_responses) .gas_price_stale_threshold( args.gas_price_stale_threshold_secs .map(Duration::from_secs), @@ -293,7 +347,7 @@ async fn setup_solver(args: &cli::ServeArgs) -> Result anyhow::Result> { diff --git a/tools/CLAUDE.md b/tools/CLAUDE.md index 89f2b77a6..1ea1aeac8 100644 --- a/tools/CLAUDE.md +++ b/tools/CLAUDE.md @@ -76,7 +76,7 @@ pipeline (`Solver::from_recording`, `test-utils` feature) to generate `expected_ the integration tests in `fynd-core/tests/integration/`. Shared fixture types live in the `fynd-test-fixtures` crate. Worker pool configuration comes from -the production `worker_pools.toml`; its SHA-256 is stored in the recording metadata so tests can +the repo `fynd.toml` pools; its SHA-256 is stored in the recording metadata so tests can detect drift. VM-backed protocol states (e.g. `vm:*` pools) cannot be serialized and are skipped. See [`tools/record-market/README.md`](record-market/README.md) for usage. diff --git a/tools/fynd-gas-audit/README.md b/tools/fynd-gas-audit/README.md index be5942966..a397600c3 100644 --- a/tools/fynd-gas-audit/README.md +++ b/tools/fynd-gas-audit/README.md @@ -33,7 +33,7 @@ tool quantifies the bias. sampled trade resolves to a single-hop swap. Use it when you want to isolate per-protocol gas accuracy without multi-hop overhead confounding the numbers. Drop the `-w` flag to fall back to Fynd's default - `worker_pools.toml`, which allows multi-hop routing. + the default pool config (`fynd.toml`), which allows multi-hop routing. 2. Run the audit in a second terminal: ```bash RPC_URL=https://reth-ethereum.ithaca.xyz/rpc \ @@ -58,7 +58,7 @@ each time you want a fresh, independent draw from the 10k dataset. ## Findings from the 2026-05-27 run (default routing, n=1000) -This run used Fynd's default `worker_pools.toml` (multi-hop enabled), so the +This run used Fynd's default pool config (multi-hop enabled), so the sample includes both single-hop and sequential routes. Mainnet gas price at the time was 0.10 gwei. Absolute ETH numbers scale linearly with gas price — the **relative** figures are the ones that matter. diff --git a/tools/record-market/README.md b/tools/record-market/README.md index e1bac954a..3da85d704 100644 --- a/tools/record-market/README.md +++ b/tools/record-market/README.md @@ -46,7 +46,7 @@ can compute gas cost deductions without a live RPC connection. ## When to Re-record Re-run the recording tool when: -- Algorithm or pool configuration changes (`worker_pools.toml`) +- Algorithm or pool configuration changes (`fynd.toml` pools) - Solver code changes that intentionally improve quote quality - Fixtures are stale (the tool stores a timestamp; tests warn if > 7 days old) diff --git a/tools/record-market/src/main.rs b/tools/record-market/src/main.rs index 7e50a63cb..e053cda67 100644 --- a/tools/record-market/src/main.rs +++ b/tools/record-market/src/main.rs @@ -100,7 +100,7 @@ async fn main() -> anyhow::Result<()> { // during serialization won't be present in the deserialized version). let recording = fynd_test_fixtures::read_recording(&recording_path)?; - let pools_toml = include_str!("../../../worker_pools.toml"); + let pools_toml = include_str!("../../../fynd.toml"); let pairs_path = PathBuf::from(format!("fynd-core/tests/fixtures/pairs/{}.json", recording.metadata.chain)); let pairs_json = std::fs::read_to_string(&pairs_path).map_err(|e| { diff --git a/tools/record-market/src/recorder.rs b/tools/record-market/src/recorder.rs index cc06cdea8..a66c1a301 100644 --- a/tools/record-market/src/recorder.rs +++ b/tools/record-market/src/recorder.rs @@ -120,7 +120,7 @@ pub async fn record_market(opts: &RecordingOptions) -> anyhow::Result