vllm-project · Zheng-Lu · Sep 18, 2026
@@ -951,8 +951,8 @@ declaration until they have a complete handler and execution path.
     (`CodexNamespaceHandler`), and `tool_search.rs` (`ToolSearchHandler`). Their calls
     are returned for the client to resolve; the gateway does not execute them.
   - **Gateway-owned / built-in** tools implement both traits: see `web_search/mod.rs`
-    (`WebSearchHandler`, backed by the configured `WebSearchProvider` in `web_search/you.rs`
-    or `web_search/brave.rs`) and `mcp/handler.rs` (`McpHandler`, backed
+    (`WebSearchHandler`, backed by the configured `WebSearchProvider` in `web_search/you.rs`,
+    `web_search/brave.rs`, or `web_search/tavily.rs`) and `mcp/handler.rs` (`McpHandler`, backed
     by `mcp/client.rs`'s MCP protocol client and `mcp/pool.rs`'s connection pool). They
     have no client translator association because the gateway owns their execution and
     public lifecycle.

@@ -20,6 +20,22 @@ All notable changes to Agentic API are documented here.
   `crawl_timeout`, and `boost_domains` arguments are ignored. Rejected credentials and HTTP 429 responses fail the
   `web_search_call` without an automatic retry, naming the key variable or the upstream `Retry-After` value and never
   echoing the secret. Each Brave `metadata[]` entry carries `"provider": "brave"`.
+- Added Tavily as a selectable backend for the gateway-owned `web_search` tool (#327, Phase 3 of #291). Select it
+  with `AGENTIC_WEB_SEARCH_PROVIDER=tavily` or `[web_search] provider = "tavily"` and supply `TAVILY_API_KEY`; the
+  endpoint defaults to `https://api.tavily.com` and can be overridden with `AGENTIC_WEB_SEARCH_BASE_URL` or
+  `[web_search] base_url`. Each query is one `POST /search` with a JSON body and a bearer token; the key is never
+  placed in the body. `allowed_domains` / `blocked_domains` and the model's `include_domains` / `exclude_domains` are
+  forwarded to Tavily's native `include_domains` / `exclude_domains` and re-checked client-side, `count` is clamped
+  to Tavily's maximum of 20, `freshness` maps to `time_range` or to `start_date` / `end_date` widened by one day on
+  each side because Tavily's bounds are exclusive, `language` keeps Tavily's documented compound tags (`zh-cn`) and
+  otherwise reduces to its primary subtag, `safesearch` maps to the boolean `safe_search`, and `country` plus the
+  You.com-specific
+  `livecrawl`, `livecrawl_formats`, `crawl_timeout`, and `boost_domains` arguments are ignored. Results fill
+  `results.web` with `published_date` as `page_age`; `results.news` stays empty because a second news search per
+  query would double credit usage. Rejected credentials, HTTP 429, and Tavily's 432/433 plan-limit statuses fail the
+  `web_search_call` without an automatic retry, naming the key variable or the upstream `Retry-After` value and never
+  echoing the secret. Each Tavily `metadata[]` entry carries `"provider": "tavily"`, Tavily's `request_id` as
+  `search_uuid`, and its `response_time` as `latency`. Tavily inherits the gateway concurrency limit.
 - Added `[web_search] max_concurrent_queries` and `AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES` to cap concurrent
   provider requests inside one batched search. Brave defaults to `1` for its free-plan rate limit; You.com keeps
   inheriting `max_concurrent_gateway_calls`. The effective ceiling is the smallest of the gateway limit, this
@@ -45,6 +61,11 @@ All notable changes to Agentic API are documented here.
   uses it. With `provider` unset, You.com behavior, configuration, and model-facing output are unchanged; a generated
   `config.toml` now records `provider = "you"` and leaves `api_key_env` unset so provider switches select the matching
   default credential variable.
+- `WebSearchProviderKind` gains a `Tavily` variant; `WebSearchProviderKind::ALL` is now a `&'static [Self]` slice
+  listing all three providers, so adding a provider no longer changes its type; and
+  `WebSearchHandler::from_config` builds the Tavily provider for it (#327). The shared `null_as_default` and
+  `read_response_limited` helpers moved from `tool/web_search/mod.rs` to `tool/web_search/provider.rs`; both were and
+  remain crate-private, so no public API changed.
 
 ### Fixed
 

@@ -54,7 +54,7 @@ flowchart LR
 ## ✨ Key Features
 
 - 🔄 **Stateful conversations**: the server manages history via `previous_response_id`. No client-side message tracking, no replaying full transcripts.
-- 🛠️ **Server-side tool execution**: an explicit tool-ownership model (gateway / client / provider) decides exactly what runs where. Web search ships today via [You.com](https://you.com) or [Brave Search](https://brave.com/search/api/), and the model executes multi-step tool chains automatically.
+- 🛠️ **Server-side tool execution**: an explicit tool-ownership model (gateway / client / provider) decides exactly what runs where. Web search ships today via [You.com](https://you.com), [Brave Search](https://brave.com/search/api/), or [Tavily](https://tavily.com), and the model executes multi-step tool chains automatically.
 - 📡 **Every transport**: non-streaming HTTP, server-sent events for token streaming, and full **WebSocket** support for interactive clients.
 - 🧰 **Codex-ready**: accepts Codex-shaped Responses traffic out of the box, preserving the tool declarations and response item shapes Codex depends on.
 - 🏃 **Background execution**: fire-and-forget requests that keep processing server-side.
@@ -193,6 +193,13 @@ AGENTIC_WEB_SEARCH_PROVIDER=brave BRAVE_API_KEY=<your-brave-api-key> \
   cargo run -p agentic-server -- --llm-api-base http://0.0.0.0:5050
 ```
 
+Or [Tavily](https://tavily.com), a search API built for LLM agents with native domain filtering:
+
+```bash
+AGENTIC_WEB_SEARCH_PROVIDER=tavily TAVILY_API_KEY=<your-tavily-api-key> \
+  cargo run -p agentic-server -- --llm-api-base http://0.0.0.0:5050
+```
+
 The default database is `~/.agentic-api/agentic_api.db`, so running an installed binary does not create state in the
 current directory. Set `AGENTIC_API_HOME` to an absolute directory to move both the default database and user
 configuration, or set `DATABASE_URL`/`--db-url` to select a different database.
@@ -226,12 +233,12 @@ llm_api_base = "http://127.0.0.1:5050"
 # database_url = "postgresql://agentic-api@localhost/agentic_api"
 
 [web_search]
-# Search backend for the gateway-owned web_search tool: "you" (default) or "brave".
+# Search backend for the gateway-owned web_search tool: "you" (default), "brave", or "tavily".
 provider = "you"
 base_url = "https://api.ydc-index.io"
 api_key_env = "YOU_API_KEY"
 # Concurrent provider requests inside one batched web-search call; unset uses
-# the provider default (Brave: 1, You.com: max_concurrent_gateway_calls).
+# the provider default (Brave: 1; You.com and Tavily: max_concurrent_gateway_calls).
 # max_concurrent_queries = 1
 
 [mcp]
@@ -304,9 +311,9 @@ every provider.
 | Setting | Environment variable | `config.toml` key | Default |
 | :--- | :--- | :--- | :--- |
 | Provider | `AGENTIC_WEB_SEARCH_PROVIDER` | `[web_search] provider` | `you` |
-| API key | variable named by `api_key_env` | `[web_search] api_key_env` | `YOU_API_KEY` / `BRAVE_API_KEY` |
-| Endpoint | `AGENTIC_WEB_SEARCH_BASE_URL` (or `YOU_API_BASE_URL` for You.com) | `[web_search] base_url` | none for You.com; `https://api.search.brave.com` for Brave |
-| Concurrent queries | `AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES` | `[web_search] max_concurrent_queries` | You.com inherits `max_concurrent_gateway_calls`; Brave `1` |
+| API key | variable named by `api_key_env` | `[web_search] api_key_env` | `YOU_API_KEY` / `BRAVE_API_KEY` / `TAVILY_API_KEY` |
+| Endpoint | `AGENTIC_WEB_SEARCH_BASE_URL` (or `YOU_API_BASE_URL` for You.com) | `[web_search] base_url` | none for You.com; `https://api.search.brave.com` for Brave; `https://api.tavily.com` for Tavily |
+| Concurrent queries | `AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES` | `[web_search] max_concurrent_queries` | You.com and Tavily inherit `max_concurrent_gateway_calls`; Brave `1` |
 
 **You.com** (`provider = "you"`) is the default and behaves exactly as before: domain filters are applied by the
 provider, `count` accepts 1–100, and the You.com-specific `livecrawl`, `livecrawl_formats`, `crawl_timeout`, and
@@ -328,14 +335,36 @@ the shared tool contract to Brave:
   automatic retry and reports the upstream `Retry-After` value; a cap lowers, but cannot eliminate, 429s on a
   per-second quota.
 
-If you switch an existing deployment to Brave, update `api_key_env` if an older `config.toml` pins it (or
+**Tavily** (`provider = "tavily"`) needs only `TAVILY_API_KEY`; its [plans](https://tavily.com) are metered in
+credits, and each query costs one credit. Requests are `POST` bodies against `https://api.tavily.com/search` with the
+key sent as a bearer token. The gateway adapts the shared tool contract to Tavily:
+
+- `allowed_domains` / `blocked_domains` (and the model's `include_domains` / `exclude_domains`) are forwarded to
+  Tavily's native `include_domains` / `exclude_domains` and re-checked by the gateway on the response as defense in
+  depth, so a filtered search can return fewer than `count` results.
+- `count` is clamped to Tavily's maximum of 20 (`max_results`); `freshness` maps to `time_range` (`day`, `week`,
+  `month`, `year`) or to a `start_date` / `end_date` pair widened by one day on each side, since Tavily's bounds are
+  exclusive and the gateway's range is inclusive; `language` keeps Tavily's documented compound tags (`zh-CN` →
+  `zh-cn`) and otherwise reduces to its primary subtag (`en-GB` → `en`); `safesearch` becomes Tavily's boolean
+  `safe_search` (anything but `off` enables it).
+- Every query is one `topic: "general"` search, so all hits land in `results.web` and `results.news` is always empty;
+  a second news search per query would double the credits spent. `page_age` carries Tavily's `published_date`.
+- `country` is ignored (Tavily expects full country names rather than ISO codes), as are the You.com-specific
+  arguments above (all logged at debug level).
+- Each per-query `metadata[]` entry carries `"provider": "tavily"` plus Tavily's `request_id` as `search_uuid` and
+  its `response_time` as `latency`.
+- Batched queries inherit the gateway concurrency limit. A rate-limited request (HTTP 429) fails that
+  `web_search_call` without an automatic retry and reports the upstream `Retry-After` value; Tavily's plan-limit
+  statuses (432, 433) fail the same way without a retry.
+
+If you switch an existing deployment to Brave or Tavily, update `api_key_env` if an older `config.toml` pins it (or
 remove it) and drop a You.com `base_url`; a mismatched key variable is reported in the failed `web_search_call`
 message. Example:
 
 ```toml
 [web_search]
-provider = "brave"
-api_key_env = "BRAVE_API_KEY"
+provider = "tavily"
+api_key_env = "TAVILY_API_KEY"
 ```
 
 Restrict the file to the service account (for example, `chmod 600 ~/.agentic-api/config.toml`), especially if you add
@@ -437,7 +466,8 @@ Claude Code's own tools (Bash, Edit, Read, …) stay **client-owned** — Claude
 
 Current Claude Code versions declare Anthropic's native `web_search_20250305` server tool. Agentic API translates that
 declaration for the upstream model and executes the resulting search server-side against the configured search backend
-(You.com or Brave Search, see [Web search providers](#web-search-providers)); no MCP server or tool alias is required:
+(You.com, Brave Search, or Tavily, see [Web search providers](#web-search-providers)); no MCP server or tool alias is
+required:
 
 ```bash
 YOU_API_KEY=<you.com-key> YOU_API_BASE_URL=<you.com-base-url> \

@@ -101,18 +101,20 @@ pub enum WebSearchProviderKind {
     #[default]
     You,
     Brave,
+    Tavily,
 }
 
 impl WebSearchProviderKind {
     /// Every selectable provider, in the order operator-facing messages list them.
-    pub const ALL: [Self; 2] = [Self::You, Self::Brave];
+    pub const ALL: &'static [Self] = &[Self::You, Self::Brave, Self::Tavily];
 
     /// Environment variable that conventionally carries this provider's API key.
     #[must_use]
     pub const fn default_api_key_env(self) -> &'static str {
         match self {
             Self::You => "YOU_API_KEY",
             Self::Brave => "BRAVE_API_KEY",
+            Self::Tavily => "TAVILY_API_KEY",
         }
     }
 
@@ -124,16 +126,18 @@ impl WebSearchProviderKind {
         match self {
             Self::You => None,
             Self::Brave => Some("https://api.search.brave.com"),
+            Self::Tavily => Some("https://api.tavily.com"),
         }
     }
 
     /// Provider-imposed default ceiling on concurrent search requests. `None`
     /// inherits the gateway-wide limit. Brave's free plan allows roughly one
-    /// request per second, so it defaults to serial queries.
+    /// request per second, so it defaults to serial queries; Tavily's plans
+    /// are metered per minute, so it inherits the gateway limit.
     #[must_use]
     pub const fn default_max_concurrent_queries(self) -> Option<NonZeroUsize> {
         match self {
-            Self::You => None,
+            Self::You | Self::Tavily => None,
             Self::Brave => Some(DEFAULT_BRAVE_MAX_CONCURRENT_QUERIES),
         }
     }
@@ -144,15 +148,17 @@ impl WebSearchProviderKind {
         match self {
             Self::You => "You.com",
             Self::Brave => "Brave Search",
+            Self::Tavily => "Tavily",
         }
     }
 
-    /// Configuration label (`you`, `brave`) matching the serialized form.
+    /// Configuration label (`you`, `brave`, `tavily`) matching the serialized form.
     #[must_use]
     pub const fn config_name(self) -> &'static str {
         match self {
             Self::You => "you",
             Self::Brave => "brave",
+            Self::Tavily => "tavily",
         }
     }
 
@@ -171,7 +177,8 @@ impl std::str::FromStr for WebSearchProviderKind {
     fn from_str(value: &str) -> Result<Self, Error> {
         let trimmed = value.trim();
         Self::ALL
-            .into_iter()
+            .iter()
+            .copied()
             .find(|kind| kind.config_name().eq_ignore_ascii_case(trimmed))
             .ok_or_else(|| {
                 let expected = Self::ALL
@@ -450,6 +457,15 @@ mod tests {
             NonZeroUsize::new(1)
         );
         assert!(!WebSearchProviderKind::Brave.is_you());
+
+        assert_eq!(WebSearchProviderKind::Tavily.to_string(), "Tavily");
+        assert_eq!(WebSearchProviderKind::Tavily.default_api_key_env(), "TAVILY_API_KEY");
+        assert_eq!(
+            WebSearchProviderKind::Tavily.default_base_url(),
+            Some("https://api.tavily.com")
+        );
+        assert_eq!(WebSearchProviderKind::Tavily.default_max_concurrent_queries(), None);
+        assert!(!WebSearchProviderKind::Tavily.is_you());
     }
 
     #[test]
@@ -464,10 +480,14 @@ mod tests {
             "you".parse::<WebSearchProviderKind>().unwrap(),
             WebSearchProviderKind::You
         );
+        assert_eq!(
+            " Tavily ".parse::<WebSearchProviderKind>().unwrap(),
+            WebSearchProviderKind::Tavily
+        );
         let error = "bing".parse::<WebSearchProviderKind>().unwrap_err();
         assert_eq!(
             error.to_string(),
-            "unknown web_search provider \"bing\"; expected one of: you, brave"
+            "unknown web_search provider \"bing\"; expected one of: you, brave, tavily"
         );
 
         assert_eq!(
@@ -478,7 +498,7 @@ mod tests {
             serde_json::from_str::<WebSearchProviderKind>("\"you\"").unwrap(),
             WebSearchProviderKind::You
         );
-        for kind in WebSearchProviderKind::ALL {
+        for kind in WebSearchProviderKind::ALL.iter().copied() {
             assert_eq!(kind.config_name().parse::<WebSearchProviderKind>().unwrap(), kind);
         }
     }

@@ -2,12 +2,13 @@
 //!
 //! `mod.rs` owns the OpenAI-facing adapter: the [`WebSearchHandler`], the
 //! mapping to public `web_search_call` output items. [`provider`] defines the
-//! private provider contract and normalized result types. [`args`] parses the model's arguments; provider modules
-//! ([`you`], [`brave`]) shape requests and map responses.
+//! private provider contract, normalized result types, and the response helpers every provider shares. [`args`]
+//! parses the model's arguments; provider modules ([`you`], [`brave`], [`tavily`]) shape requests and map responses.
 
 pub(crate) mod args;
 pub(crate) mod brave;
 mod provider;
+pub(crate) mod tavily;
 pub(crate) mod you;
 
 use std::collections::HashMap;
@@ -18,15 +19,17 @@ use std::pin::Pin;
 use std::sync::Arc;
 
 use futures::{StreamExt, TryStreamExt};
-use serde::{Deserialize, Deserializer, Serialize};
+use serde::Serialize;
 use serde_json::Value;
 use tokio::sync::Semaphore;
 
 use self::args::{MAX_WEB_SEARCH_QUERIES, WebSearchArguments};
 use self::brave::BraveSearchProvider;
 use self::provider::{
     ApiKey, WebSearchProvider, WebSearchProviderMetadata, WebSearchProviderResponse, WebSearchResult, clean_base_url,
+    null_as_default, read_response_limited,
 };
+use self::tavily::TavilySearchProvider;
 use self::you::{YOU_API_BASE_URL, YOU_API_KEY, YouSearchProvider};
 use super::handler::MAX_GATEWAY_TOOL_OUTPUT_BYTES;
 use super::handler::{GatewayExecutor, GatewayToolEventPlan, ToolError, ToolHandler, ToolOutput};
@@ -227,6 +230,15 @@ impl WebSearchHandler {
                     .or(WebSearchProviderKind::Brave.default_max_concurrent_queries())
                     .unwrap_or(max_concurrent_gateway_calls),
             )),
+            WebSearchProviderKind::Tavily => Arc::new(TavilySearchProvider::from_values(
+                client,
+                config.api_key.clone(),
+                config.base_url.clone(),
+                config
+                    .max_concurrent_queries
+                    .or(WebSearchProviderKind::Tavily.default_max_concurrent_queries())
+                    .unwrap_or(max_concurrent_gateway_calls),
+            )),
         };
         let effective = effective_query_concurrency(provider.as_ref(), requested);
         Self::with_provider_and_query_concurrency(provider, effective)
@@ -357,38 +369,6 @@ struct WebSearchToolOutput<'a> {
     metadata: Vec<WebSearchProviderMetadata>,
 }
 
-/// Deserializes an explicit JSON `null` as the field's default instead of
-/// failing, so a degenerate provider response cannot fail the whole search.
-pub(crate) fn null_as_default<'de, D, T>(deserializer: D) -> Result<T, D::Error>
-where
-    D: Deserializer<'de>,
-    T: Default + Deserialize<'de>,
-{
-    Option::<T>::deserialize(deserializer).map(Option::unwrap_or_default)
-}
-
-/// Reads a provider HTTP response body, failing as soon as it exceeds
-/// [`MAX_GATEWAY_TOOL_OUTPUT_BYTES`] so an oversized provider reply is never
-/// buffered in full. Every provider module reads its responses through here.
-pub(super) async fn read_response_limited(
-    resp: reqwest::Response,
-    provider: WebSearchProviderKind,
-) -> Result<String, ToolError> {
-    let mut stream = resp.bytes_stream();
-    let mut body = Vec::new();
-    while let Some(chunk) = stream.next().await {
-        let chunk = chunk
-            .map_err(|error| ToolError::Execution(format!("failed to read {provider} search response: {error}")))?;
-        if chunk.len() > MAX_GATEWAY_TOOL_OUTPUT_BYTES.saturating_sub(body.len()) {
-            return Err(ToolError::Execution(format!(
-                "{provider} search response exceeded {MAX_GATEWAY_TOOL_OUTPUT_BYTES} bytes"
-            )));
-        }
-        body.extend_from_slice(&chunk);
-    }
-    String::from_utf8(body).map_err(|_| ToolError::Execution(format!("{provider} search response was not valid UTF-8")))
-}
-
 impl ToolHandler for WebSearchHandler {
     type ToolParams = WebSearchToolParam;