[Feature] Add Brave Search as a selectable web_search provider

[Zheng-Lu]

Problem statement / motivation

Part of Phase 2 of #291
Status: Blocked on Phase 1 #293
Scope: Brave Search integration, configuration plumbing, and tests

With the typed contract established in Phase 1 #293, web_search_preview still only has one backend (You.com).
Brave Search is the ideal first alternative:

• High-quality search results with web and news combined in a single request.
• Free developer tier (≈2,000 queries/month) allowing local and evaluation deployments without paid contracts.
• Differences from You.com that must be explicitly adapted: no server-side domain filtering, count capped at 20, distinct freshness syntax, and a free tier rate limit of ~1 QPS.

Proposed solution

A. Provider Implementation (tool/web_search/brave.rs)

• Endpoint: GET {base_url}/res/v1/web/search with result_filter=web,news and X-Subscription-Token header.
• Deserialization: Minimal structs with #[serde(default)] and without deny_unknown_fields (forward-compatible).
• Transport Rule: Do not send Accept-Encoding: gzip (core reqwest is built without gzip support in crates/agentic-server-core/Cargo.toml:26).

B. Input Adaptation Policy Matrix

Parameter / Input	You.com Behavior	Brave Search Behavior	Gateway Adaptation Policy
allowed_domains / blocked_domains	Native server-side filter	No server-side filter	Post-filter results client-side by host suffix on label boundary using url::Url. An allowlist is a hard contract; returning fewer than count is correct.
count	1..=100	1..=20	Clamp to 20 with tracing::debug!. Never error, as models cannot predict provider limits.
freshness	day/week/month/year	pd/pw/pm/py or date range	Parsed into typed Freshness enum in args.rs and rendered per provider.
News	Single request	Single request	Included in the same call via result_filter=web,news.
country / language	Native	country / search_lang	Mapped to Brave's corresponding query parameters.
You.com specifics (livecrawl, etc.)	Native	Not supported	Ignored with tracing::debug!; documented as You.com-specific.

C. Concurrency & 429 Rate Limiting

• Concurrency Ceiling: Add WebSearchProvider::max_concurrent_requests() (Brave defaults to 1; You.com inherits gateway limit).
• No Automatic 429 Retry in Phase 2: Automatic retries risk masking tool timeouts (executor/gateway.rs:263-274). A 429 immediately returns a failed web_search_call containing the upstream Retry-After header.

D. Configuration & Precedence Matrix

Setting	Environment Variable	config.toml Key	Default Value
Provider	AGENTIC_WEB_SEARCH_PROVIDER	[web_search].provider	"you"
API Key	Named by api_key_env	[web_search].api_key_env	"BRAVE_API_KEY" (for Brave)
Base URL	AGENTIC_WEB_SEARCH_BASE_URL	[web_search].base_url	https://api.search.brave.com
Max Concurrency	AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES	[web_search].max_concurrent_queries	1 (for Brave)

Note: YOU_API_BASE_URL remains fully supported for backward compatibility when provider = "you".

Decisions on #291 Open Questions (Defaults)

Question	Default Decision	Rationale
Q2 (You.com Base URL Default)	No default URL	Avoid changing existing behavior for You.com. Doc inconsistencies to be fixed in a separate docs: PR.
Q3 (Base URL Env Name)	AGENTIC_WEB_SEARCH_BASE_URL	Clean generic override for alternative providers; preserves YOU_API_BASE_URL for You.com.
Q5 (Metadata)	Include provider name	Surface provider name in metadata[] for visibility.

Scope Boundaries

In Scope for Phase 2	Explicitly Out of Scope (Deferred)
✅ BraveSearchProvider implementation	❌ SearXNG and Tavily (follow-up issues)
✅ Domain post-filtering & count clamping	❌ Query rewriting with site: operators
✅ Configuration plumbing (env, toml, generated)	❌ Automatic retry on HTTP 429
✅ Local mock HTTP tests (Axum)	❌ Public plugin trait / dynamic registration
✅ Documentation updates (README, kubernetes.md)	❌ Pagination support

Verification Checklist

• 200 OK (Web + News): Verify mixed results properly map to WebSearchResult and public sources.
• Empty Results: Handled gracefully without errors.
• 401 / 403 Authentication: Fails web_search_call, mentions the env variable, never leaks secrets.
• 429 Rate Limit: Fails web_search_call without retry, surfaces Retry-After in message.
• Domain Filtering: Comprehensive test covering allowlist, blocklist, subdomains, and label boundaries.
• Query Parameter Clamping: count clamped from 50 → 20; freshness converted to Brave syntax.
• Concurrency Limit: Multi-query batch respects max_active == 1 under Brave.
• Backward Compatibility: With provider unset, You.com behaves identically to Phase 1.
• Code Hygiene:
  • cargo clippy --all-targets -- -D warnings (clean)
  • cargo fmt -- --check (clean)
  • pre-commit run --all-files (clean)

[iamalanlui]

Hey — #294’s Brave web_search provider (You.com today; Tavily/SearXNG deferred) is still “add the next search API key so agentic-api stays up.” A 429 fails the tool; the free tier is ~1 QPS.

I built apifare (formerly apipay) so the gateway embeds one prepaid meter behind web_search_preview: one bearer token; governed proxy; no BRAVE_API_KEY / You.com key in config for those calls; caps + kill switch. Founding users get +1,500 credits on the first successful call.

Worth a look if a prepaid meter is the search hop, instead of the next selectable provider key?