Conversation
Web search resolved to Exa only, so it required a paid third-party key. Add fastCRW alongside it, which needs no key when self-hosted. - new provider module implementing the existing WebSearchExtension - "crw" added to WebSearchProviderSchema; providerName became a lookup - crwApiKey / crwBaseUrl on ProviderClientConfig - source selection refactored to an ordered first-configured pick so precedence is stated once - tool availability ambient derives its source union from WebSearchProviderSchema instead of restating the literal - WebSearchStructuredOutputUnsupported so a schema request fails loudly rather than returning a success envelope with no data
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
why
web_searchcurrently has one working source. inpackages/sdk/src/provider-client.ts:without
EXA_API_KEYthe tool fails withWebSearchNotConfigured, anddocs/get-started.mdxsends people to exa.ai to buy a key.after the move to fully local, web search is the one part of the agent that
still needs a paid third-party account.
this adds a second source: fastCRW, an AGPL-3.0
search engine that runs as a single static binary. self-hosted it needs no key
and no account, so
web_searchworks with nothing but a base url.disclosure up front: i maintain fastCRW.
what changed
follows
packages/providers/AGENTS.mdand mirrors the exa module:packages/providers/src/crw/{contract.ts,web-search.ts}implementing theexisting
WebSearchExtensionfrompackages/ai"crw"added toWebSearchProviderSchema;providerNamebecame a lookupcrwApiKey/crwBaseUrlonProviderClientConfig, wired intocreateProviderClient()first-configured pick, so precedence is stated once and a fourth source is a
one-line change
crw/contract.test.ts+crw/web-search.test.ts(8 cases), allagainst a local
Bun.servefixture, no network and no keytwo changes outside the new module, both deliberate
1. the ambient availability schema now derives from
WebSearchProviderSchema.packages/agent/src/ambient/tool-availability-ambient.tsrestatedSchema.Literal('magnitude','exa'), so adding a provider brokepackages/acntypecheck. i pointed it at the existing schema instead of typinga third literal, otherwise the next provider breaks the build the same way.
this is persisted event-sourced state, so flagging it rather than burying it:
the union only widens, so previously written events still decode. if you would
rather own that change separately i can drop it and you can take it on your own
terms, but then this PR cannot go green.
2. a new
WebSearchStructuredOutputUnsupportederror. see below.the structured-output gap
webSearch(query, schema)takes an optional schema, which exa serves viaoutputSchema. fastCRW's search endpoint has no equivalent.returning results with
dataabsent looked like the wrong call: the tooldeclares
dataas optional output, so the model gets a clean success envelopewith no data and cannot tell "nothing matched" from "this source cannot do
that", which usually turns into a retry loop or an invented value.
so it fails loudly with a typed error naming the reason. if you would rather it
degrade quietly and be documented instead, say so and i will switch it.
behaviour
CRW_API_URL=http://localhost:3000and nothing else: works, no key, and noAuthorizationheader is sent at allCRW_API_KEY=...: uses the hosted engineunavailable, exactly as todayone thing worth knowing:
CRW_API_KEY/CRW_API_URLare also fastCRW's ownCLI config vars, so a user who has run
crw setupalready has them exported.that means web search can become available without them setting anything for
magnitude specifically. i think that is the behaviour you want given exa reads
EXA_API_KEYthe same way, but it is a wider blast radius than exa'ssingle-purpose variable, so it is your call. every routing test pins all
sources explicitly so the suite does not depend on ambient env.
verification
self-host route prefix, against a locally running engine:
packages/providers: 174 passed (14 new, across contract and web-search)packages/sdk: 56 passed, including 5 source-selection casespackages/agentweb-search tool test: passedCRW_API_KEY/CRW_API_URLexported, still greencli,packages/sdk,packages/acn,packages/icn-protocol,packages/releaseall cleanWebSearchResult; aschema request returns
WebSearchStructuredOutputUnsupportednote:
bun run --cwd packages/icn typecheckfails onsrc/lifecycle/cuda-compatibility.test.ts(38,3), but it fails identically withthis branch stashed, so it is not from this change.
scope
only
web_search. i measuredweb_fetchtoo and on pages a plain GET canalready fetch, your
dom-extractoutput beats what i get back and is roughlyten times faster, so i am not proposing to touch it.
happy to close this if a third-party search source is not a direction you want.