Skip to content

feat: add Tavily as parallel search provider option#1

Open
ParakhJaggi wants to merge 2 commits intointerskh:mainfrom
tavily-ai:feat/tavily-migration/querit-search-tavily
Open

feat: add Tavily as parallel search provider option#1
ParakhJaggi wants to merge 2 commits intointerskh:mainfrom
tavily-ai:feat/tavily-migration/querit-search-tavily

Conversation

@ParakhJaggi
Copy link

Summary

Adds Tavily as a configurable search and content extraction provider alongside the existing Querit.ai integration. This is an additive change — all existing Querit code, dependencies, and env vars remain untouched.

What changed

Provider selection logic

  • New selectProvider() helper in search.js reads SEARCH_PROVIDER env var ("tavily" or "querit")
  • Falls back to auto-detection: if TAVILY_API_KEY is set and QUERIT_API_KEY is absent, uses Tavily; otherwise defaults to Querit

search.js

  • Added tavilySearch() function using @tavily/core SDK
  • Maps CLI flags to Tavily params: --site-includeincludeDomains, --site-excludeexcludeDomains, --datetimeRange, --countrycountry, -nmaxResults (capped at 20)
  • Prints warning when --lang is used with Tavily (unsupported)
  • Normalizes Tavily results to match existing formatResults() shape
  • Updated usage() to document both providers

content.js

  • Added tavilyExtract() function using Tavily Extract API
  • Added extractWithProvider() dispatcher: tries Tavily Extract when configured, falls back to existing jsdom+Readability pipeline on failure
  • Existing extraction code completely unchanged

Documentation

  • SKILL.md: Added TAVILY_API_KEY and SEARCH_PROVIDER as optional env vars in metadata
  • README.md: Added Tavily configuration section, provider selection docs, and Tavily-specific notes (max results, unsupported filters)

Files changed

  • package.json — added @tavily/core dependency
  • package-lock.json — lockfile updated
  • search.js — provider selection + Tavily search path
  • content.js — Tavily Extract with jsdom fallback
  • SKILL.md — optional env var documentation
  • README.md — Tavily setup and provider selection docs

Dependency changes

  • Added @tavily/core@^0.7.2 to package.json

Environment variable changes

  • TAVILY_API_KEY — Tavily API key, required when using Tavily provider
  • SEARCH_PROVIDER — optional, "tavily" or "querit"; auto-detects if unset

Notes for reviewers

  • Tavily max results is 20 (vs Querit's 100); a warning is emitted if -n exceeds 20
  • --lang filter has no Tavily equivalent; a warning is printed and the filter is ignored
  • Tavily Extract failure gracefully falls back to the existing jsdom pipeline

Automated Review

  • Passed after 2 attempt(s)
  • Final review: The migration is well-implemented and all four issues from the previous review attempt are correctly addressed: SKILL.md requires.env is now empty with both keys moved to optionalEnv; content.js usage() documents TAVILY_API_KEY; extractWithProvider() warns when SEARCH_PROVIDER=tavily but TAVILY_API_KEY is missing; and SKILL.md limitations shows provider-specific max results. The Querit path is fully preserved as an else-branch, the auto-detection logic is consistent between search.js and content.js, dependencies are correctly updated in both package.json and package-lock.json, and README.md is updated with clear dual-provider setup instructions. Three minor issues remain but none are blocking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant