Skip to content

feat: add OpenRouter as LLM and embedding provider#56

Open
oglenyaboss wants to merge 1 commit intorepowise-dev:mainfrom
oglenyaboss:feat/openrouter-provider
Open

feat: add OpenRouter as LLM and embedding provider#56
oglenyaboss wants to merge 1 commit intorepowise-dev:mainfrom
oglenyaboss:feat/openrouter-provider

Conversation

@oglenyaboss
Copy link
Copy Markdown

@oglenyaboss oglenyaboss commented Apr 8, 2026

Summary

  • Add OpenRouter as a first-class LLM and embedding provider, enabling access to 200+ models through a single OPENROUTER_API_KEY
  • Both repowise init (doc generation) and repowise serve (chat/search) work with OpenRouter out of the box
  • No new pip dependency — uses the existing openai package (OpenAI-compatible API)

LLM Provider

  • OpenRouterProvider with generate() + stream_chat() support
  • Default model: anthropic/claude-sonnet-4.6
  • Sets recommended HTTP-Referer and X-Title headers for OpenRouter dashboard tracking
  • Rate limits: 60 RPM / 200K TPM (conservative defaults, user-overridable)

Embedding Provider

  • OpenRouterEmbedder for semantic search and chat RAG
  • Default model: google/gemini-embedding-001 (768 dims)
  • One API key covers both LLM and embeddings

Integration Points

  • Registered in both LLM and embedding registries (lazy import)
  • CLI auto-detection from OPENROUTER_API_KEY env var
  • Interactive provider/embedder selection in repowise init and repowise serve
  • Server provider catalog updated for web UI

Known Limitations

  • Cost tracking disabled for OpenRouter: since it proxies 200+ models with varying prices, the fallback pricing would show inflated numbers. Users should check the OpenRouter dashboard for actual costs. A future PR could fetch real pricing via the /api/v1/models endpoint.
  • Chat and embeddings not fully tested — LLM generation was tested end-to-end on a real 800+ file project using qwen/qwen3.6-plus via OpenRouter. Chat and embedding functionality was not integration-tested but follows the same OpenAI-compatible patterns as the existing OpenAI provider.

Test plan

  • 13 unit tests for OpenRouterProvider (construction, generation, error mapping, headers)
  • Registry test updated (builtin count 6 → 7, openrouter in list)
  • Integration test added (skipped without OPENROUTER_API_KEY)
  • Manual end-to-end test: repowise init with OpenRouter on a real project (328 wiki pages generated successfully)
  • Chat via web UI with OpenRouter as active provider
  • Embedding search with OpenRouterEmbedder

Add OpenRouter as a first-class provider, enabling access to 200+ models
(Claude, GPT, Gemini, Llama, Qwen, etc.) through a single API key.

LLM provider:
- New OpenRouterProvider using OpenAI-compatible endpoint
- Supports generate() and stream_chat() (ChatProvider protocol)
- Sets recommended HTTP-Referer and X-Title headers
- Default model: anthropic/claude-sonnet-4.6
- Rate limits: 60 RPM / 200K TPM
- Cost tracking intentionally disabled (OpenRouter proxies models
  with varying prices — users should check the OpenRouter dashboard)

Embedding provider:
- New OpenRouterEmbedder for vector search and chat
- Default model: google/gemini-embedding-001 (768 dims)
- One OPENROUTER_API_KEY covers both LLM and embeddings

Integration:
- Registered in LLM and embedding registries (lazy import)
- CLI auto-detection from OPENROUTER_API_KEY env var
- Interactive provider selection in `repowise init`
- Embedder selection in `repowise serve`
- Server provider catalog for web UI
- No new pip dependency (uses existing openai package)

Tests:
- 13 unit tests (construction, generation, error mapping, headers)
- Registry test updated (builtin count 6 → 7)
- Integration test (skipped without OPENROUTER_API_KEY)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant