Problem
The LLM integration is currently hardcoded to invoke the claude CLI via subprocess. This works but limits model portability — switching models requires a CLI tool that accepts the same stdin/stdout contract.
Proposed solution
Add support for alternative LLM backends behind the existing MODEL_COMMANDS registry:
- litellm — unified Python API for 100+ LLM providers (OpenAI, Anthropic, Gemini, local models)
- langgraph — for users who want agent-style orchestration
This could be implemented as:
- A
--backend flag (cli, litellm, langgraph) on pipeline/summarize/propose/exam commands
- Refactor
llm.py:invoke_sync to dispatch to the selected backend
- Keep the CLI subprocess path as the default for backward compatibility
Context
Related to #17 (stringly-typed-llm-integration). The current subprocess-based architecture makes model swapping look trivial but prompt templates may embed implicit model-specific format expectations. A Python SDK backend would enable structured output parsing and better error handling.
Workaround
Currently users can add entries to MODEL_COMMANDS for any CLI tool that accepts prompts on stdin and returns completions on stdout.
Problem
The LLM integration is currently hardcoded to invoke the
claudeCLI via subprocess. This works but limits model portability — switching models requires a CLI tool that accepts the same stdin/stdout contract.Proposed solution
Add support for alternative LLM backends behind the existing
MODEL_COMMANDSregistry:This could be implemented as:
--backendflag (cli,litellm,langgraph) on pipeline/summarize/propose/exam commandsllm.py:invoke_syncto dispatch to the selected backendContext
Related to #17 (stringly-typed-llm-integration). The current subprocess-based architecture makes model swapping look trivial but prompt templates may embed implicit model-specific format expectations. A Python SDK backend would enable structured output parsing and better error handling.
Workaround
Currently users can add entries to
MODEL_COMMANDSfor any CLI tool that accepts prompts on stdin and returns completions on stdout.