Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,23 @@ SMALLCODE_CLASSIC_TUI=false
# ─── OpenRouter Example ──────────────────────────────────────────────────────
# SMALLCODE_BASE_URL=https://openrouter.ai/api/v1
# SMALLCODE_MODEL=openai/gpt-4o-mini
# OPENAI_API_KEY=sk-or-v1-...
# OPENROUTER_API_KEY=sk-or-v1-...

# ─── Multi-Model Routing (optional) ──────────────────────────────────────────
# Auto-pick model based on task complexity
# Auto-pick model based on task complexity. Each tier may use its own endpoint.
# SMALLCODE_MODEL_FAST=gemma-4-e4b
# SMALLCODE_MODEL_DEFAULT=qwen3-8b
# SMALLCODE_MODEL_STRONG=qwen3-14b
# SMALLCODE_BASE_URL_FAST=http://localhost:1234/v1
# SMALLCODE_BASE_URL_DEFAULT=http://localhost:1234/v1
# SMALLCODE_BASE_URL_STRONG=https://openrouter.ai/api/v1
#
# Local default + OpenRouter strong model:
# SMALLCODE_MODEL=qwen3:8b
# SMALLCODE_BASE_URL=http://localhost:11434/v1
# SMALLCODE_MODEL_STRONG=openai/gpt-4o-mini
# SMALLCODE_BASE_URL_STRONG=https://openrouter.ai/api/v1
# OPENROUTER_API_KEY=sk-or-v1-...

# ─── RTK (Rust Token Killer) ─────────────────────────────────────────────────
# RTK auto-rewrites bash commands (git, tests, ls, etc.) for 60-90% token savings.
Expand Down
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,27 @@
# Changelog

## Unreleased

### feat: per-tier endpoint routing

`SMALLCODE_MODEL_FAST`, `SMALLCODE_MODEL_DEFAULT`, `SMALLCODE_MODEL_MEDIUM`,
and `SMALLCODE_MODEL_STRONG` can now be paired with matching
`SMALLCODE_BASE_URL_*` variables. `smallcode.toml` also supports
`[models.fast]`, `[models.default]`, `[models.medium]`, and
`[models.strong]` sections with `name` and `baseUrl`, so users can keep
default work on localhost while routing larger tiers to OpenRouter.

Complexity-based tier selection and adaptive failure-rate routing now resolve
the matching endpoint per request (`activeModelTarget`). Auth headers follow
the selected URL — OpenRouter keys and headers for cloud tiers, no auth for
local endpoints. Primary `[model]` config remains env-first; TOML tier sections
always merge; env tier vars override TOML.

### Verification

- 90/90 unit tests pass (`npm test`) — 7 new in `test/config_normalize.test.js`,
`test/model_routing.test.js`, and `test/provider_compat.test.js`

## [1.2.1] - 2026-05-24

### fix: provider compatibility — 7 bugs causing 400 errors on cloud/local LLMs
Expand Down
32 changes: 31 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,40 @@ SMALLCODE_BASE_URL=http://localhost:1234/v1
# Optional: escalation (auto-fallback to cloud on hard fail)
# ANTHROPIC_API_KEY=sk-ant-...
# OPENAI_API_KEY=sk-...
# OPENROUTER_API_KEY=sk-or-v1-...
# DEEPSEEK_API_KEY=sk-...
```

See `.env.example` for all options. Also supports `smallcode.toml` for backwards compatibility.

SmallCode can route each model tier to a different endpoint. This lets you keep
fast/default work local while sending complex tasks to a larger OpenRouter model:

```bash
SMALLCODE_MODEL=qwen3:8b
SMALLCODE_BASE_URL=http://localhost:11434/v1

SMALLCODE_MODEL_STRONG=openai/gpt-4o-mini
SMALLCODE_BASE_URL_STRONG=https://openrouter.ai/api/v1
OPENROUTER_API_KEY=sk-or-v1-...
```

Equivalent `smallcode.toml`:

```toml
[model]
provider = "openai"
name = "qwen3:8b"
baseUrl = "http://localhost:11434/v1"

[models.strong]
name = "openai/gpt-4o-mini"
baseUrl = "https://openrouter.ai/api/v1"
```

Tier URLs are optional. If `SMALLCODE_BASE_URL_STRONG` or
`[models.strong].baseUrl` is omitted, that tier uses the primary base URL.

## Architecture

SmallCode is built with a modular architecture:
Expand Down Expand Up @@ -227,12 +256,13 @@ When 3 or more files are edited in a single agent turn, `coordinateMultiFileEdit
When `patch` fails because `old_str` no longer exists in the file, `semanticMerge()` asks the model to merge the intended change into the current file content. Returns the complete corrected file. Replaces the hard error with a recovery attempt. TTL 1 min (content-specific).

### Adaptive Model Select (Rank 8)
`AdaptiveModelRouter` in `src/model/adaptive_router.js` tracks per-model call/fail counts. When the primary model's failure rate exceeds 0.3 (medium) or 0.6 (strong), `chatCompletion` automatically overrides `body.model` with `SMALLCODE_MODEL_MEDIUM` or `SMALLCODE_MODEL_STRONG`. Requires at least 3 calls before routing decisions activate. Reset via `router.reset()`.
`AdaptiveModelRouter` in `src/model/adaptive_router.js` tracks per-model call/fail counts. When the primary model's failure rate exceeds 0.3 (medium) or 0.6 (strong), `chatCompletion` automatically routes to `SMALLCODE_MODEL_MEDIUM` or `SMALLCODE_MODEL_STRONG` and their matching `SMALLCODE_BASE_URL_*` endpoint when configured. Requires at least 3 calls before routing decisions activate. Reset via `router.reset()`.

```bash
# Optional: configure fallback models for adaptive routing
SMALLCODE_MODEL_MEDIUM=qwen2.5-coder:32b
SMALLCODE_MODEL_STRONG=gpt-4o
SMALLCODE_BASE_URL_STRONG=https://openrouter.ai/api/v1
```

### Benchmark Harness
Expand Down
29 changes: 29 additions & 0 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,40 @@ SMALLCODE_BASE_URL=http://localhost:1234/v1
# 可选:模型升级(硬失败时自动回退到云端)
# ANTHROPIC_API_KEY=sk-ant-...
# OPENAI_API_KEY=sk-...
# OPENROUTER_API_KEY=sk-or-v1-...
# DEEPSEEK_API_KEY=sk-...
```

查看 `.env.example` 了解所有选项。同时支持 `smallcode.toml` 以保持向后兼容。

SmallCode 可以为每个模型层级配置不同 endpoint。这样可以让 fast/default
任务继续使用本地模型,而复杂任务使用 OpenRouter 上的更大模型:

```bash
SMALLCODE_MODEL=qwen3:8b
SMALLCODE_BASE_URL=http://localhost:11434/v1

SMALLCODE_MODEL_STRONG=openai/gpt-4o-mini
SMALLCODE_BASE_URL_STRONG=https://openrouter.ai/api/v1
OPENROUTER_API_KEY=sk-or-v1-...
```

等价的 `smallcode.toml`:

```toml
[model]
provider = "openai"
name = "qwen3:8b"
baseUrl = "http://localhost:11434/v1"

[models.strong]
name = "openai/gpt-4o-mini"
baseUrl = "https://openrouter.ai/api/v1"
```

层级 URL 是可选的。如果省略 `SMALLCODE_BASE_URL_STRONG` 或
`[models.strong].baseUrl`,该层级会使用主 `baseUrl`。

## 架构

SmallCode 采用模块化架构:
Expand Down
2 changes: 2 additions & 0 deletions bin/commands.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ module.exports = function createCommandHandler(config, conversationHistory, impr
} else {
const newModel = parts.slice(1).join(' ');
config.model.name = newModel;
delete config.activeModelTarget;
console.log(` ${chalk.green('✓')} Switched to ${chalk.cyan(newModel)}`);
}
console.log('');
Expand All @@ -73,6 +74,7 @@ module.exports = function createCommandHandler(config, conversationHistory, impr
console.log(chalk.gray(' Switch: /endpoint http://host:port/v1'));
} else {
config.model.baseUrl = parts[1];
delete config.activeModelTarget;
console.log(` ${chalk.green('✓')} Endpoint: ${chalk.gray(parts[1])}`);
}
console.log('');
Expand Down
Loading
Loading