Skip to content

feat(llm): add GRAFT_REASONING_EFFORT / --reasoning-effort - #378

Open
Def3nder wants to merge 1 commit into
trailhq:mainfrom
Def3nder:feat/reasoning-effort
Open

Def3nder wants to merge 1 commit into
trailhq:mainfrom
Def3nder:feat/reasoning-effort

Conversation

@Def3nder

Copy link
Copy Markdown

Problem

With a local reasoning model behind an OpenAI-compatible endpoint (qwen3.8:27b
on Ollama), every forced-tool call in the deep pass came back empty:

[001] -> qwen3.8:27b | 2 msg, 31262 chars | tool_choice: record_graph | max_tokens: 8192
[001] <- 200 after 195.0s | finish: length | content: EMPTY | tool_calls: 0
                          | reasoning: 15386 chars | completion_tokens: 8192

The model spent the entire maxTokens budget in its reasoning channel. Ollama's
parser puts that in message.reasoning, which fromResponse() does not read, so
graft sees an empty message and records a miss — after three minutes of GPU time.
The reasoning text itself shows the work was done; it ends with "Let me write
the final output now."

synthesize and crux both request maxTokens: 8192, which is exactly the
budget the reasoning consumes.

Why the existing remedy does not reach this

openai.ts already knows the fix — isRejectedToolsWithReasoning() retries with
reasoning_effort: "none" when a provider rejects tools while reasoning is on:

if (isRejectedToolsWithReasoning(err) && attempt.reasoning_effort === undefined) {
  attempt = { ...attempt, reasoning_effort: "none" } as ChatParams;

That is reactive, and it needs an HTTP 400. A server that answers 200 with an
empty message
never triggers it. This PR adds the proactive form of the same
value.

Change

reasoning_effort is sent on every request when configured, following the
existing env-plus-flag pattern of every other provider option:

  • GRAFT_REASONING_EFFORT / --reasoning-effort <level>
  • resolved in providers.ts, carried through factory.ts, applied in
    openai.ts (so litellm and orcarouter inherit it — their option types are
    aliases); anthropic is untouched
  • unset by default: no behaviour change unless the user asks for it

Result

Same call, GRAFT_REASONING_EFFORT=none:

finish_reason: tool_calls | tool_calls: 1 | reasoning: absent | completion_tokens: 3258

12 nodes, 29 links — from a call that previously returned nothing.

Note on the cast

(params as unknown as Record<string, unknown>).reasoning_effort = this.reasoningEffort;

"none" is not in the ReasoningEffort union of the pinned openai SDK, and
the value is user-supplied, so it is written through an index signature. The
existing reactive path solves the same typing problem differently
({ ...attempt, reasoning_effort: "none" } as ChatParams). If you would rather
have one shape for both, say so and I will fold them into a small helper.

@trailhq-graft

trailhq-graft Bot commented Sep 13, 2026

Copy link
Copy Markdown

🌱 graft blast radius

3 areas changed → 3 areas can be affected. 5 dependent symbols, depth 2.
Tests: no test reaches Provider Configuration, CLI Setup; Chat Model Factory has tests the diff did not touch.
Tag: @anirudhkumar-nanonets — 5 of 6 areas · @shhdwi — CLI Setup · lovejones2914-spec — Chat Model Factory, LLM Integrations

flowchart TB
  A0(("LLM Integrations<br/>2 symbols"))
  A1(("Cluster Naming<br/>2 symbols"))
  A2(("Pull Request Review<br/>1 symbol"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2 reached;
Loading
Can be affected Symbols Nearest hop Reached from
LLM Integrations 2 src/ai/llm/litellm.ts:L25-L33 LiteLLMChatModel — extends, depth 1 Chat Model Factory
Cluster Naming 2 src/blast/name.ts:L274-L302 nameReport — calls, depth 1 Chat Model Factory, Provider Configuration
Pull Request Review 1 src/app/review.ts:L45-L99 reviewPullRequest — calls, depth 2 Chat Model Factory, Provider Configuration
Who knows this code — 4 people across 6 areas
Area Who knows it
Chat Model Factory · changed @anirudhkumar-nanonets — 2 commits, last 24d ago · lovejones2914-spec — 1 commit, last 13d ago
Provider Configuration · changed @anirudhkumar-nanonets — 7 commits, last 2mo ago
CLI Setup · changed @anirudhkumar-nanonets — 41 commits, last 3d ago · @shhdwi — 24 commits, last 1mo ago
LLM Integrations · affected lovejones2914-spec — 1 commit, last 13d ago · Prodman Devokadev — 1 commit, last 17d ago
Cluster Naming · affected @anirudhkumar-nanonets — 7 commits, last 16d ago
Pull Request Review · affected @anirudhkumar-nanonets — 3 commits, last 12d ago

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 5 dependent symbols, grouped by area

LLM Integrations — 2 symbols in 2 files

  • src/ai/llm/litellm.ts:L25-L33 — LiteLLMChatModel (extends, depth 1)
    25: export class LiteLLMChatModel extends OpenAIChatModel {
  • src/ai/llm/orcarouter.ts:L25-L33 — OrcaRouterChatModel (extends, depth 1)
    25: export class OrcaRouterChatModel extends OpenAIChatModel {

Cluster Naming — 2 symbols in 2 files

  • src/blast/name.ts:L274-L302 — nameReport (calls, depth 1)
    279: const { resolveConfig } = await import("../ai/providers.js");
  • src/blast/blast-cli.ts:L132-L136 — nameClusters (calls, depth 2)

Pull Request Review — 1 symbol in 1 file

  • src/app/review.ts:L45-L99 — reviewPullRequest (calls, depth 2)
Test signal per changed area — 1 ⚠ · 2 ✗

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • Chat Model Factory — 3 of 4 reached · 3 test files reach it, none changed here
    • not reached: constructor
  • Provider Configuration — 0 of 1 reached · no test file reaches it
    • not reached: resolveConfig
  • CLI Setup — 0 of 2 reached · no test file reaches it
    • not reached: cliConfig, chatModel
3 test suites also reference this code

3 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/litellm-adapter.test.ts
  • test/llm-adapters.test.ts
  • test/orcarouter-adapter.test.ts

graft blast · origin/main...HEAD · depth 2 · 5 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

github-actions Bot added a commit that referenced this pull request Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant