Skip to content

fix: default SDK eval model fallback to grok-code-fast#311

Open
zhao0112 wants to merge 1 commit into
darrenhinde:mainfrom
zhao0112:feat/fix-default-eval-model
Open

fix: default SDK eval model fallback to grok-code-fast#311
zhao0112 wants to merge 1 commit into
darrenhinde:mainfrom
zhao0112:feat/fix-default-eval-model

Conversation

@zhao0112

@zhao0112 zhao0112 commented Apr 24, 2026

Copy link
Copy Markdown

Summary

  • Fixes a regression where the eval runner default model could be opencode/grok-code, which is currently not available.
  • Changes evals/framework/src/sdk/test-runner.ts fallback default to opencode/grok-code-fast to match the CLI’s documented default and current provider availability.

Related Issue

Validation

  • Requested independent review via requesting-code-review workflow style (delegate reviewer): passed.
  • Verified diff is a single-line default-model fallback update.
  • Ran focused tests:
    • npx vitest run src/__tests__/error-handling.test.ts src/__tests__/framework-confidence.test.ts
      • Result: 5 existing failures in repository baseline tests; none were from touched path (pre-existing).
    • npm run build in evals/framework currently reports pre-existing TS issues in integration/evaluator areas.
    • npm run eval:sdk -- --help currently fails on an existing test data issue (invalid enum value in a repository test fixture).

Notes

  • No schema/API changes.
  • No new dependencies added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant