Skip to content

Count eight cases in the eval tables - #80

Merged
kaveraa merged 1 commit into
mainfrom
evals/eight-cases
Nov 13, 2026
Merged

kaveraa merged 1 commit into
mainfrom
evals/eight-cases

Conversation

@kaveraa

@kaveraa kaveraa commented Nov 13, 2026

Copy link
Copy Markdown
Owner

The eval table, the eval card and the gain card now count eight cases, zod-email included.

Model Score with / without Turns with / without Session cost with / without
Fable 5.2 0.98 / 0.89 4.3 / 6.0 0.197 / 0.189 USD
Opus 5.5 0.99 / 0.86 4.8 / 5.7 0.089 / 0.086 USD
Sonnet 5.5 0.99 / 0.87 4.4 / 5.6 0.043 / 0.040 USD
Haiku 5.5 0.98 / 0.86 5.3 / 6.4 0.003 / 0.003 USD

This is not a new run of the whole suite. It joins the seven cases and zod-email, measured two hours apart on the same day, with the same judge, the same Claude Code and the same plugin code (checked with git diff v0.15.3 main -- hooks skills agents). One case was run again, explain-bug on Opus, which had lost a run to a spawn error. The notebook says so.

README and cards in both languages, gain card, notebook, CHANGELOG. 97 tests passing.

@kaveraa
kaveraa merged commit dc4f28d into main Nov 13, 2026
5 checks passed
@kaveraa
kaveraa deleted the evals/eight-cases branch November 13, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant