[pejmantheory]

Summary

• Reran nvidia/nemotron-3-super-120b-a12b with pnpm run eval:model -- --model nvidia/nemotron-3-super-120b-a12b --thinking all --runs 3 --concurrency 3.
• The run used the current dataset: 85 items, 46 hard, including trap-h4. Recorded git SHA c8d84957. Temperature 1, top-p 0.95, 16,000 max tokens, 240s timeout. No rate-limit or timeout losses.
• Thresholds were not changed. Super reasoning-low is QUALIFIED. Super reasoning-on is NOT QUALIFIED because p50 is 6746ms. Super reasoning-off is NOT QUALIFIED because the hard-tier score is 39/46.
• Datasets and graders are unchanged between the Lightning SHA 6ff90e47 and this SHA. The runs were not collected in the same session. No production routing or deploy.

Test plan

• pnpm run contract
• pnpm run check

Made with Cursor

---

Note

Low Risk
Changes are committed eval artifacts, documentation, and unit-test expectations only; no graders, datasets, routing, or contract threshold logic changed in code.

^{Reviewed by Cursor Bugbot for commit 480dc40. Bugbot is set up for automated code reviews on this repo. Configure here.}