chore(release): 0.7.6 — digest-pinned, honesty-classified confined baseline - #1501
Merged
Conversation
…seline Workspace version 0.7.5 -> 0.7.6 (package + 17 internal =0.7.x pins + Cargo.lock). Banks the confined (OCAP-on) tb-30 scoreboard under a stricter methodology: - honesty classifier: a run counts only if the model made a real tool-call attempt on >=25/30 tasks, else quarantined (3 quarantined this cycle: granite4.1, nemotron-3-nano canonical + bare — see #1500). - each local score pinned to the served GGUF sha256 (a name is not an identity; cf. the Gemma 4 silent re-upload). Confined champions: ornith 36.7%, qwen3.6 26.7%, qwen3-coder 13.3%, kimi-linear 10.0%, glm-4.7-flash 3.3%. A re-baseline under stricter measurement, not a raw champion-defeat; tb-30 is a 30-task sample (~+/-10pp variance). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Release prep for 0.7.6: bumps the workspace version 0.7.5 → 0.7.6 (package + the 17 internal
=0.7.xpins +Cargo.lock), cutsCHANGELOG [0.7.6], and banks the confined (OCAP-on) tb-30 scoreboard under the honesty-classified, digest-pinned methodology.Confined champions:
ornith36.7%,qwen3.626.7%,qwen3-coder13.3%,kimi-linear10.0%,glm-4.7-flash3.3%. Three models were quarantined for under-engagement (granite4.1, nemotron-3-nano canonical + bare) — tracked in #1500.This is a re-baseline under stricter measurement, not a raw champion-defeat; tb-30 is a 30-task sample (~±10pp variance). Parity (OCAP-on ≈ off) is pursued forward via pre-granted permissions, not gated here.
Test plan
python3 scripts/eval/bench_scoreboard.py --self-test→ OKcargo update --workspaceresolves cleanly (lock synced to 0.7.6)just checkon this PROut of scope
model_digestemission fromnewt solve(folds into the feat(bench): newt bench — matrix orchestration, one model at a time (#1490) #1491 re-scope).Refs #1500.
Note
Low Risk
Version and documentation/eval-data only; no application logic, auth, or runtime behavior changes.
Overview
Release 0.7.6 bumps the workspace and all lock-step internal crate pins from
0.7.5to0.7.6(Cargo.toml,Cargo.lock) and adds a CHANGELOG [0.7.6] entry documenting the confined (OCAP-on) tb-30 re-baseline under stricter measurement (≥25/30 tool-call engagement, GGUFmodel_digestpinning, quarantine for under-engaged models).Benchmark artifacts extend
scripts/eval/bench-results.jsonlwithmodel_digeston key OCAP-on runs and new confined results forkimi-linear_48bandglm-4.7-flash. The auto-generated README scoreboard andbench_scoreboard.pyheader now describe 0.7.6 as establishing that honesty-classified baseline and note that OCAP parity is pursued forward, not gated on this release.Reviewed by Cursor Bugbot for commit 2ffb69b. Configure here.