feat(eval): shared perplexity harness with llama.cpp-comparable sliding-window mode - #418

Open

jamesburton wants to merge 14 commits into

kkokosa:mainfrom

jamesburton:issue/231-perplexity-harness

Commits on Jul 30, 2026

design(eval): perplexity harness contract + design doc (#231)

jamesburton
and
claude
committed
7af01b7
plan(eval): perplexity harness implementation plan (#231)

jamesburton
and
claude
committed
2cd9a62
feat(eval): numerically stable log-softmax for perplexity scoring (#231)

jamesburton
and
claude
committed
e78e13d
test(eval): deterministic IPerplexityModel double recording forward windows (#231)

jamesburton
and
claude
committed
3639027
feat(eval): PerplexityEvaluator teacher-forced mode for all-rows backends (#231)

jamesburton
and
claude
committed
a11821e
feat(eval): growing-prefix teacher-forced path for last-row-only backends (#231)

jamesburton
and
claude
committed
75cf1bf
feat(eval): llama.cpp-comparable sliding-window perplexity mode (#231)

jamesburton
and
claude
committed
23ea529
feat(eval): streaming corpus tokenizer for perplexity runs (#231)

jamesburton
and
claude
committed
d40e4f0
feat(eval): IPerplexityModel adapter over TransformerModel (#231)

jamesburton
and
claude
committed
fe4c516
feat(cli): dotllm perplexity verb (#231)

jamesburton
and
claude
committed
7e9f4c2
fix(eval): separate window advance from scored span; drop absolute positions (#231)

jamesburton
and
claude
committed
e303be4
feat(eval): per-window observer + token-stream diagnostics (#231)

jamesburton
and
claude
committed
f9499a4

Commits on Jul 31, 2026