Repository navigation
Conversation
alexandraabbas
left a comment
There was a problem hiding this comment.
Review posted on behalf of @alexandraabbas (written with Claude Code).
Thanks, the diagnosis is right: stitch_phases() already recomputes the spread from the final per-turn confidences after a relabel, and the confidence_spread = 0.0 override in _apply_verdicts() discards that for a merged phase. Removing the line is the correct fix. A few things before this can go in.
Must fix
- Rebase and re-bump. The branch bumps 0.4.4 to 0.4.5, but
mainis at 0.7.0. Rebase ontomainand bump to 0.7.1 (patch) inpyproject.tomlanduv.lock. - Keep the docs true. The column contract in
src/transect/frames/phases.py(theconfidence_spreadentry) was not touched. That docstring is the frame contract per AGENTS.md, so state the new semantics there, not only in the scanner docstring. - Say what the spread means for a mixed-source phase. After this change a phase with
confidence_source = "verifier"can carry a spread computed over two judge confidences and one verifier confidence. The per-turn rows for overturned turns are stampedconfidence_pm = 0.0, so the two surfaces now describe the overturned turn differently. Either document plainly that the phase spread is over the final per-turn confidences whatever their source (and thatconfidence_sourcenames the overturning verdict, not the whole sample), or restrict the spread to rows matchingconfidence_source. We prefer the first (document it) since it is the smaller change, but the contract has to say it. - The
StitchedPhasefield description overstates. "after any verifier updates and display merging" is not what happens: a weak relabel or a non-overturning review never re-stitches (applied_anystays False). Word it as "recomputed from the final per-turn confidences when an applied overturn re-stitches the phases".
Trim the tests
The suite target is a small, heavily parametrized set (see AGENTS.md). Five new tests for a one-line removal is more than it needs:
- Keep the parametrized
_apply_verdictstest; it pins the behaviour well. - Drop
test_merged_confidence_spread_survives_scanning_and_frame_projection, or reduce it to a single assertion thatphases_dfpasses the recomputed spread through. It re-runs the "merge-with-unreviewed" case end to end and re-asserts review-population facts (original_confidence,verifier_confidence,n_relabelled) thatreviewed_frame()tests already cover in the same file.
Once those are in we are happy to merge.
53d2540 to
2921737
Compare
|
Thank you for the review and sorry for the late reply. I rebased on |
A verifier correction can merge neighbouring phases whose confidence values still differ. In that case, the exported spread was always zero. I reproduced this with final confidences
[0.95, 0.60, 0.95]: the expected spread is0.229, but the phases frame contained0.In
_apply_verdicts()insrc/transect/scanners/phases_verify.py,stitch_phases()already rebuilds the merged phases and recomputes their statistics after a relabel. The finalif p.verifier.overturnedblock then overwroteconfidence_spreadwith0.0.decision_phases()stored that value, andphases_df()exported it unchanged inframes["phases"].confidence_spread.Removing that assignment preserves the calculation from the final turn confidences. Uniform confidence values still naturally produce zero spread.
Five regression tests cover merged and separate phases, including the public scanner and exported frame. Three fail before the fix. The unit suite passes (560 tests), along with Ruff, mypy, Pyright and package builds. Seven browser tests were skipped because Chromium could not run here. Local tests disabled only Scout’s progress memory monitor because /proc is unavailable.
Related to #101.