Skip to content

identifyChangedSections collides headings that differ only by level, so review mode is told an unchanged section changed #283

Description

@mmcky

headingToId strips the level marker before building its lookup map, so ### Applications in Economics and #### Applications in Economics produce the same id. identifyChangedSections keys beforeById on that id with a plain Map, so the later heading silently overwrites the earlier one, and the earlier section is then compared against the other section's content — which always differs. The result is a section reported as modified on every diff of the document, whether or not it changed.

Reproduced on the harness fixture at @v0 (v0.26.0). A one-word edit confined to ## Eigenvalues and Eigenvectors in test-translation-sync/lecture.md reports:

["modified:#### Applications in Economics", "modified:## Eigenvalues and Eigenvectors"]

The first entry is spurious — that section is byte-identical on both sides. The same entry appears for an edit to ## Matrix Operations and for an edit to ## Vector Spaces, i.e. for any edit to the file.

Why it matters. changedSections is not diagnostic output — it becomes the ## IMPORTANT: Changed Sections in This PR block in the review prompt, which tells the model "Your suggestions MUST focus ONLY on these changed sections". A spurious entry widens the reviewer's licence to raise findings on text the PR did not touch, and every such finding is a candidate gating finding under the any-minor-in-gating-category rule. Since 2026-08-04 that rule is what the shadow window is calibrating, so the noise lands directly in the calibration data.

Scope. 16 of 228 English lectures across lecture-python.myst, lecture-python-intro and lecture-python-programming (7%) contain at least two headings that collide under headingToId — e.g. cass_fiscal.md (inelastic-labor-supply, steady-state, shooting-algorithm each twice), ifp_egm.md (set-up three times, solver twice), aiyagari.md (equilibrium twice). Every sync PR touching one of those files carries the defect.

Fix shape. Key the map on level + id (or on document position) rather than on id alone; duplicates at the same level remain genuinely ambiguous and should be matched positionally, which is what the targetAfterSections[i] lookup already assumes further down the same function.

Found while building the injected-defect certification harness for QuantEcon/project-translation#28. Not fixed there deliberately: the reviewer is frozen while it is being measured, and this changes the prompt it receives.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions