You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two full transcript-processing pipelines exist (see #40). There is no test that runs the same transcript through both and compares outputs.
Without such a test, the drift instances catalogued in #40 (contraction tokenization, SYS/TOOL round anchoring) and any future divergences are invisible in CI. A metric value change caused by a backend bug, a core/ bug, or a spec ambiguity all look the same to the golden-drift harness: only one pipeline ran.
For metrics where documented divergence is intentional (e.g. contraction splitting), the test should assert the known difference explicitly rather than skipping:
# Documented: backend splits contractions, core does not.# Backend bone_count > core bone_count by ~N per contraction in fixture.assertbackend_result["bone_count"] >=core_result["bone_count"]
This forces divergences to be named and reasoned about rather than silently tolerated.
Two full transcript-processing pipelines exist (see #40). There is no test that runs the same transcript through both and compares outputs.
Without such a test, the drift instances catalogued in #40 (contraction tokenization, SYS/TOOL round anchoring) and any future divergences are invisible in CI. A metric value change caused by a backend bug, a core/ bug, or a spec ambiguity all look the same to the golden-drift harness: only one pipeline ran.
Proposed test structure:
For metrics where documented divergence is intentional (e.g. contraction splitting), the test should assert the known difference explicitly rather than skipping:
This forces divergences to be named and reasoned about rather than silently tolerated.