Match flat Diagnostics scores label to the per-example path - #68
Merged
Conversation
The per-example Diagnostics renderer relabels the reserved `scores`
key to "Scores (Higher is Better)", carrying the higher-is-better cue
forward from the upstream GEPA optimizer. The flat/aggregate path (used
when an evaluator emits one side_info for the whole batch instead of
per-example records) rendered the same key as a raw `scores: {...}`
line, silently dropping that cue.
The flat path now relabels `scores` the same way, keeping its existing
` label: value` line shape rather than growing a markdown header.
This is presentation only: side_info["scores"] is unchanged as the
data-contract key read by the parser, executor, population, and the
objective/cartesian frontier modes.
Also re-cite the ported GEPA closure by file + symbol instead of file +
line number, since upstream line numbers rot independently of this repo.
KE7
force-pushed
the
fix/flat-diagnostics-scores-header
branch
from
August 26, 2026 10:57
87ea484 to
8a6da0b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The mutation-prompt Diagnostics section has two renderers. The per-example
renderer relabels the reserved
scoreskey toScores (Higher is Better),a header carried over from the upstream optimizer this repo derives its
Diagnostics format from. The flat/aggregate renderer — used when an
evaluator emits one
side_infofor the whole batch instead of per-examplerecords — rendered the same key as a raw
scores: {...}line, so the samedata got two different presentations and the flat path silently dropped
the "higher is better" cue.
This PR makes the flat path relabel
scoresthe same way, in its existinglabel: valueline shape (no markdown header, since that isn't thispath's style):
Per-example rendering is unchanged:
This is a presentation-only change.
side_info["scores"]is a datacontract read by the parser, executor, population, and the
objective/cartesian frontier modes — none of those, nor any evaluator, nor
the wire format, were touched. Only how the flat renderer displays the key
to the model changed.
Also fixed
_render_per_example_diagnostics's docstring cited the upstream GEPAsource it mirrors by file and line number
(
src/gepa/strategies/instruction_proposal.py:54-95). Upstream linenumbers rot independently of this repo, so it's re-cited by file + symbol
instead: the
format_samplesclosure insideInstructionProposalSignature.prompt_rendererinsrc/gepa/strategies/instruction_proposal.py. Verified against a freshclone of
gepa-ai/gepa—format_samples(and therender_valueclosureinside it) live exactly where the docstring says, nested in that method.
The attribution itself is kept, only the citation shape changed.
Test plan
tests/unit/test_mutator.py::TestPerExampleDiagnosticswith a test that renders both Diagnostics paths on the same
scoreskey and asserts they carry the same label.uv run python -m pytest— 923 passeduv run ruff check src/ tests/— all checks passeduv run mypy --strict src/helix/— no issues in 26 source files