Repository navigation
feat: run index for a sample scanned over several epochs - #121
Merged
Merged
Conversation
Takeover of the contributor's index: one report per epoch plus report_index.html first in report_paths, with each epoch's recorded execution status, the judged surfaces' reliability pooled per recorded judge setup and vocabulary from the frames' own agreement and the original ballots, and the run-wide scan status; a single judge says it has no agreement to measure; a surface that breaks its frame contract gets a row naming the breach. Rendered from the report orchestrator, files named by epoch, links by file. The balanced statistic and the cross-run narrative recipe are not carried. Version 0.7.0, fixture stores regenerated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TJKrL8rQH1dN8B49eALQZn
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TJKrL8rQH1dN8B49eALQZn
Epochs with judged units but no recorded judge setup get a row saying so; a non-voting row carries no agreement numbers; the units the judge never saw share one constant with the per-epoch audit; column guards name the frames' contract; the narrow-column rule targets alpha and AC1; copy says "no data" as everywhere else; README and the report guide name both verifier columns. Tests merged: denominators over surfaces and setups in one parametrized test, the errored epoch as one case, the quoted file links pinned through a stem with a space, the single-judge pipeline run and the markup pins dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TJKrL8rQH1dN8B49eALQZn
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TJKrL8rQH1dN8B49eALQZn
alexandraabbas
marked this pull request as ready for review
November 5, 2026 09:39
This was referenced Nov 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Takeover of #100 (Kozzy's run index), rebased onto main and reduced to what the package can stand behind. Closes #100 as superseded once merged.
What changes
render()writesreport_epoch1.html,report_epoch2.html, ... plusreport_index.html, first inreport_pathsand opened by default; a single epoch renders one report as before. Each epoch report links back to the index. The index is rendered byreport/render.py(render_index), reads no transcripts and loads no chart runtime.Errors recordedwhen any scanner recorded an error for that transcript (fromscan_status.errors), elseRan.report/collection.py): each judged surface the scan ran (Phases, Sub-agents, custom layers declaring an audit), pooled over the epochs with one row per recorded judge setup and label vocabulary: valid ballots against the setup's expected count, the frames' own per-unitjudge_agreementaveraged, pairwise agreement, α and AC1 from the original ballots with transcript-scoped unit keys, verifier relabels and spot-check overturns. A single judge gets a row saying there is no agreement to measure; a surface whose frames break their contract gets a row naming the breach; epochs without a recorded setup pool into one "no judged units" row.What is deliberately not carried over from #100
reliability.equal_sample_agreement:transect()scans one sample per invocation, so the statistic always equalled the plain mean.judge_agreementwhere it counted fewer than two ballots: the frames already stamp that decision, and the re-count made the index disagree with the per-epoch audits.run001_epoch2file names and#transcript-Nanchors: every file holds one epoch.Validation
Rendered on the three-epoch fixture with a two-roll gpt-5-mini judge and verifier (table with numbers), the two-epoch Petri demo with a solo judge (single-judge row), and single-transcript scans (no index); links and back links checked in Chromium.
make checkpasses (674 tests).Review
Two no-context reviews of the original PR and one of the takeover; confirmed findings applied (README sections lost in a trim restored, no-regime epochs pooled, Sub-agents units limited to classified lanes, vocabulary keyed as a set, verifier counts from
relabel_rate, NaN epochs refused, docstrings and legend wording).🤖 Generated with Claude Code
https://claude.ai/code/session_01TJKrL8rQH1dN8B49eALQZn