Internal UX testing on 5/28 (Abby): "The inference -> scoring part - I thought there were other steps too in the pipeline?"
Minsoo clarified the intent: the run page intentionally shows only run-scope stages (Inference, Scoring) because suite-level stages (Systematize, Test Set) are cached and re-used across runs. This is correct behavior but invisible to first-time users.
Source:
viewer/src/routes/suite/[suite_id]/[run_id]/+page.svelte:829-860 renders only data.manifest.stagesassert_eval/runner.py:775-786 writes only run-scope stagesassert_eval/display.py:52-55 maps to Inference and Scoring
Proposed fix
Show the full pipeline (Systematize → Test Set → Inference → Scoring) with per-stage indicators of whether the stage was cached (re-used from suite) or regenerated for this run. This makes the suite-vs-run distinction visible instead of implicit.
Acceptance
First-time user looking at a run page sees the full pipeline and understands which stages are run-specific vs suite-scoped.
Internal UX testing on 5/28 (Abby): "The inference -> scoring part - I thought there were other steps too in the pipeline?"
Minsoo clarified the intent: the run page intentionally shows only run-scope stages (Inference, Scoring) because suite-level stages (Systematize, Test Set) are cached and re-used across runs. This is correct behavior but invisible to first-time users.
Source:
viewer/src/routes/suite/[suite_id]/[run_id]/+page.svelte:829-860renders onlydata.manifest.stagesassert_eval/runner.py:775-786writes only run-scope stagesassert_eval/display.py:52-55maps toInferenceandScoringProposed fix
Show the full pipeline (Systematize → Test Set → Inference → Scoring) with per-stage indicators of whether the stage was cached (re-used from suite) or regenerated for this run. This makes the suite-vs-run distinction visible instead of implicit.
Acceptance
First-time user looking at a run page sees the full pipeline and understands which stages are run-specific vs suite-scoped.