[Jamie] Stop Consolidated Report runs from appearing in the Legal Benchmarks Runs tab - #5204
Open
pitoi wants to merge 2 commits into
Open
[Jamie] Stop Consolidated Report runs from appearing in the Legal Benchmarks Runs tab#5204pitoi wants to merge 2 commits into
pitoi wants to merge 2 commits into
Conversation
…chmarks Runs tab ## Problem The Legal Benchmarks "Runs" tab for a task shows `LEGAL_BENCHMARK_CONSOLIDATED` runs (cross-run Consolidated Report comparisons, a separate Stakwork workflow) mixed in with the task's actual `LEGAL_BENCHMARK_RUNNER` runs (workflow 57179). ## Root cause `src/hooks/useLegalBenchmarkRunList.ts` tags fetched Consolidated Report rows with the same `"recursion"` `runType` used for genuine recursion/eval rows. A code comment claims this tag keeps them "invisible" in the Runs tab table, but `src/components/legal/BenchmarkRunsHistory.tsx`'s `secondaryRows` filter is a deny-list (`runType !== "manual"`) that does not actually exclude them — so they render. ## Fix - Tag Consolidated Report rows with a new, distinct `"consolidated"` runType instead of reusing `"recursion"`. - Keep fetching/merging them (still needed so the Recursion tab gets live Pusher updates for in-flight/completed consolidated-report generation after a page refresh). - Change `BenchmarkRunsHistory.tsx`'s `secondaryRows` filter from a deny-list to an explicit allow-list (`runType === "recursion"`), so only genuine eval/recursion rows land in the table's secondary rows, and the new `"consolidated"` tag is excluded by construction. ## Scope Client-side/UI only — no API route, Prisma schema, or dispatch/webhook changes.
pitoi
added a commit
that referenced
this pull request
Aug 31, 2026
…chmarks Runs tab (complete fix) (#5209) ## Problem The Legal Benchmarks "Runs" tab for a task shows `LEGAL_BENCHMARK_CONSOLIDATED` runs (cross-run Consolidated Report comparisons, a separate Stakwork workflow/ID) mixed in with the task's actual `LEGAL_BENCHMARK_RUNNER` runs (workflow 57179). ## Root cause `src/hooks/useLegalBenchmarkRunList.ts` tags fetched Consolidated Report rows with the same `"recursion"` `runType` used for genuine eval/recursion rows. A code comment claims this tag keeps them "invisible" in the Runs tab table, but `src/components/legal/BenchmarkRunsHistory.tsx`'s `secondaryRows` filter is a deny-list (`runType !== "manual"`) that does not actually exclude `"recursion"`-tagged rows — so Consolidated rows render there. ## Fix (single, complete change — supersedes two abandoned prior attempts, PR #5204 and #5206, which split this across mismatched branches) 1. `src/hooks/useLegalBenchmarkRunList.ts`: add a new `"consolidated"` value to the `BenchmarkRunType` union; tag `rawConsolidatedRows` with `"consolidated"` instead of `"recursion"` when merging. Keep fetching/merging them (still needed for the Recursion tab's Pusher-driven live status after a page refresh). 2. `src/components/legal/RecursionBox.tsx`: update `existingConsolidated`'s filter to match the new `runType === "consolidated"` tag (instead of `"recursion"`), so the Recursion tab's own in-flight-consolidated-run lookup keeps working correctly with the renamed tag. 3. `src/components/legal/BenchmarkRunsHistory.tsx`: change `secondaryRows`'s filter from a deny-list (`runType !== "manual"`) to an explicit allow-list (`runType === "recursion"`), so only genuine eval/recursion rows land in the table; the new `"consolidated"` tag is excluded by construction. 4. `src/__tests__/unit/components/RecursionBox.test.tsx`: update the `makeConsolidatedRow` test fixture's `runType` field from `"recursion"` to `"consolidated"` to match the renamed tag, so the existing "detects a PENDING in-flight consolidated run via the allRuns prop" test continues to pass against the new logic. ## Scope Client-side/UI + test-fixture only — no API route, Prisma schema, or dispatch/webhook changes. All four edits land together in this one PR/branch, avoiding the split-branch mismatch that broke the two earlier attempts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Legal Benchmarks "Runs" tab for a task shows
LEGAL_BENCHMARK_CONSOLIDATEDruns (cross-run Consolidated Report comparisons, a separate Stakwork workflow) mixed in with the task's actualLEGAL_BENCHMARK_RUNNERruns (workflow 57179).Root cause
src/hooks/useLegalBenchmarkRunList.tstags fetched Consolidated Report rows with the same"recursion"runTypeused for genuine recursion/eval rows. A code comment claims this tag keeps them "invisible" in the Runs tab table, butsrc/components/legal/BenchmarkRunsHistory.tsx'ssecondaryRowsfilter is a deny-list (runType !== "manual") that does not actually exclude them — so they render.Fix
"consolidated"runType instead of reusing"recursion".BenchmarkRunsHistory.tsx'ssecondaryRowsfilter from a deny-list to an explicit allow-list (runType === "recursion"), so only genuine eval/recursion rows land in the table's secondary rows, and the new"consolidated"tag is excluded by construction.Scope
Client-side/UI only — no API route, Prisma schema, or dispatch/webhook changes.