Repository navigation
Conversation
…same document The #2401 test asserted an absolute 1 s per resolution. CI reached 0.89 s of test wall and a contended local run 600-915 ms per call, so a slow runner would fail it (finding 1326). Time a structured clone of the same document in the same run and require resolution under 25 clones, best of up to three rounds; a miss past 250 clones is the regression and is not retried. The reverted indexing fails at about 5,800 clones (123 s); at 2,000 models it already fails at about 340.
With four copies of the test sharing two CPUs a single round reached 17.7 clones (p50 4.6, 160 samples) against the bound of 25. Allow 50 and stop retrying past 500; the reverted indexing still fails at about 320 clones with 2,000 models and about 5,800 at full size.
…inly Review nits: the 4-copy rerun peaked at 19.7 clones, the shrunk mutant figure is about 320 with 2,000 models, and the loop stops once under the bound.
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
GitHub Actions
Failure
GitHub Actions
Failure
clawsweeper
Bot
added
P3
|
Codex review: needs maintainer review before merge. What this changesThis PR replaces a one-second Harness-resolution deadline with a bound relative to cloning the same document. Example: Resolve 50,000 model fallbacks against a 50,000-row catalog on a busy runner.
Review scores
ProductKind: Test only · Worth it: Yes · Fix scope: Complete Merge readiness✅ Ready for maintainer review Keep open: this focused test repair reduces runner-speed sensitivity while retaining the original regression cases; no actionable patch defect was found. Priority: P3 Before mergeNone. FindingsNone. Agent review detailsHow this fits togetherOCC resolves Configuration model policies into a Harness identity or a validation error; this conformance test guards resolution cost for large catalogs and shared fallback selections. flowchart TB
A[Configuration document] --> B[Harness resolver]
B --> C[Harness identity or refusal]
D[Conformance test] --> B
D --> E[Clone same document]
B --> F[Relative timing assertion]
E --> F
Technical reviewBest possible solution: Retain the existing regression inputs and semantic assertions while using the calibrated relative timing bound. Do we have a high-confidence way to reproduce the issue? The absolute deadline is verified in current main, but a runner-induced failure was not reproduced; the PR reports near-bound measurements and contention experiments. Is this the best way to solve the issue? The change is proportionate: it preserves independent Harness assertions, follows an existing relative-baseline pattern, and reports mutation evidence that the original quadratic regression still fails. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against ea23d9025c42. Provenance checked
TestingProof path: unit tests only. SecurityNone. EvidenceWhat I checked:
Likely related people:
Review metrics
LabelsLabel changes:
Label justifications:
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment Reviewed October 14, 2026, 7:06 AM ET / 11:06 UTC. |
Problem
The Harness resolution test from #2401 (50k fallbacks against a 50k-row catalog, and 50k entries sharing 50k fallbacks) asserts an absolute 1 s per resolution. In CI the whole test has reached 0.89 s of wall time (run 38545875684), and a contended local run takes 600-915 ms per call. A slow or busy runner would fail it (finding 1326).
Change
Tests only. The bound now compares resolution with a
structuredCloneof the same document, timed in the same run, as #2405 does for the large roster test. Each round times one resolution and then one clone. There are up to three rounds, and each time keeps its best. Resolution must stay under 50 clones. A round past 500 clones is the regression, not noise, so the test stops retrying and fails at once. Every round still checks the resolved Harness.Verification
All runs below are pinned to 2 CPUs (
taskset -c 0,1).configured-harness.tschange reverted (git apply -R), the full test fails after one round. The ratio is 5,845 (123.5 s against a 21 ms clone), and the test stops in 124 s. With the input shrunk to 2,000 models, the mutant fails at a ratio of about 320 in under 1 s. The fixed code at that size measures 1.2-2.Risks
Low, because this changes one test only. If a busy runner makes a clone look faster than resolution, a slow resolution could pass. Even so, the quadratic regression is 100 times past the bound.