Add an independent IK baseline and restart ablation - #9
Merged
Merged
Conversation
Freeze the target cohort, solver configurations, paired analyses, common success rule, and outcome-retaining verification gates before implementing or running the comparison. Generated-by: OpenAI Codex (GPT-5; accessed 2026-08-18) Signed-off-by: Dylan Gallagher <76841713+Dylan-Gallagher@users.noreply.github.com>
Implement the frozen four-variant comparison, common FK-based outcome rule, full CSV/report rendering, paired statistics, and byte-exact artifact verification without changing the library solver. Generated-by: OpenAI Codex (GPT-5; accessed 2026-08-18) Signed-off-by: Dylan Gallagher <76841713+Dylan-Gallagher@users.noreply.github.com>
Render the predeclared exact paired-test probability in scientific notation instead of rounding a nonzero value to 0.000000. Solver configurations, targets, outcome rules, and result rows are unchanged. Generated-by: OpenAI Codex (GPT-5; accessed 2026-08-18) Signed-off-by: Dylan Gallagher <76841713+Dylan-Gallagher@users.noreply.github.com>
Retain all 1,000 targets and 4,000 solver outcomes, add the restart ablations and independent k baseline to the evidence index, record exact validation provenance, and replay the artifacts in CI. Generated-by: OpenAI Codex (GPT-5; accessed 2026-08-18) Signed-off-by: Dylan Gallagher <76841713+Dylan-Gallagher@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
k0.32.0 on the existing 1,000-target full-pose cohortResults
k, home onlyk, home + the same 8 restart vectorsFor the restart-enabled variants, both solve 714 targets, arm-lab alone solves 264,
kalone solves four, and neither solves 18. Their union solves 982/1,000. The exact paired sign/McNemar value is8.998e-73, reported only as a descriptive statistic for this frozen cohort.The pre-existing 97.8% arm-lab control was known before the protocol was frozen and is explicitly labeled non-blind. The cold-start, independent-solver, paired-cell, and union outcomes were not known before the evaluator ran.
Reproducibility
d9078f1b95a159cc09acf2995f895cd91c23aa6e6b3320a5e73406eaa9c497eeba3bc96960322e62Validation
cargo fmt --all -- --checkClaim boundaries
Targets are generated by FK from in-limit configurations, so they are reachable under this exact model but are not Cartesian-uniform or task-representative. Evaluation is collision-unaware and simulation-only. Maximum starts and iteration caps are matched, but per-iteration work, update rules, damping, and early stopping differ; this is not a runtime, compute, hardware, sim-to-real, workspace-wide, or universal solver ranking.
Generated-by: OpenAI Codex (GPT-5; accessed 2026-08-18)