Frontier-resolution certificate + near-duplicate option advisory - #117
Merged
Conversation
CI failed test_certify_explicit_run_ids with KeyError: 'run_id'. Not a
behavior change — a race.
`solve run` returns the result inline when it finishes inside the wait budget
and a {status:"running", job_id} handle otherwise, so which branch a caller
gets depends on machine speed. TestCertify subscripted ["run_id"] directly, so
it passes on a fast machine and fails on a slower runner. The CI timings put
these exact HiGHS solves right at the boundary — the failing test at 10.01s
and its sibling at 18.62s, against a ~10s inline budget.
Adds TestCertify._run: poll `solve status` when handed a handle, then take the
run id. Applied to all six solves in the class, since any of them can cross the
threshold, not just the one that happened to fail.
Verified by forcing the backgrounded path (wait_seconds=0 on every run) — the
helper resolves where the bare subscript raised. Certify tests 11 passed.
Note for the surface, not fixed here: test_solve_async documents the assumption
that "a tiny solve finishes within the inline budget." A small binary problem's
exact solve sitting at that boundary on ordinary hardware suggests the advertised
scale band is optimistic — the corpus runs showed the same thing at 300 options
(~100s, 8 polls, still reported as "interactive").
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Two additions, both generalized from patterns surveyed in external optimization tooling and reduced to what Frontier didn't already have.
frontier_resolutionblock in theexplore certifycertificate (engine/explorer.py). Wherecoverage/completenessaudit the NSGA and exact runs against each other, this bounds the reported frontier against the true continuous frontier: on a 2-objective LP/QP overlay, the certified points' chords (above, by convexity) and their own ε-floor shadow-price tangents (below, by the envelope theorem) sandwich the tradeoff curve; a dual-free monotonicity floor covers segments without a priced tangent, so a missing dual loosens the bound rather than voiding it. The widest chord-to-envelope gap is a certified resolution — "between the anchors, no feasible plan improves the primary over straight-line interpolation of the certified points by more than X" — the frontier-side sibling ofexplore audit's whole-region guarantee. Consumes only duals already on the run (no extra solves); present-as-nullon shapes that can't carry the claim (3+ objectives, integer overlays, no tension), matchingcompleteness. The widest segment is named with its endpoint solution ids as a targeted-tightening handle.near_duplicate_optionsadvisory indata_metrics(engine/metrics.py). Pairs of complete-scored options within 2% of each objective's score spread on every objective — the "same bet entered twice" case strict dominance structurally misses (an exact tie dominates nothing). Advisory only: capped at 10 pairs + total (thedominated_optionsconvention); the engine flags, the user merges or confirms. A dominated-by-a-hair twin rides both lists; the skill copy says the twin read ("pick either") is the truthful framing there.Shared cleanups from review:
_floor_dual+_FLOOR_ROLESextraction (the ε-floor role tuple had 6 literal sites),_complete_optionsshared between the dominance and twin detectors.Docs: architecture.md (certify tool row, Solver Backends paragraph, data-metrics diagram), README (one clause),
solution_interpreter› Reading the Certificate (new block paragraph, fixed a stale block count),data_collection› Score Quality Signals (twin bullet), explore tool docstring.Review and testing
test_mcp_stdio_source.pybootstrap.The agent test also surfaced pre-existing payload frictions outside this diff (the
exact: falseflag besidesolver_used: "highs"; knee rationale at corner-adjacent inflections; regret id-space mixing; hypervolume scale mismatch across payloads) — filed separately, not addressed here.