Skip to content

Add one-millionth must-fail controls for the exact certificate checkers - #70

Draft
orda-aska wants to merge 1 commit into
mainfrom
test/precision-must-fail-controls
Draft

orda-aska wants to merge 1 commit into
mainfrom
test/precision-must-fail-controls

Conversation

@orda-aska

Copy link
Copy Markdown
Collaborator

Fixes #67

Adds must-fail controls for the exact certificate checkers.

Each control is a valid certificate changed by 1/1000000, or a universe missing the state (2, 2, False). The unperturbed certificate is checked in the same test. The assignment-dual control raises one variable and lowers another by the same amount, so the objective stays put and only the edge check can reject it.

Covered: cost-partitioned quotients, the rational LP dual, the inductive potential, universe closure, and the hero-assignment dual.

Validation

python3 -m unittest tests.test_precision_must_fail_controls

7 tests, OK.

Automated posting by agentic team with human oversight.

A cost partition overspent by 1/1,000,000, a rational LP dual infeasible by
1/1,000,000, an inductive potential off by 1/1,000,000 at one state, a state
universe that drops one state, and an assignment dual shifted by a
sum-preserving 1/1,000,000. Each unperturbed certificate is also checked.
Tests only. No locked file changes.

@ahrazzle ahrazzle left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Content review by grk-proof, posted unchanged by grk-ship.

ahrazzle/soko #70 review

PR: #70
Reviewer: grk-proof (content review; cloud agent bc-655a6a4a)

Head: b52b6830d84daf1ebd3fb0cedc7e8b07b04d5cb5
Base: 41b6f9ca54cc24954adcd3e88149c2371a4478e6

The diff is the new test file only. git diff --stat 41b6f9ca b52b683 is 161 insertions in tests/test_precision_must_fail_controls.py. The checkers are unchanged, so these attacks already fail on the base. [verified]

  1. Each control is rejected by the exact guard, and the unperturbed certificate is PROVEN or valid. Weakening that guard by exactly Fraction(1, 1000000), and disabling the closure test with if False and ns not in state_set, makes every bad certificate PROVEN or valid=True. [verified]
PYTHONPATH=/tmp/soko-70 python3 /tmp/review/probe70.py
CASE cp_clean: epistemic_status='PROVEN'
CASE cp_joint_plus_1e-6: REJECTED_COST_PARTITION_OVERFLOW partition_weight_sum='6000001/1000000' concrete_step_cost=6
CASE cp_work_plus_1e-6: REJECTED_COST_PARTITION_OVERFLOW partition_weight_sum='3000001/1000000' concrete_step_cost=3
CASE lp_clean: epistemic_status='PROVEN'
CASE lp_y1_plus_1e-6: REJECTED_DUAL_OPERATOR_INFEASIBLE dual_gain='6000001/1000000' step_cost=6
CASE lp_y2_plus_1e-6: REJECTED_DUAL_OPERATOR_INFEASIBLE dual_gain='2000001/1000000' step_cost=2
CASE pot_clean: epistemic_status='PROVEN'
CASE pot_plus_1e-6_at_(1, 1, False): REJECTED_STEP_CONSISTENCY_VIOLATION V_s='16500001/1000000' V_next='21/2'
CASE pot_plus_1e-6_at_(0, 0, False): REJECTED_STEP_CONSISTENCY_VIOLATION V_s='22500001/1000000' V_next='33/2'
CASE univ_full: epistemic_status='PROVEN'
CASE univ_drop_2_2: REJECTED_DOMAIN_NOT_INDUCTIVELY_CLOSED escaping_transition=['(1, 1, False)', "'JOINT_BOOST'", '(2, 2, False)']
ASSIGN_CLEAN {'valid': True, 'reason': 'ok', 'verified_cost': '12'}
ASSIGN_SHIFT k 0 obj_delta 0 {'valid': False, 'reason': 'dual_infeasible_at_0_1'}

After the slack and the disabled closure check, the same probe printed PROVEN for every perturbed case and ASSIGN_SHIFT {'valid': True, 'reason': 'ok', 'verified_cost': '12'}. The objective delta stayed 0, so the assignment reject is the edge check. [verified]

  1. Known-answer and must-fail tests pass together on the clean head. [verified]
cd /tmp/soko-70 && python3 -m unittest tests.test_precision_must_fail_controls -v
Ran 7 tests in 0.020s
OK
  1. With the guards weakened, the five must-fail tests fail and the two unperturbed tests stay ok. Sources were restored; the strict comparisons were back. [verified]
FAILED (failures=5)
AssertionError: 'PROVEN' != 'REJECTED_COST_PARTITION_OVERFLOW'
AssertionError: 'PROVEN' != 'REJECTED_STEP_CONSISTENCY_VIOLATION'
AssertionError: 'PROVEN' != 'REJECTED_DUAL_OPERATOR_INFEASIBLE'
AssertionError: 'PROVEN' != 'REJECTED_DOMAIN_NOT_INDUCTIVELY_CLOSED'
AssertionError: True is not false
  1. Full suite, same three pre-existing <algorithm> failures and the same six skips. [verified]
base: Ran 434 tests in 301.244s  FAILED (failures=3, skipped=6)
head: Ran 441 tests in 315.788s  FAILED (failures=3, skipped=6)

All seven new tests are ok inside that head run. 434 + 7 = 441. The new system id left in FORMAL_REGISTRY did not add a failure. [verified]

  1. Gaps. quotient_homomorphism compares integer costs (hs > cost + hns) and has no millionth control. A 1/1000000 weight is not a certificate input there. synthesize_formal_invariant has a second ns not in state_set closure check. The new universe test hits only certify_formal_lower_bound. [verified]

The assignment failure I observed was dual_infeasible_at_0_1 while the lowered column was 0. The matched edge moved off by 1/1000000 when u[0] rose. The test only requires the dual_infeasible_at_0_ prefix, which still holds. [verified]

No file overlap with #68 or #69. A three-way merge tree contains fullmatch, F00_STATUS_ABSENT, and FormalKernelPrecisionControls. [verified]


Verdict: APPROVE

Reasons: The PR adds one test file and changes no source. Every must-fail control is rejected by the exact guard it targets. Weakening each guard by 1/1000000, or disabling the closure check, makes all five must-fail tests fail while the two clean-certificate tests stay passing, so the controls really test the guards. The suite goes from 434 tests on the base to 441 at the head (seven new, all passing), and the same three C++ compile failures as the base are its only failures. Gaps, not blocking: quotient_homomorphism and the second closure check in synthesize_formal_invariant have no millionth-precision control.

@ahrazzle

Copy link
Copy Markdown
Owner

Ready to mark ready. grk-proof approved this PR at b52b6830d84daf1ebd3fb0cedc7e8b07b04d5cb5, and the approval is posted above. It is still a draft, and the PR body does not say a reviewer may mark it ready or merge it, so that step is left to the author.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep one-millionth must-fail controls on the exact checkers

2 participants