Repository navigation
Conversation
An empty object and an unknown status produced no findings, and the auditor reported VERIFIED. A rejected or conjectural record was relabeled the same way. Those records now keep their own status, and a missing or unknown status is a finding. CONJECTURE still needs no pins.
ahrazzle
left a comment
There was a problem hiding this comment.
Content review by grk-proof, posted unchanged by grk-ship.
ahrazzle/soko #69 review
PR: #69
Reviewer: grk-proof (content review; cloud agent bc-655a6a4a)
Head: f11b35871498843286019a485f13c00716cd00d0
Base: 41b6f9ca54cc24954adcd3e88149c2371a4478e6
- Empty, blank, and unknown certificates come back
VERIFIEDon the base andREJECTEDon the head. [verified]
PYTHONPATH=/tmp/soko-base python3 /tmp/review/probe69.py
empty: passed=True status='VERIFIED' rules=[]
blank: passed=True status='VERIFIED' rules=[]
banana: passed=True status='VERIFIED' rules=[]
rejected: passed=True status='VERIFIED' rules=[]
conjecture: passed=True status='VERIFIED' rules=[]
verified_pins: passed=True status='VERIFIED' rules=[]
PYTHONPATH=/tmp/soko-69 python3 /tmp/review/probe69.py
empty: passed=False status='REJECTED' rules=['F00_STATUS_ABSENT']
blank: passed=False status='REJECTED' rules=['F00_STATUS_ABSENT']
banana: passed=False status='REJECTED' rules=['F00_STATUS_UNKNOWN']
rejected: passed=True status='REJECTED' rules=[]
conjecture: passed=True status='CONJECTURE' rules=[]
verified_pins: passed=True status='VERIFIED' rules=[]
proven_pins: passed=True status='VERIFIED' rules=[]
lower_pins: passed=True status='LOWER_BOUND' rules=[]
- Known-answer tests pass at the head. [verified]
cd /tmp/soko-69 && python3 -m unittest tests.test_audit_status_fail_closed tests.test_mde_soko.MdeSokoTests.test_f01_through_f08_certificate_soundness_detectors -v
Ran 5 tests in 0.000s
OK
A pinned VERIFIED record stays VERIFIED. Bare VERIFIED, PROVEN, and LOWER_BOUND still fail F03 and F08. CONJECTURE still needs no pins. [verified]
- Restoring the base auditor and keeping the new tests fails three of the four new tests. The pinned
VERIFIEDcontrol still passes. Restored afterward;F00_STATUS_ABSENTandF00_STATUS_UNKNOWNwere back. [verified]
git show 41b6f9ca54cc24954adcd3e88149c2371a4478e6:mde_soko/domain_oracle.py > mde_soko/domain_oracle.py
python3 -m unittest tests.test_audit_status_fail_closed -v
FAILED (failures=3)
AssertionError: empty certificate passed: {'passed': True, 'epistemic_status': 'VERIFIED', ...}
AssertionError: 'VERIFIED' != 'REJECTED'
AssertionError: unknown status passed: {'passed': True, 'epistemic_status': 'VERIFIED', ...}
- Full suite, same command as #68. Same three pre-existing
<algorithm>compile failures. No new failure. [verified]
base: Ran 434 tests in 301.244s FAILED (failures=3, skipped=6)
head: Ran 438 tests in 317.524s FAILED (failures=3, skipped=6)
The four new tests and test_f01_through_f08_certificate_soundness_detectors are ok in the head suite. The extra four tests are exactly this PR. [verified]
- This is why the verdict is request changes. Issue #66 says any recognized status keeps its own name. [source claim] A pinned
PROVENcertificate still comes backepistemic_status='VERIFIED'. A pinnedLOWER_BOUNDkeepsLOWER_BOUND. The new tests never build a passingPROVENcertificate, so they do not lock the issue sentence. The mapping is deliberate in the diff (status in ("VERIFIED", "PROVEN")), and it is not a fail-open: the record already clearedF03andF08. It is still the relabel the issue forbids. [verified]
Other status paths, not the blocking item. None becomes "NONE" and is F00_STATUS_UNKNOWN, not absent. Still rejected. 0 and True are unknown and rejected. " verified " and "VERIFIED\n" are stripped to VERIFIED and pass when the pins are good. That strip was already on the base. [verified]
REFUSED is a Plane 1 label in mde_soko/spec/plane1_soko_spec.md and is not in RECOGNIZED_CERT_STATUSES. On the head it is F00_STATUS_UNKNOWN. On the base it was VERIFIED. Fail-closed, and not preserved. [verified]
LINT_PASS, LOWER_BOUND_UNWITNESSED, and UNVERIFIED_CALLER_BOUNDS pass with no pins and keep their own names. passed means no finding fired. REJECTED and CONJECTURE are the same shape, and the new tests require that. A caller that trusts passed and ignores the status still accepts those records. The VERIFIED label is gone. [verified]
VERIFIED_ON_BOUNDED_REACHABLE_SUBGRAPH with no complete_space still fails F06. With complete_space: True it passes and keeps that status. [verified]
A witness digest plus \n still passes on this head. That is #68's bug. The merge of #68 and #69 is clean and keeps both edits. [verified]
Verdict: REQUEST CHANGES
Reasons: The fail-closed part is right. An empty, blank, or unknown status is rejected at the head, and three of the four new tests fail on the base auditor. The PR doesn't meet its own spec, though. Issue #66 and the PR body both say "Any other recognized status keeps its own name". At the head, a pinned PROVEN certificate still comes back as epistemic_status='VERIFIED', because of status in ("VERIFIED", "PROVEN"), and no test covers a passing PROVEN record. Required: return PROVEN as PROVEN, and add a test with a pinned PROVEN certificate. Owner decision, not blocking: REFUSED, a Plane 1 label, is now rejected as unknown. Also, passed=True still comes back for REJECTED and CONJECTURE records, so a caller that reads only passed would still accept them.
|
The change requested in the review is in commit 5. A passing PROVEN certificate came back as VERIFIED [verified] In Tests in
Before and after, on the same tests: The same 7 tests pass under Full suite [verified] On this machine the 20 errors are an import of the missing Not changed, owner decision [proposed] Two items in the review are left as they are. |
Fixes #66
audit_soko_certificateno longer reports VERIFIED when the certificate names no recognized status.An empty object and an unknown status are findings. REJECTED and CONJECTURE keep their own status. CONJECTURE still needs no pins. A VERIFIED record with both digests still passes. The existing F01 through F08 detector test still passes.
Validation
5 tests, OK.