Skip to content

Reject a certificate that names no recognized status - #69

Draft
orda-aska wants to merge 2 commits into
mainfrom
fix/audit-status-fail-closed
Draft

orda-aska wants to merge 2 commits into
mainfrom
fix/audit-status-fail-closed

Conversation

@orda-aska

Copy link
Copy Markdown
Collaborator

Fixes #66

audit_soko_certificate no longer reports VERIFIED when the certificate names no recognized status.

An empty object and an unknown status are findings. REJECTED and CONJECTURE keep their own status. CONJECTURE still needs no pins. A VERIFIED record with both digests still passes. The existing F01 through F08 detector test still passes.

Validation

python3 -m unittest tests.test_audit_status_fail_closed tests.test_mde_soko.MdeSokoTests.test_f01_through_f08_certificate_soundness_detectors

5 tests, OK.

Automated posting by agentic team with human oversight.

An empty object and an unknown status produced no findings, and the
auditor reported VERIFIED. A rejected or conjectural record was relabeled
the same way. Those records now keep their own status, and a missing or
unknown status is a finding. CONJECTURE still needs no pins.

@ahrazzle ahrazzle left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Content review by grk-proof, posted unchanged by grk-ship.

ahrazzle/soko #69 review

PR: #69
Reviewer: grk-proof (content review; cloud agent bc-655a6a4a)

Head: f11b35871498843286019a485f13c00716cd00d0
Base: 41b6f9ca54cc24954adcd3e88149c2371a4478e6

  1. Empty, blank, and unknown certificates come back VERIFIED on the base and REJECTED on the head. [verified]
PYTHONPATH=/tmp/soko-base python3 /tmp/review/probe69.py
empty: passed=True status='VERIFIED' rules=[]
blank: passed=True status='VERIFIED' rules=[]
banana: passed=True status='VERIFIED' rules=[]
rejected: passed=True status='VERIFIED' rules=[]
conjecture: passed=True status='VERIFIED' rules=[]
verified_pins: passed=True status='VERIFIED' rules=[]

PYTHONPATH=/tmp/soko-69 python3 /tmp/review/probe69.py
empty: passed=False status='REJECTED' rules=['F00_STATUS_ABSENT']
blank: passed=False status='REJECTED' rules=['F00_STATUS_ABSENT']
banana: passed=False status='REJECTED' rules=['F00_STATUS_UNKNOWN']
rejected: passed=True status='REJECTED' rules=[]
conjecture: passed=True status='CONJECTURE' rules=[]
verified_pins: passed=True status='VERIFIED' rules=[]
proven_pins: passed=True status='VERIFIED' rules=[]
lower_pins: passed=True status='LOWER_BOUND' rules=[]
  1. Known-answer tests pass at the head. [verified]
cd /tmp/soko-69 && python3 -m unittest tests.test_audit_status_fail_closed tests.test_mde_soko.MdeSokoTests.test_f01_through_f08_certificate_soundness_detectors -v
Ran 5 tests in 0.000s
OK

A pinned VERIFIED record stays VERIFIED. Bare VERIFIED, PROVEN, and LOWER_BOUND still fail F03 and F08. CONJECTURE still needs no pins. [verified]

  1. Restoring the base auditor and keeping the new tests fails three of the four new tests. The pinned VERIFIED control still passes. Restored afterward; F00_STATUS_ABSENT and F00_STATUS_UNKNOWN were back. [verified]
git show 41b6f9ca54cc24954adcd3e88149c2371a4478e6:mde_soko/domain_oracle.py > mde_soko/domain_oracle.py
python3 -m unittest tests.test_audit_status_fail_closed -v
FAILED (failures=3)
AssertionError: empty certificate passed: {'passed': True, 'epistemic_status': 'VERIFIED', ...}
AssertionError: 'VERIFIED' != 'REJECTED'
AssertionError: unknown status passed: {'passed': True, 'epistemic_status': 'VERIFIED', ...}
  1. Full suite, same command as #68. Same three pre-existing <algorithm> compile failures. No new failure. [verified]
base: Ran 434 tests in 301.244s  FAILED (failures=3, skipped=6)
head: Ran 438 tests in 317.524s  FAILED (failures=3, skipped=6)

The four new tests and test_f01_through_f08_certificate_soundness_detectors are ok in the head suite. The extra four tests are exactly this PR. [verified]

  1. This is why the verdict is request changes. Issue #66 says any recognized status keeps its own name. [source claim] A pinned PROVEN certificate still comes back epistemic_status='VERIFIED'. A pinned LOWER_BOUND keeps LOWER_BOUND. The new tests never build a passing PROVEN certificate, so they do not lock the issue sentence. The mapping is deliberate in the diff (status in ("VERIFIED", "PROVEN")), and it is not a fail-open: the record already cleared F03 and F08. It is still the relabel the issue forbids. [verified]

Other status paths, not the blocking item. None becomes "NONE" and is F00_STATUS_UNKNOWN, not absent. Still rejected. 0 and True are unknown and rejected. " verified " and "VERIFIED\n" are stripped to VERIFIED and pass when the pins are good. That strip was already on the base. [verified]

REFUSED is a Plane 1 label in mde_soko/spec/plane1_soko_spec.md and is not in RECOGNIZED_CERT_STATUSES. On the head it is F00_STATUS_UNKNOWN. On the base it was VERIFIED. Fail-closed, and not preserved. [verified]

LINT_PASS, LOWER_BOUND_UNWITNESSED, and UNVERIFIED_CALLER_BOUNDS pass with no pins and keep their own names. passed means no finding fired. REJECTED and CONJECTURE are the same shape, and the new tests require that. A caller that trusts passed and ignores the status still accepts those records. The VERIFIED label is gone. [verified]

VERIFIED_ON_BOUNDED_REACHABLE_SUBGRAPH with no complete_space still fails F06. With complete_space: True it passes and keeps that status. [verified]

A witness digest plus \n still passes on this head. That is #68's bug. The merge of #68 and #69 is clean and keeps both edits. [verified]


Verdict: REQUEST CHANGES

Reasons: The fail-closed part is right. An empty, blank, or unknown status is rejected at the head, and three of the four new tests fail on the base auditor. The PR doesn't meet its own spec, though. Issue #66 and the PR body both say "Any other recognized status keeps its own name". At the head, a pinned PROVEN certificate still comes back as epistemic_status='VERIFIED', because of status in ("VERIFIED", "PROVEN"), and no test covers a passing PROVEN record. Required: return PROVEN as PROVEN, and add a test with a pinned PROVEN certificate. Owner decision, not blocking: REFUSED, a Plane 1 label, is now rejected as unknown. Also, passed=True still comes back for REJECTED and CONJECTURE records, so a caller that reads only passed would still accept them.

@orda-aska

Copy link
Copy Markdown
Collaborator Author

The change requested in the review is in commit 760376f on fix/audit-status-fail-closed.

5. A passing PROVEN certificate came back as VERIFIED [verified]

In audit_soko_certificate, the output status is now status when no finding fired and REJECTED otherwise. The status in ("VERIFIED", "PROVEN") mapping is gone. A pinned PROVEN record now returns PROVEN. LOWER_BOUND and VERIFIED are unchanged.

Tests in tests/test_audit_status_fail_closed.py:

  • test_pinned_proven_keeps_its_own_status builds a PROVEN certificate with good pins and a witness digest. It expects passed and PROVEN.
  • test_every_passing_recognized_status_keeps_its_name loops over VERIFIED, PROVEN, LOWER_BOUND, the pin-free statuses, and VERIFIED_ON_BOUNDED_REACHABLE_SUBGRAPH with complete_space: True.
  • test_bare_proven_is_still_rejected checks that PROVEN without pins still fails F03 and F08.

Before and after, on the same tests:

python3 -m unittest tests.test_audit_status_fail_closed     (previous head)
FAILED (failures=2)
python3 -m unittest tests.test_audit_status_fail_closed     (this commit)
Ran 7 tests ... OK

The same 7 tests pass under python3 -O -B -W error. python3 -m soko_mcp.integrity reports source_integrity_verified: true. No integrity-locked file changed.

Full suite [verified]

python3 -m unittest discover -s tests
Ran 323 tests in 319.054s
FAILED (failures=5, errors=20, skipped=10)

On this machine the 20 errors are an import of the missing pysat module. The 5 failures are in test_experiment_record and test_record_system. They are in files this commit does not touch. The previous head, run the same way, gives Ran 320 tests, failures=5, errors=20, skipped=10, and the list of failing test names is identical. The 3 extra tests are the new ones and they pass.

Not changed, owner decision [proposed]

Two items in the review are left as they are. REFUSED is rejected as unknown, and passed is still true for REJECTED and CONJECTURE records. Both need a ruling before code changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Certificate auditor reports VERIFIED for an empty certificate

2 participants