Per-statement encodability in check_agreement#69
Merged
Conversation
check_agreement() previously aborted the entire contract on the first UnencodableConstruct/NonlinearArithmetic, before any of the seven checks ran. Encodability is now per-statement: a statement that fails to encode is recorded in CheckResult.unencodable (text/reason/verb) and excluded; every remaining statement still proceeds through all seven checks. encodable is now False only when zero statements encoded; checked counts encoded deontic entries; skipped_reason is set only when encodable is False. _DeonticEntry.index stays dense over successfully-encoded entries only, so Z3 tracked-assertion names never collide. check 7's _check_constant_predicate also surfaces an unencodable `define` via the new field, in addition to its existing inconclusive finding. Version bump 0.17.1 -> 0.18.0 (additive field, no breaking change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
check_agreement()previously aborted the entire contract when any single statement failed to encode — the abort happened inside_collect_deontic_entries, before any of the seven checks ran, sofindingswas never computed rather than computed-and-discarded. This makes encodability per-statement: a statement that fails to encode is recorded (text/reason/verb) in a newCheckResult.unencodablelist and excluded; every remaining statement still proceeds through all seven checks.UnencodableStatementdataclass (text,reason,verb) added alongsideFinding.CheckResultgainsunencodable: list[UnencodableStatement] = field(default_factory=list)— last, defaulted, so existing positional construction sites keep working._collect_deontic_entriesand_check_constant_predicate(check 7) now each return(results, unencodable)tuples, catchingUnencodableConstruct/NonlinearArithmeticper statement instead of letting them propagate to the outer contract-level catch.checkedcounts statements that actually encoded;encodableisFalseonly when zero statements encoded;skipped_reasonis set only whenencodableisFalse;unencodablemay be non-empty whileencodableisTrue.Consistency invariants satisfied
encodable === (checked > 0)holds in everyCheckResultpath, including both outerexceptbranches (unencodable=[]added to each).unencodableis always a list, never null.UnencodableStatement.textisrender(node)— the same canonical rendering used elsewhere, untouched, so the join with Observatory'snode.canonicalholds._DeonticEntry.indexstays dense —index=len(entries)evaluated after skipping unencodable statements, so Z3 tracked-assertion names never collide.entrieslist.A real edge case this surfaced
A contract containing only a
define(no require/forbid/permit/expect) and zero deontic statements now reportsencodable=False/checked=0even when that onedefine's own check-7 encoding succeeds and produces a finding — becausechecked/encodableare scoped strictly to deontic entries per the locked spec's literal implementation, andfindingsis never gated onchecked. Updatedtest_check_source_finds_constant_predicate_regression_for_define_nodeto match (previously this combination —checked=0yetencodable=True— was only possible becauseencodablewas hardcodedTrueon any non-exceptional path).Test plan
pytest tests/— 1851 passed, 0 failed, 0 skipped.pytest tests/test_vocabulary.py— 18 passed; reserved-word count unchanged at 61.pytest tests/test_checker.py— 137 passed, including 6 new tests covering: partial encodability, total failure, clean-contract regression (unencodable == []), nonlinear exclusion alongside a clean statement, index density across a mid-list exclusion, and an unencodabledefinethat still emits its inconclusive finding._collect_deontic_entries/_check_constant_predicatecall sites bulk-fixed; two assertions moved fromskipped_reasoncontent tounencodable[].reasonsinceskipped_reasonis now a generic string for the all-unencodable case).🤖 Generated with Claude Code