[danielbrownjr]

Summary

The first, deliberately narrow slice of Assess: offline evaluation of completed DragonSniff evidence against acceptance criteria that the product supplies.

DragonSniff still never defines product policy. A finding is derived output, and PASS is not a safety certification. Every report says so.

The two original commits are preserved, followed by one correctness-fix commit:

1. docs: define the Assess evidence-assessment contract — docs/assess.md fixes the semantics before any code. It also replaces the thermal-capture.md sentence saying DragonSniff never decides whether results pass.

2. assess: add the first offline evidence-assessment slice — the implementation and tests.

3. assess: preserve uncertainty and exact numeric criteria — fixes the four review findings and advances the contract to semantics version 2.

Contract highlights

• sequence is arrival order. Host monotonic_ns is the only duration arithmetic. Wall time is provenance only.
• A polled value is known only somewhere inside an interval: [request, completion] for Dragon HTTP, [observed_monotonic_ns, monotonic_ns] for PrusaLink. No exact sample time is invented.
• Failed, unparsed, missing, null, wrong-type, NaN, infinite, or malformed observations are unknown and never coerced.
• Strict no-hold is the default. A continuous claim can't be proven from polls without a declared, bounded max_hold_s, which the report echoes.
• A closed run and contiguous sequence prefix from 1 through terminal are required. Any gap makes every check INCONCLUSIVE, with all time unknown; even explicit windows require a terminal. This prevents deletion from repairing malformed/duplicate request pairing into a decisive result.
• Post-terminal records cannot affect findings, including duplicate IDs and later run markers. Explicit non-null error fields invalidate observations for both sources.
• Integer criteria remain exact. Seconds use exact parsed-number ratios: holds/start offsets round down, end offsets round up. Fraction decisions use integer cross-products.
• holds_throughout: FAIL only on a proven violation fully inside the window. PASS only when the whole window is proven true. Otherwise INCONCLUSIVE, with reason codes.
• time_in_band: time-weighted in integer nanoseconds. The in-band fraction is reported as [lower, upper]. PASS only if lower >= min_fraction, FAIL only if upper < min_fraction. min_fraction = 0 is rejected, because it would pass with no evidence.
• Deleting evidence never moves a finding toward PASS. This is stated formally in the contract and tested as a property.

Implementation

src/dragonsniff/assess/ has five modules, about 1,000 lines. The API takes bytes and returns text: assess(evidence, profile, *, metadata=None, assessed_at=None).

• evidence.py: SHA-256 and byte length. Refuses corruption: duplicate or out-of-order sequence, torn tail, bad JSON, bad core fields, non-terminal session metadata. Reports gaps and cross-thread timestamp inversions without rejecting them.
• profile.py: closed-schema TOML via stdlib tomllib. Unknown keys are rejected at every level, so on_fail, action, webhook, and similar keys cannot enter a profile.
• engine.py: observation extraction, the three-valued timeline, and the two checks.
• report.py: the envelope, with assessment_format 1, assess_semantics_version 2, profile and evidence identity, assumptions, coverage, reason codes, and decisive record sequences. JSON is deterministic and has no timestamp unless the caller supplies assessed_at.

One change outside assess/ — please review

dragonsniff/__init__.py eagerly imported Observer. Any subpackage import therefore loaded client, prusalink, socket, ssl, http.client, and urllib. Assess's own imports were clean, but the process still loaded the whole network stack.

The package re-exports are now lazy (PEP 562). from dragonsniff import Observer and __all__ behave exactly as before; __version__ stays eager. import dragonsniff.assess now loads only its six modules, and a subprocess test enforces that. scripts/verify_release_wheel.py now also requires dragonsniff/assess/__init__.py.

Validation after correctness fixes

• Four historical regressions: each fails on exactly 5e20cf0325aa4137e88ba54908ed7722ee45f6d4 with an assertion demonstrating its intended semantic defect, and passes unchanged on the fixed tree. No helper/API/import failures count as proof.
• Full suite: 352 Python tests with warnings as errors, 1 expected platform skip. 30/30 JS tests and both syntax checks pass. compileall and git diff --check pass.
• Focused Assess suite: 75 tests pass, including import/read-only boundaries, parse/decode-error cases in both sources, terminal-horizon isolation, exact integers around 2^53 and up to 10^400, nanosecond rounding, and deterministic replay.
• Mutation testing: 15/15 targeted mutants killed by semantic assertions. A hold-expiry coverage gap found during mutation testing received a dedicated regression.
• Deletion properties: existing 1,000 randomized deletions plus 3,200 expanded trials covering duplicate requests/responses, reused and malformed IDs, reversed completions, error-bearing responses, and ambiguous run markers. Both assertion families are checked, including fraction-bound widening.
• Reproduce historical proof and mutation checks with python scripts/verify_assess_correctness.py. It operates in temporary copies without changing the checkout.
• The correctness-fix validation was run from source; an installed-wheel validation was not rerun for this follow-up.

Prior real-evidence gate: #95 Long Haul captures

These results were originally produced before the correctness fixes and are retained for provenance. The exact two captures were subsequently recovered, verified by SHA-256, and rerun against the corrected semantics-v2 head e9c32ad; the real-capture acceptance gate passed.

Both files were verified by content before use, not by filename.

	run 7016ab13f3f449479d632125e3224877	run 1dd62a5fba694212b9e20315f14453a1
SHA-256	86175047…	859346b7…
Records / sequences	12,490, 1–12,490, no gaps	12,490, 1–12,490, no gaps
State observations extracted	5,761 (all valid)	5,761 (all valid)
Manual figures from Assess-extracted observations	mean 61.46, p2p 16.4, ±1 °C 22.5% — match	mean 61.51, p2p 16.1, ±1 °C 17.3% — match
mode == power_on from heater start to end	FAIL at seq 5997: the known lease expiry	PASS (hold 10 s)
mode == power_on, settled window	PASS (hold 10 s)	PASS (hold 10 s)
Time-weighted ±1 °C, settled, hold 10 s	[22.2%, 22.7%]	[17.1%, 17.6%]
Same, strict no-hold	[0%, 100%], INCONCLUSIVE	[0%, 100%], INCONCLUSIVE
Replay	byte-identical	byte-identical

The loader reproduces the independent manual analysis exactly from the observations it extracts. The time-weighted in-band intervals bracket the manual sample-weighted figures. The known lease-expiry truncation is found at the exact sequence the manual analysis identified.

The captures are 20 MB of device evidence, so they are not committed. The gate ran from a local script.

Not in this slice

Live evaluation, UI, server routes, a CLI, events and triggers, latency, other measurements, and multi-signal predicates. The contract says so explicitly.