diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b934aac0d..193e8e07f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ ## Unreleased +### Fixed: the harness auditor judged absence from a file it had only partly read (2026-09-09) +- **Why:** `scripts/harness/audit.py` asks a model to list signals a runtime exposes that ClawMetry's adapter does NOT capture, then files an issue per gap. That task is entirely about reporting ABSENCE, which makes it uniquely vulnerable to truncation: a clipped file reads exactly like a missing feature and the model has no way to tell. `_adapter_source` read the first 60,000 characters; `clawmetry/adapters/openclaw.py` is 193,340. Both runtimes the OSS audit covers map to that one file, so every run judged a two-thirds-clipped adapter. Then the prompt told the model "verify against the FULL adapter above (it is provided in full)", which converts "I did not see it" into "it is not there". +- **What it cost:** issue #5750, filed automatically at severity HIGH, claiming no code path reads `NEMOCLAW_TRACE_FILE`. That reader is at `clawmetry/adapters/openclaw.py:1705`, roughly 18k characters past the cut. It resolves `NEMOCLAW_TRACE_FILE`, then `NEMOCLAW_TRACE_DIR`, then the harness default, and puts `nemoclawOnboardTraceStatus`, `nemoclawOnboardTraceSpanCount`, `nemoclawOnboardTraceErrors` and `nemoclawOnboardSlowSpans` on the detection record. `clawmetry/adapters/nemo.py` reads it too, and REQ-OBS-RSO-034 specifies the whole capability, delivered by PR #5198. The cost of this class of bug is not a wasted CI minute, it is a high-severity issue a person or an agent has to read, reproduce and disprove against code that already shipped. +- **The second time, so the fix is not a bigger number.** The docstring records the first: aider's conditional COST at line ~527 was cut, the audit flagged "no COST", and the cap was raised to 60k in response. A file that grows defeats any constant. Now the budget is far above any adapter here AND `_adapter_source` reports whether it trimmed instead of trimming silently; when it does trim the prompt says so rather than claiming completeness; and `_adapter_index` always carries every `def` and every UPPER_CASE string from the WHOLE file, so an absence claim stays checkable even under trimming. +- **The four sibling issues are NOT false positives, and that was checked rather than assumed.** #5746 to #5749 came from the same truncated runs, but each reasons about functions inside the old window (`_openclaw_doctor_findings` at char 4,899, `_clawrouter_detect` at 6,053, `_sandbox_inference_configs` at 46,194), and grepping the full adapter confirms every one of their claims: no share/publish reader, no meeting-library reader, no connected-accounts reader, and only `_gateway_migration_warning` rather than update candidate state. #5750 was the only fiction of the five. +- **Verified:** `tests/test_harness_audit_reads_whole_adapter.py` auto-discovers from `manifest.json`, so an adapter that outgrows the budget tomorrow fails there instead of quietly filing fiction, and one case pins the `NEMOCLAW_TRACE_FILE` witness past the old 60k mark so the test keeps proving something. Guard proven red: restoring the cap and the completeness claim reds 5 of 6, including both audited runtimes. Closes #5750. + ### Fixed: the first-run report published our probe map, and the account name with it (2026-09-09) - **Why:** the first-run report (0.12.844) said where ClawMetry looked by serving the EXPANDED probe path for all 30 runtimes from `/api/entitlement/runtime-detection`, in both payload builders, and rendering them as a table in the Overview panel. Founder review of a screenshot rejected it. Two exposures, and the repo already had a rule against one of them. First, an expanded path carries the account name (`/Users//...`) onto the screen most likely to reach a screenshot, a screen-share, a support ticket or a marketing shot of a fresh install; `AC-OBS-RSO-030.7` already forbids a report carrying a full filesystem path, and the detector surface holds itself to it. Second, a complete, copy-pasteable map of where the product looks for every supported runtime is a different artefact from the same table sitting in a source file: rendering it ships it with every install, and runtime coverage is the deliberate open-core split. - **What:** the map moved to `clawmetry diagnose`, a local command whose output a person runs and chooses to share. Every reported path is home-collapsed (`~/.codex/sessions`) at the probe itself, so even that output pastes into an issue without naming anyone; `~` is the home directory on every supported platform and the shell expands it, so the tilde form is exactly as actionable. Both runtime-detection payload builders stop forwarding `paths`. The panel says HOW MANY runtimes were checked, which is what makes "nothing detected" trustworthy, and points at the command; its lead sentence no longer promises a list it does not show.