Skip to content

fix: clarify observation and runtime refusal diagnostics - #727

Merged
steipete merged 2 commits into
mainfrom
fix/phase5-observation-refusal-guidance-20260912
Sep 13, 2026
Merged

fix: clarify observation and runtime refusal diagnostics#727
steipete merged 2 commits into
mainfrom
fix/phase5-observation-refusal-guidance-20260912

Conversation

@steipete

Copy link
Copy Markdown
Collaborator

Observation binding failures used one sentence template for both target fields and diagnostic phrases, producing errors such as “did not match the requested unexpected application evidence.” The message now identifies the inconsistent evidence directly. Generic runtime-refusal guidance asks callers to verify the selected runtime, since an evidence bug can occur with matching current builds; structured refusal codes, escalation values, and specific update hints remain unchanged. The troubleshooting guide explains how --verbose identifies the selected host/socket.

Addresses the diagnostic portion of #710; capture attribution and file publication are handled by #722 and #725.

Before:

The desktop observation provider returned response evidence that did not match the requested unexpected application evidence.
⛔ Capture refused before dispatch; update the runtime before retrying

After:

The desktop observation provider returned inconsistent response evidence: unexpected application evidence.
⛔ Capture refused before dispatch; verify the selected runtime before retrying

Validation: the new observation-message regression failed against the old implementation. All 8 Bridge result tests and 39 CLI error-output tests pass; lint, format, and docs checks pass. Isolated Codex review found no actionable P0–P2 findings. A signed external native executable built against the real Bridge validator refused two synthetic responses and emitted the expected messages for both unexpected application evidence and requested application identity. No desktop capture or model call was required.

@clawsweeper

clawsweeper Bot commented Sep 13, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@cursor

cursor Bot commented Sep 13, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Copy, documentation, and test-only changes to error messages and CLI guidance; no changes to validation logic, refusal codes, or security-sensitive paths.

Overview
Improves user-facing diagnostics when observation attestation fails or a command is refused for runtime incompatibility, without changing refusal codes or escalation behavior.

Bridge observation errors now say the provider returned inconsistent response evidence and name the mismatch directly (e.g. unexpected application evidence), instead of phrasing that read like the mismatch was part of the request (did not match the requested …).

CLI human status lines for .updateRuntime escalation tell users to verify the selected runtime rather than update the runtime, so same-build verification bugs are not framed as an outdated install. A regression test locks the Capture + runtimeIncompatible wording.

Docs and changelog add an unreleased note and a see troubleshooting tip: inconsistent-response-evidence means capture could not be verified against the request; use --verbose for host/socket context and do not assume an update fixes every verification refusal.

Reviewed by Cursor Bugbot for commit 972f9b7. Bugbot is set up for automated code reviews on this repo. Configure here.

@clawsweeper clawsweeper Bot added P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 13, 2026
@clawsweeper

clawsweeper Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 12, 2026, 9:45 PM ET / September 13, 2026, 01:45 UTC (Revision 2).

ClawSweeper review

What this changes

Clarifies Bridge observation error messages and CLI runtime-refusal guidance, with two regression tests, troubleshooting documentation, and a changelog entry.

Merge readiness

Ready for maintainer review

The diagnostics fix remains useful: current main and v4.3.4 retain the misleading wording. No actionable correctness or security findings remain, and the related merged fixes address separate behavior.

Priority: P3
Reviewed head: 972f9b7ccff68e86fa6f1d24356ac1bd1b330875

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused, well-supported diagnostics repair with regression coverage and no actionable findings.
Proof confidence 🌊 off-meta tidepool Not applicable: The contributor gate is exempt for this collaborator-authored PR, and no authority change triggers additional proof; the body also reports real Bridge-validator diagnostic output from a signed native executable.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: The contributor gate is exempt for this collaborator-authored PR, and no authority change triggers additional proof; the body also reports real Bridge-validator diagnostic output from a signed native executable.
Evidence reviewed 8 items Introduced change is limited to diagnostics: The complete pinned delta changes two production strings, adds two regression tests, and updates documentation and the changelog. Validation conditions, error codes, escalation values, and publication behavior are unchanged.
Current main still needs the wording fix: Current main retains both the requested-mismatch sentence template and the unconditional update-runtime guidance. The validator can return diagnostic phrases such as unexpected application evidence, explaining the malformed sentence directly.
Release check: The latest supplied release, v4.3.4, also contains the old observation-error template; its CLI renderer retains update-runtime guidance.
Findings None None.
Security None None.

How this fits together

Peekaboo’s Bridge validates desktop observations returned by a runtime host. Rejected evidence becomes an error that the CLI presents with recovery guidance.

flowchart TD
    A[Desktop observation request] --> B[Runtime host capture]
    B --> C[Bridge evidence validation]
    C --> D[Accepted observation]
    C --> E[Refusal and evidence detail]
    E --> F[CLI diagnostic and recovery guidance]
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

None.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #710
Summary: This PR addresses the diagnostic portion of the canonical report; the related merged PRs cover separate capture and publication defects.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Technical review

Best possible solution:

Keep structured refusal semantics stable while describing inconsistent evidence accurately and directing users to inspect the selected runtime.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: current main interpolates diagnostic phrases into a target-oriented sentence and maps runtime incompatibility to unconditional update advice; no native reproduction was executed during this review.

Is this the best way to solve the issue?

Yes. Changing the two presentation strings and documenting existing verbose diagnostics is a narrow repair that preserves machine-readable behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 6916848ab2fa.

Labels

Label justifications:

  • P3: This is a focused diagnostic and troubleshooting improvement with unchanged runtime behavior.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The contributor gate is exempt for this collaborator-authored PR, and no authority change triggers additional proof; the body also reports real Bridge-validator diagnostic output from a signed native executable.

Evidence

What I checked:

Likely related people:

  • unknown: The claimed source-line change could not be verified from bounded local history. (role: source history unknown; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-09-13T01:15:42.513Z sha ce3bb8e :: needs maintainer review before merge. :: none

@steipete
steipete merged commit 2c52766 into main Sep 13, 2026
12 checks passed
@steipete
steipete deleted the fix/phase5-observation-refusal-guidance-20260912 branch September 13, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant