Skip to content

test: replace stale agent scaffolding with production contracts - #726

Merged
steipete merged 2 commits into
mainfrom
test/phase5-restore-agent-contract-coverage-20260912
Sep 13, 2026
Merged

test: replace stale agent scaffolding with production contracts#726
steipete merged 2 commits into
mainfrom
test/phase5-restore-agent-contract-coverage-20260912

Conversation

@steipete

Copy link
Copy Markdown
Collaborator

Several legacy test files contained only commented-out APIs, while resume tests asserted copied formatter logic and literal values. Replace those checks with production CLI parsing, exact session IDs and options, Unicode/long continuation preservation, real session JSON projection, and the production time formatter. Add a disk round trip that verifies conversation messages, cached-session deletion, and missing-session lookup. Remove the obsolete temporary-session comment.

Coverage is retained at its current owners: AgentSessionManagerStorageTests covers persistence; UIAutomationServiceWaitTests covers snapshot-based waits; ScreenCaptureServiceFlowTests covers capture routing and failures; MCPToolExecutionPolicyTests and AgentSessionExecutionPolicyTests enforce the current shell boundary. The removed legacy shell cases described execution that Agent policy deliberately forbids. Deleted files were wholly commented, import-only, or an empty no-assertion placeholder. No failing behavior test was removed or weakened.

Validation: 44 CLI checks and 54 owning-module checks passed. SwiftFormat, SwiftLint (zero violations), docs lint, and CLI build passed. Isolated Codex review found no actionable P0–P2 findings. Native proof used the signed built CLI: agent run with Unicode/quoted input and --dry-run --json preserved the instruction, returned success with background-only authority, and reported zero model/tool calls; agent resume --help exposed the current exact-ID interface. Runtime behavior is unchanged.

@clawsweeper

clawsweeper Bot commented Sep 13, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@cursor

cursor Bot commented Sep 13, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Test-only changes and comment cleanup; production CLI/session code paths are exercised more faithfully without altering runtime behavior.

Overview
Realigns agent CLI and session persistence tests with current production behavior instead of commented-out legacy APIs and duplicated helper logic.

AgentResumeCLITests is rewritten to assert real parsing: default AgentCommand flags, agent resume subcommand options (latest session vs exact ID, model/max-steps/foreground), --resume-session with Unicode/long continuation text, sessionJSONObject output (ISO8601 timestamps and policy fields), and shared formatTimeAgo. Several wholly obsolete or comment-only suites are removed (AgentResumeTests, AgentShellCommandTests, ScreenCaptureTests, WaitForElementTests, PlaceholderTests). AgentSessionManagerStorageTests gains a disk round-trip that checks message preservation, summary messageCount, delete clearing cache/files, and missing-session loads; the test session factory now accepts custom messages. A stale “temporary” comment is dropped from AgentSessionInfo.

Reviewed by Cursor Bugbot for commit 82a3af2. Bugbot is set up for automated code reviews on this repo. Configure here.

@clawsweeper clawsweeper Bot added P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 13, 2026
@clawsweeper

clawsweeper Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 12, 2026, 11:03 PM ET / September 13, 2026, 03:03 UTC (Revision 2).

ClawSweeper review

What this changes

Replaces obsolete agent test scaffolding with checks of production CLI parsing, session output, formatting, and disk persistence, and removes an outdated source comment.

Merge readiness

Ready for maintainer review

This remains a useful, focused cleanup with no actionable patch findings. The replacement coverage is absent from the reviewed main and latest release, and collaborator-authored work is protected from automatic closure.

Priority: P3
Reviewed head: 82a3af2eed9d0ba6eab9e9efa7252b70af15983a

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused cleanup with meaningful production-contract assertions and no identified correctness defects.
Proof confidence 🌊 off-meta tidepool Not applicable: This collaborator-authored test cleanup changes no production behavior beyond a comment; the reported CLI dry-run is supplemental and no authority boundary is modified.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This collaborator-authored test cleanup changes no production behavior beyond a comment; the reported CLI dry-run is supplemental and no authority boundary is modified.
Evidence reviewed 7 items Applicable repository policy: Read the complete root AGENTS.md. No nested AGENTS.md or maintainer-notes directory was found. Applied the module ownership and validation guidance within the read-only review constraint.
Verified scope and retained coverage: The introduced delta contains seven test files and one production-comment deletion: test lines +84/-843, production lines +0/-1. Deleted suites contain commented-out code, imports only, or an empty assertion-free placeholder. Current wait, capture, and execution-policy suites retain active coverage.
Production storage contract: The added test loads through a fresh manager, compares persisted messages, populates the cache, deletes the session, and checks both cache lookup and file removal. The production manager implements those operations directly through JSON files and its instance cache.
Findings None None.
Security None None.

How this fits together

Peekaboo’s agent CLI accepts task text and resume options, then uses the agent runtime to retrieve saved conversations. These tests exercise the parsing, display, and storage contracts connecting those layers.

flowchart LR
  A[Task and resume arguments] --> B[CLI parsing]
  B --> C[Agent session runtime]
  C --> D[Session files and cache]
  D --> C
  C --> E[Session JSON and display]
  F[Contract tests] --> B
  F --> D
  F --> E
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta Production +0/-1; tests +84/-843 The production deletion is a comment; the reduction primarily removes inactive scaffolding and copied assertions.

Technical review

Best possible solution:

Keep contract tests at their production owners and retire inactive scaffolding without changing runtime behavior.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is test maintenance, and source inspection confirms the obsolete scaffolding being replaced.

Is this the best way to solve the issue?

Yes: calling production parsing, formatting, and storage owners removes copied logic while preserving the existing runtime contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 8ecf8a9159ed.

Labels

Label justifications:

  • P3: This improves test quality without changing user-facing runtime behavior.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This collaborator-authored test cleanup changes no production behavior beyond a comment; the reported CLI dry-run is supplemental and no authority boundary is modified.

Evidence

What I checked:

  • Applicable repository policy: Read the complete root AGENTS.md. No nested AGENTS.md or maintainer-notes directory was found. Applied the module ownership and validation guidance within the read-only review constraint. (AGENTS.md:1, 82a3af2eed9d)
  • Verified scope and retained coverage: The introduced delta contains seven test files and one production-comment deletion: test lines +84/-843, production lines +0/-1. Deleted suites contain commented-out code, imports only, or an empty assertion-free placeholder. Current wait, capture, and execution-policy suites retain active coverage. (82a3af2eed9d)
  • Production storage contract: The added test loads through a fresh manager, compares persisted messages, populates the cache, deletes the session, and checks both cache lookup and file removal. The production manager implements those operations directly through JSON files and its instance cache. (Core/PeekabooCore/Tests/PeekabooAgentRuntimeTests/AgentSessionManagerStorageTests.swift:163, 82a3af2eed9d)
  • Production CLI contracts: Replacement assertions call AgentCommand.parse, AgentResumeSubcommand.parse, sessionJSONObject, and the exported production time formatter. Source inspection confirms the tested options, JSON fields, and relative-time results. (Apps/CLI/Tests/CLIAutomationTests/AgentResumeCLITests.swift:7, 82a3af2eed9d)
  • Main and release necessity check: The pinned main retains the replaced scaffolding; inspection of the supplied latest release also shows the old literal-value resume tests. Live GitHub metadata confirms this PR remains open and unmerged at the reviewed head. A broader PR search was unavailable, so no canonical replacement is asserted. (Apps/CLI/Tests/CLIAutomationTests/AgentResumeCLITests.swift:7, 59a805b253c1)
  • Area history and review continuity: Main history identifies Peter Steinberger as a repeated contributor to agent resume tests and session behavior. The two replacement test files are unchanged from the previous reviewed head, whose supplied review contained no findings. A deeper blame lookup failed on an unavailable historical object; no source-line introduction attribution is claimed. (Apps/CLI/Tests/CLIAutomationTests/AgentResumeCLITests.swift, f85e37675d69)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-09-13T01:07:29.167Z sha 5fc4d38 :: needs maintainer review before merge. :: none

@steipete
steipete merged commit d1eb5b4 into main Sep 13, 2026
14 checks passed
@steipete
steipete deleted the test/phase5-restore-agent-contract-coverage-20260912 branch September 13, 2026 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant