test(maestro-case): guard I/O binding direct authoring

[song-zhao-25]

Summary

• add a deterministic guard against mutating Maestro Case authoring commands in the I/O-binding eval
• catch the task-entry-conditions add regression observed in the July 31 run

Validation

• YAML parse passed
• CLI verb audit passed
• git diff --check passed
• Remote Codex coder-eval: pending

[github-actions]

Claude finished @song-zhao-25's task in 1m 45s —— View job

---

PR Review: test(maestro-case): guard I/O binding direct authoring

• Read review criteria and project conventions
• Get PR diff and read full changed file
• Check test task structure and conventions
• Post review findings

---

Summary

Adds a command_not_executed guard to the io_binding eval task, preventing agents from using mutating uip maestro case CLI subcommands (add, update, remove) instead of direct JSON authoring. Guards a regression observed in the July 31 run where the agent called task-entry-conditions add.

Change-by-Change Review

1. tests/tasks/uipath-maestro-case/io_binding/io_binding.yaml (lines 61–66)

Severity: OK

The new criterion is well-formed and consistent with the repo:

• Pattern reuse: The command_pattern and description are identical to the established guard in golden_rebuild/cm_golden_expense/cm_golden_expense.yaml:114-118 — good consistency.
• Criterion type: command_not_executed with tool_name: "Bash" matches the convention used across sibling tasks (artifact_safety_guard.yaml, registry_handoff_action_case_sdd_only.yaml).
• Weight: 3.0 matches the weight used for the same guard in cm_golden_expense.yaml:118 and sits between the primary validation (6.0) and supporting checks (1.0) — appropriate for a guardrail that prevents a known regression.
• Regex coverage: The pattern covers all mutating verbs (add, add-connector, update, remove) across the six known case-authoring noun groups (cases, stages, tasks, case-entry-conditions, case-exit-conditions, stage-entry-conditions, stage-exit-conditions, task-entry-conditions, task-exit-conditions). The \b anchor prevents false positives on substrings.
• Placement: Correctly placed after the two run_command criteria and before post_run — clean ordering.
• No whitespace issues: git diff --check is clean.

What's Missing

Nothing identified.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	No frontmatter changes
E2E Tests	OK	Existing e2e task strengthened with a regression guard
Skill Body	N/A	No skill changes
References & Assets	N/A	No reference changes
Repo Hygiene	OK	No secrets, no unrelated changes, scoped to single task

Issues for Manual Review

None found.

Conclusion

Clean, minimal, well-motivated change. The guard is consistent with the identical criterion already established in the cm_golden_expense sibling task and directly addresses a documented regression. No issues found — approve.

---

| Branch

[github-actions]

Claude finished @song-zhao-25's task in 1m 59s —— View job

---

Coder-eval task lint (advisory)

• Read lint rubric
• Identify changed task YAMLs
• Check PR body for evidence of passing run
• Read full task file and apply rubric
• Compose final lint comment

---

Coder-eval task lint (advisory)

1 task YAML changed; verdicts: 0 Critical, 1 High, 0 Medium, 0 Low, 0 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body states "Remote Codex coder-eval: pending" but does not claim the modified task has been run and passed. Please edit the PR description to add a line like: Ran skill-case-io-binding-matrix locally and it passed. once the pending run completes, or note the passing run ID.

Per-task lint

tests/tasks/uipath-maestro-case/io_binding/io_binding.yaml — verdict: High

The only issue is the missing passing-run claim (see above). The new command_not_executed criterion (lines 61–66) is well-formed:

• Self-report anti-pattern: OK — criteria grade CLI validate output and a Python structural checker, not agent-written summaries.
• Prompt over-specification: OK — prompt states the goal (build from the staged SDD) with constraint context; no step-by-step recipe. Ground-truth anchor carve-out applies for entity/variable names.
• Meaningful coverage: OK — three criteria cover validate exit code (weight 3.0), structural/debug correctness via Python checker (weight 6.0), and the new direct-authoring guard (weight 3.0). Good mix of "did the thing" and "output is correct."
• Could pass for the wrong reason: OK — the Python checker runs debug (1080s timeout) and validates I/O binding matrix correctness; the validate criterion checks schema validity. Neither is gameable without actually building the case.
• Near-duplicate: OK — compared against entry_points_io (trigger I/O schema, single-node), expense_runnable_e2e (different scenario, full e2e), aged_invoice_structural (different scenario, no debug), cm_golden_expense (golden rebuild with connectors), registry_handoff_sdd_only (Phase 1 only). All materially distinct operations.
• Validate-only flow tests: N/A — task is uipath-maestro-case, not uipath-maestro-flow. Description confirms harness runs debug after structural grading.
• Redundant or pinned uip CLI: OK — no env_packages.
• Run-limit fields under agent: OK — run limits correctly in run_limits: block.
• CLI verb reachability: The command_not_executed pattern is a negative guard (asserting commands were NOT run); verb paths (maestro case cases/stages/tasks/...) are valid sub-commands. Pattern matches the identical guard already shipping in cm_golden_expense.yaml:117 — good consistency.

Issues:

• [High] Evidence of passing run: PR body says "Remote Codex coder-eval: pending" — no confirmed passing run claim yet (see above).

Suggested fixes:

• Edit the PR description to add a passing-run claim once the pending Codex run completes (e.g. Ran skill-case-io-binding-matrix — passed on <run-id>).

Within-PR duplicates

• No duplicate clusters detected (only 1 task YAML changed).

Conclusion

• ⚠ 1 task has issues, max severity High. Advisory only — not blocking merge. The sole issue is the missing passing-run claim; the criterion itself is clean and consistent with the existing guard in cm_golden_expense.yaml.