feat(agent): plan mode, governance rendering, and agent instructions - #49
Conversation
- add submit_plan tool requiring user approval before agent executes any
multi-step operation; execute returns {approved: true} so the agent
continues after Run is clicked
- add post-skill _hint in da_read_skill result nudging the agent to call
submit_plan before proceeding, mirroring AO's post-skill hook behaviour
- update system prompt: replace :::plan directive instructions with
submit_plan call instructions; keep :::task-item for per-step progress
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…_mode Replace submit_plan with the two-tool bracket pattern used by AO's built-in planning tools so skills work consistently across both platforms. - rename submit_plan → exit_plan_mode (approval-gated, carries plan data) - add enter_plan_mode (no-op signal, no approval, mirrors AO semantics) - update system prompt: enter_plan_mode → reason → exit_plan_mode bracket - update da_read_skill _hint to reference the new tool names Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- require character-for-character identical labels between exit_plan_mode tasks and :::task-item directives to prevent status merge mismatches - clarify that `done` must be emitted immediately after each tool result, before any commentary, and must never be skipped - add Content-Length: 0 header to HEAD /chat response to signal end of response faster Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- add run_preflight tool with needsApproval: true and zod schema validating title, readiness (0-100 int), categories/checks, summary - instruct agent to auto-include preflight as final plan step when creating/updating HTML page documents (.html files only) - instruct agent not to re-enter plan mode or re-read the document when executing the preflight step - add tests for tool schema, needsApproval, execute, and prompt instructions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
anfibiacreativa
changed the title
…ight
- add url field to run_preflight input schema (Live Preview URL)
- execute calls POST {GOVERNANCE_AGENT_URL}/api/v0/evaluate/page with IMS token
- maps text_evaluation.evaluations (check_title, alignment YES/NO/NA) to card schema
- falls back to approved:true when governance is unavailable or unconfigured
- add imsToken + governanceUrl to DAToolsOptions; pass from tool-assembly
- update prompt to make preflight conditional on a configured skill (not auto)
- update tests: add url to schema fixtures, add rejects-missing-url case
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…preflight as dumb gate - remove REST call from execute — agent calls mcp__governance-agent__evaluate_page directly - remove imsToken/governanceUrl from DAToolsOptions and tool-assembly (no longer needed) - update prompt: instruct agent to call evaluate_page, map results, then call run_preflight - update tests to match new prompt wording Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…inuation pop-up. It should be code driven
| // provider namespace and is ignored by the Bedrock provider. | ||
| ...(isContinuationGated | ||
| ? { providerOptions: { daAgent: { continuationApproval: true } } } | ||
| : {}), |
There was a problem hiding this comment.
Vercel SDK provides natively the needsApproval method to gate before a tool executes.
It doesn't provide anything native for gating a tool after it finishes the execution, so this is the solution Claude came up with: found a freeform metadata, where we can mark tools that we want to gate after.
There was a problem hiding this comment.
I just want to point out that our tacked-on approvals system is the biggest source of crashes in the agent. We should be aware of this after merging and reconsider if we see that this is not behaving as expected.
| // Tools approved this request are executed in resolveApprovals and injected as tool-results, | ||
| // so streamText treats them as prior context and never re-emits their output. Surface those | ||
| // outputs to the client (for tool cards) by merging them into the streamText UI stream below. | ||
| const newlyResolvedOutputs = getNewlyResolvedToolOutputs(messages, processedMessages); |
There was a problem hiding this comment.
This was a bug I discovered during implementation:
Tool that required pre-approval did not return the tool's result to the browser.
|
Discussed with Natalia and implemented: Removed the preflight tool -> Replaced with continuation approvals. Programatic gates, for tools that want to stop the agentic loop after they finished the execution, LLM doesn't get to decide. Additional fix: tools requiring pre-approval were not returning the output to the browser. Discovered during testing for the card rendering in the frontend. The tool output was not being added to the message history. |
hannessolo
left a comment
hannessolo
left a comment
There was a problem hiding this comment.
Looks good, let's just keep an eye on the approvals.
andreituicu
changed the title
| // delivered to the client but never merged into message history. (Outputs of tools | ||
| // approved this round were already streamed above from `executedOutputs`.) | ||
| const reader = result.toUIMessageStream().getReader(); | ||
| let finishChunk: Awaited<ReturnType<typeof reader.read>>['value'] | null = null; |
There was a problem hiding this comment.
the reorder here (hold finish, emit data-continuation, then finish) is the load-bearing bit and it's the one part with no test. buildContinuationParts is well covered on its own, but if the ordering regresses the client shows Continue/Stop at the wrong time or after finish. can we add an integration test that drives a gated tool through the stream and asserts …tool-output, data-continuation, finish?
cc: @AlexRRR
|
🎉 This PR is included in version 1.29.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
3 participants
Must be merged together with adobe/da-nx#522
Summary
Why
Agents executing multi-step content operations need a user-reviewable plan before acting
Content generated by the agent should be checked for readiness (brand, SEO, accessibility, etc.) before the user publishes
AO has enter_plan_mode / exit_plan_mode built in — da-agent needs parity so both backends behave consistently in the chat UI
What Changed
src/tools/tools.ts — added enter_plan_mode (no approval), exit_plan_mode (needsApproval: true), and run_preflight (needsApproval: true, structured readiness result with title, readiness %, categories/checks, summary)
src/prompt-builder.ts — added planning bracket instructions, preflight scoping rules (HTML pages only, not images/config/skills), and no-re-plan instruction for preflight execution
src/server.ts — minor wiring changes
Test Plan
Risks / Follow-ups
Preflight categories are LLM-generated, not validated against a fixed schema
Image governance (brand compliance for uploads) is a separate follow-up ticket
Fix SITES-46674