Skip to content

feat(composables): add agent bridge for form controls - #32

Merged
antfu merged 3 commits into
mainfrom
feat/agent-bridge
Nov 18, 2026
Merged

antfu merged 3 commits into
mainfrom
feat/agent-bridge

Conversation

@antfubot

Copy link
Copy Markdown
Collaborator

Summary

AI agents currently drive @antfu/design UIs the slow way: they take screenshots or walk the accessibility tree, then click and type. This PR adds an opt-in agent bridge, so an agent can read control state and call control actions directly. The idea comes from UI SDK. In their benchmark, a bridge like this cut tool calls from 17 to 7 and tokens by about 4x, compared with accessibility-tree browser tools.

  • createAgent({ allow? }) creates a bridge. Install it with app.use(agent), or scope it to a subtree with provideAgent(agent).
  • agent.snapshot() returns { ref, component, label, value, disabled?, options?, actions } for each registered control.
  • agent.act({ steps: [{ ref, action, args }] }) runs the steps in order and stops at the first failure. It returns per-step results and a fresh snapshot. A step's ref can be a control ref (c1) or its exact label.
  • agent.tools() returns ui_snapshot / ui_act tool definitions, so the bridge can be registered with WebMCP or passed to any agent SDK.

Actions go through each component's own v-model. As a result, an agent can only do what a user can:

  • Select, combobox, radio group, segmented control and tabs only accept enabled options.
  • FormNumberInput clamps to min/max, and increment/decrement use the same stepBy as the stepper buttons.
  • Disabled controls refuse every action.

Each step also goes through the app's allow(step, control) guard. While a step runs, getChangeSource() returns 'agent', so an update handler can tell agent changes from user changes.

If no bridge is installed, useAgentControl returns right after inject and registers nothing.

Change map

Area Before After
composables/agent.ts (new) - createAgent, provideAgent, useAgentControl, getChangeSource, assertAgentOption, provideAgentFieldLabel
Form controls + LayoutTabs v-model only also register with the bridge: FormTextInput, FormTextarea, FormNumberInput, FormSelect, FormCombobox, FormSwitch, FormCheckbox, FormRadioGroup, FormSegmentedControl, LayoutTabs
FormField label only also supplies its label as the agent-facing name of the control it wraps
Docs - README "Agent bridge" section, skill core-components.md entry, Utilities/AgentBridge story, regenerated Overview.mdx

A control's label comes from its label prop, then aria-label, then the label of an enclosing FormField, then placeholder. FormSearchField gets registered automatically through the FormTextInput it wraps.

Visual changes

Utilities/AgentBridge is a new story, so main has no "before" version. These screenshots show the story's initial state, the state after "Run agent steps", and a step that the allow() guard rejects.

Initial After agent steps Guard rejects a step
initial after rejected

Boundaries and risks

Invariant Failure mode Protection Evidence
Agents can't set values a user couldn't pick agent sets a disabled or unknown option assertAgentOption throws, the step fails, later steps don't run test/agent.test.ts "stops at the first failing step", radio c rejection
App stays in control agent performs a destructive change allow() guard and getChangeSource() "honors the allow guard", "acts through v-model and tags the source"
Zero cost when unused overhead for apps without a bridge useAgentControl returns right after inject "is inert without a bridge" (mount without a bridge doesn't throw)
No leaked registrations stale controls in the snapshot unregister in onBeforeUnmount "unregisters on unmount"

Known gaps (not blocking):

  • Snapshot order follows mount order, not DOM order.
  • Overlays, menus and ActionToggle* aren't instrumented yet.
  • getChangeSource() is module-level and only valid synchronously inside the handler.
  • The bridge hasn't been tested against a real WebMCP browser.

User prompts

Claude Code (Opus 5.5) in Paseo.

  • Asked what we can learn from https://ui-sdk.dev/.
  • Picked "prototype the agent bridge" from the suggested ideas.
  • Asked to extend it to the remaining form controls, add a story and skill docs, then open a PR.

Verification

  • pnpm -F @antfu/design exec vitest run: 183 passed (9 new in test/agent.test.ts)
  • pnpm -F @antfu/design run typecheck (vue-tsc): no errors
  • pnpm run lint: clean
  • pnpm -F @antfu/design run docs:gen and git diff --exit-code -- packages/design/README.md skills: in sync after the docs commit
  • Ran Storybook and drove Utilities/AgentBridge in Chromium: all 5 steps applied, "Last change by" showed agent, the forbidden step was rejected, and the console had no errors.
  • Not run locally: nr storybook:build and the playwright a11y job (left to CI).

Written by Claude Code. The human chose the direction (prototype the bridge, then expand it and open a PR) and hasn't reviewed the diff yet, so the reviewer is the first human to look.

@antfu
antfu merged commit 8567d4e into main Nov 18, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants