feat: UI testing infrastructure — agent-browser default, CDP fallback

[ittaiz]

Summary

• Enable agents to verify UI changes via agent-browser (Playwright) connecting to the remote access server on a fixed port — no CEF bundling needed
• Remote access server listens on DEV3_RPC_PORT (from port pool) with localhost auth bypass for automation
• Native/CDP mode available as opt-in via .dev3_cdp sentinel file for terminal/PTY and WebKit-specific testing
• window.__dev3 automation bridge (navigate + getState) works in both modes
• Rewrote dev3-ui-control skill: agent-browser as default, agent-electrobun extracted to native-cdp.md

Test plan

• dev3 dev-server start → check dev3 dev-server status → note DEV3_PORT0
• Open http://localhost:PORT in a browser → app loads with working RPC (browser mode)
• touch .dev3_cdp && dev3 dev-server restart → build bundles CEF (native mode)
• rm .dev3_cdp && dev3 dev-server restart → returns to browser mode
• Multiple concurrent task dev servers each get unique ports, no conflicts

🤖 Generated with Claude Code

[h0x91b]

Hi, this is Claude (AI assistant working on this branch).

The idea and the refactor toward agent-browser are great — removing the CEF bundle from dev builds is a solid win. A few things need to be addressed before merging; details in the inline comments.

Must-fix

1. Security (critical): isLocalhostDevBypass checks the Host header, not the actual TCP address. The server binds to 0.0.0.0, so any LAN host can forge Host: localhost and bypass auth.
2. Bug: The useEffect for window.__dev3 depends on state — it re-runs on every dispatch, and between cleanup and setup the bridge is undefined.
3. Tests reimplement production logic — they don't catch regressions. The function should be exported and tested directly.
4. devScript in .dev3/config.json — nested if/else/fi in a single line. Extract to a separate .sh file.

Happy to re-review once these are addressed.

[h0x91b]

🔴 Security — Host header spoofing bypasses auth.

new URL(req.url).hostname returns whatever the client sent in the HTTP Host header, not the actual TCP remote address. The server binds to 0.0.0.0 (line 218), the port is known ($DEV3_PORT0, range 10000–20000). Any host on the LAN can hit http://<lan-ip>:<DEV3_RPC_PORT>/rpc with Host: localhost — and pass through without a token, gaining full RPC access (read projects/files, spawn processes).

Decision doc 033 says "Acceptable for local development" — but that only holds if the bypass is actually restricted to the loopback interface.

Fix options:

1. Check the real socket address via server.requestIP(req):

function isLocalhostDevBypass(req: Request, server: Server): boolean {
    if (!process.env.DEV3_RPC_PORT) return false;
    const ip = server.requestIP(req);
    return ip?.address === "127.0.0.1" || ip?.address === "::1";
}

Requires threading server into isSessionAuthenticated / isLocalhostDevBypass.

2. Or bind the server to 127.0.0.1 when DEV3_RPC_PORT is set — the problem disappears entirely, but LAN access from other devices is lost (for the agent use-case, localhost is sufficient).

[h0x91b]

🔴 Tests reimplement production logic — they don't catch regressions.

The comment honestly says "Reimplements isLocalhostDevBypass logic from remote-access-server.ts" and copies the function with a different signature (hostname, hasRpcPort). The tests verify their own copy, not the real code — if someone breaks isLocalhostDevBypass in the production module tomorrow, these tests stay green.

Especially important given the security issue flagged in remote-access-server.ts — the tests pass and give a false sense of safety.

Fix: export isLocalhostDevBypass from remote-access-server.ts (like serveStatic is already exported) and call the real function. Same applies to resolveSafePath and MIME_TYPES blocks above — reimplementation instead of import.

[h0x91b]

🟠 Bug: useEffect recreates window.__dev3 on every state change.

state changes on every reducer dispatch (dozens/hundreds of times per session). Each time:

1. Cleanup — delete window.__dev3
2. Setup — window.__dev3 = { ... }

Between (1) and (2) there is a micro-window where an agent's eval() will get undefined and crash. Decision 032 says "cheap (object creation)" — but the race between cleanup and setup is not mentioned.

Fix via ref:

const stateRef = useRef(state);
stateRef.current = state;

useEffect(() => {
    if (!(globalThis as any).__DEV3_AUTOMATION) return;
    (window as any).__dev3 = {
        navigate,
        getState: () => stateRef.current,
    };
    return () => { delete (window as any).__dev3; };
}, [navigate]);

Also navigate: (route: Route) => navigate(route) can be simplified to just navigate — the wrapper is unnecessary.

[h0x91b]

🟡 Extract into a separate .sh file.

Nested if/then/if/then/.../fi && .../fi in a single JSON string is very hard to read and dangerous to edit — one missing semicolon and the dev server won't start.

Suggestion: create scripts/dev-server.sh:

#!/usr/bin/env bash
set -euo pipefail

if [ -f .dev3_cdp ]; then
    DEV3_CDP_PORT="${DEV3_PORT0}" bun run dev
else
    BUILD_JSON="build/dev-macos-arm64/dev-3.0-dev.app/Contents/Resources/build.json"
    if [ -f "$BUILD_JSON" ] && grep -q '"defaultRenderer":"cef"' "$BUILD_JSON"; then
        echo '[dev3] Stale CEF build detected without .dev3_cdp — removing app bundle'
        rm -rf build/dev-macos-arm64/dev-3.0-dev.app
    fi
    DEV3_RPC_PORT="${DEV3_PORT0:-19191}" bun run dev
fi

Then in config.json simply: "devScript": "bash scripts/dev-server.sh". Decision 033 says "mode switching belongs in the devScript" — agreed, but the devScript doesn't have to be a one-liner.