diff --git a/docs/BRIDGE_CONTRACT_V1.md b/docs/BRIDGE_CONTRACT_V1.md new file mode 100644 index 0000000..c408bf8 --- /dev/null +++ b/docs/BRIDGE_CONTRACT_V1.md @@ -0,0 +1,146 @@ +# OpenSIN-Bridge Contract v1 + +> Status: stable +> Schema: `opensin.bridge.contract/v1` +> Source of truth: `extension/src/contract/v1/index.js` +> Worker counterpart: `OpenSIN-AI/A2A-SIN-Worker-heypiggy/bridge_contract.py` + +## Why + +Worker issue [#69](https://github.com/OpenSIN-AI/A2A-SIN-Worker-heypiggy/issues/69) +demands a versioned contract for browser commands, events, retries, and failure +classes. Previously the bridge surface was the union of `extension/src/tools/*` +plus an organic alias layer in `aliases.js`. That worked for ad-hoc agents but +gave downstream workers no machine-readable promise about: + +- which methods are safe to retry, +- which methods mutate browser state, +- what failure shape the worker can branch on. + +This document is the human-readable view. The machine-readable view ships in +`extension/src/contract/v1/index.js` and is exposed at runtime via +`bridge.contract`. + +## Read it + +```js +const contract = await rpc("bridge.contract") +contract.version // "opensin.bridge.contract/v1" +contract.revision // 1 +contract.methods // [{ name, idempotent, mutates, raises, retryHint, ... }] +contract.errorCodes // ["transport_error", "target_gone", ...] +contract.retryHints // { transport_error: "safe_retry", ... } +contract.internalToContract // { TIMEOUT: "timeout", ... } +``` + +The contract is also pinned in CI via +`scripts/validate-bridge-contract.mjs` (`pnpm run validate:bridge-contract`). + +## Idempotency model + +Every method declares an `idempotent` boolean. The rule is: + +- `idempotent: true` → calling it twice with the same params produces the + same observable browser state and is therefore safe to retry on transient + failure. +- `idempotent: false` → the worker MUST NOT blindly retry. Either it has + postconditions to verify the action took effect, or it must escalate to + recovery. + +Examples: + +| Method | Idempotent | Mutates | Reasoning | +| ----------------- | ---------- | ------- | -------------------------------------------------------- | +| `tabs.list` | yes | no | Read-only. | +| `tabs.activate` | yes | yes | Same tabId twice converges to the same active tab. | +| `nav.goto` | no | yes | SPAs may build different state per navigation. | +| `dom.click` | no | yes | Two submits = double order. Worker must verify. | +| `dom.snapshot` | yes | no | Read-only. | +| `session.invalidate` | yes | yes | Already-invalid session stays invalid; converges. | +| `session.restore` | no | yes | Restoring twice can clobber fresh state. Worker decides. | + +## Error taxonomy + +Failures map onto a small, stable code set (full list: +`extension/src/contract/v1/index.js`, `ERROR_CODES`). The bridge never invents +new error codes at runtime — every internal `BridgeError` is translated via +`INTERNAL_TO_CONTRACT`. + +### Transport / RPC envelope + +| Code | Meaning | +| ----------------- | ------------------------------------------------------ | +| `transport_error` | WS / native host disconnected, request never reached. | +| `rpc_invalid` | Malformed RPC envelope or invalid params. | +| `unknown_method` | Method not registered in this contract revision. | +| `rate_limited` | Bridge-internal rate limit hit. Retry with backoff. | +| `timeout` | Tool exceeded its timeout budget. | + +### Browser surface + +| Code | Meaning | +| --------------------- | -------------------------------------------------------- | +| `target_gone` | Tab/frame closed during call. Recover then retry. | +| `navigation_aborted` | Navigation interrupted (programmatic or user). | +| `navigation_timeout` | Navigation didn't complete within timeoutMs. | +| `cdp_failed` | Chrome DevTools Protocol failure (debugger detached). | +| `frame_detached` | The selected frame is gone. | + +### DOM / interaction + +| Code | Meaning | +| -------------------------- | ---------------------------------------------------- | +| `element_not_found` | Selector / ref didn't resolve. | +| `element_not_actionable` | Resolved but covered, disabled, or zero-size. | +| `postcondition_failed` | Mutating action did not produce expected change. | +| `duplicate_action` | Submit-style guard prevented a second click. | + +### Session + +| Code | Meaning | +| ----------------------- | ------------------------------------------------------ | +| `session_invalid` | Manifest invalidated (logged out, cookie purged, ...). | +| `session_stale` | TTL expired. Worker can extend or reacquire. | +| `session_locked` | Another worker holds the session lock. | +| `origin_not_permitted` | Origin not in the bridge's allowed list. | + +### Adversarial environment + +| Code | Meaning | +| --------------------- | ------------------------------------------------------ | +| `anti_bot_challenge` | Cloudflare / DataDome / PerimeterX detected. | +| `captcha_required` | reCAPTCHA / hCaptcha / Turnstile interactive challenge.| +| `rate_limit_remote` | Remote site (not bridge) is throttling. | + +## Retry hints + +Every error code carries a hint (`safe_retry`, `recover_then_retry`, `abort`). +The worker's retry loop SHOULD use this to decide between: + +- `safe_retry` → exponential backoff in place +- `recover_then_retry` → re-acquire context (session, frame, tab) first +- `abort` → escalate to higher-level policy + +Hints are advisory. The worker is the policy authority — the bridge is the +classifier. + +## Stability guarantees + +Within v1: + +- No method is removed. +- No error code is repurposed. +- Adding a method or relaxing idempotency bumps `revision`. + +A breaking change forces a new file under `extension/src/contract/v2/index.js` +and a parallel exposure (`bridge.contract` always returns the active major). + +## Related tools + +- `bridge.contract` — get the full contract object +- `bridge.contract.method` — get one method's metadata +- `bridge.contract.translate` — translate an internal code to a contract code +- `bridge.contract.idempotent` — yes/no shortcut +- `bridge.contract.version` — version + revision only +- `bridge.evidenceBundle` — pair with the contract for failure triage (#70) +- `bridge.traces` — recent dispatches, optionally by `traceId` diff --git a/docs/GAPS.md b/docs/GAPS.md new file mode 100644 index 0000000..de571b8 --- /dev/null +++ b/docs/GAPS.md @@ -0,0 +1,91 @@ +# Gaps and honest limitations + +Status as of the rewrite alignment PR. This document is the counterpart to +`A2A-SIN-Worker-heypiggy/docs/GAPS.md`. Anything listed here is **not** +solved by the contract-v1 layer and must not be assumed production-grade. + +## 1. Target resolution in the interaction path + +- `dom.click` / `dom.type` accept a selector or a target object. Selectors + are matched with a plain `querySelector` fallback. Shadow DOM, iframes + and cross-origin frames are **not** transparently resolved. +- No built-in retry for "element exists but is covered by an overlay". + Callers must compose `stealth.detectChallenge` + dismiss logic themselves. +- Accessibility snapshots returned by `dom.snapshot` do not yet include + ancestor chain or stable path hashes. Resolvers relying only on + `name` or `ariaLabel` will collide on pages with repeated text. + +**Planned fix:** extend `dom.snapshot` to return `{ role, name, path, bbox, +visible, interactable, ancestors: [...] }` per node, and add +`dom.resolve` that scores candidates. + +## 2. Stealth surface is a heuristic, not a defence + +- `stealth.assess` checks locale / timezone / viewport / `navigator.webdriver`. + Modern anti-bot stacks (Cloudflare Turnstile, DataDome, PerimeterX, + Kasada, Akamai Bot Manager) fingerprint dozens of additional signals: + canvas, WebGL, AudioContext, font metrics, TLS JA3/JA4, HTTP/2 SETTINGS. + None of those are covered here. +- `stealth.detectChallenge` is a DOM signature matcher. It **cannot** + distinguish a silently-triggered silent challenge from a clean page. +- There is no mouse or keyboard humanisation primitive in the bridge. + `dom.click` fires a single synthetic event at the target centre. + +**Planned fix:** add `input.humanMove`, `input.humanClick`, `input.humanType` +primitives in a new `tools/input.js` driven by CDP +`Input.dispatchMouseEvent` / `Input.dispatchKeyEvent`, with jitter and +motion traces. + +## 3. Session lifecycle gaps + +- `session.manifest` tracks TTL in wall-clock seconds. There is no + refresh-token aware flow; reauth is handled outside the bridge. +- `session.lastKnownGood` snapshot contains cookies only. IndexedDB + and ServiceWorker registrations are not captured. +- Cross-profile isolation depends on Chrome's cookie stores and is not + enforced inside the bridge. Two origins sharing a cookie jar leak + into each other's manifests. + +## 4. Observability gaps + +- Evidence bundles embed screenshots as base64 inline. For long + sessions this blows up disk usage. No rotation or compression. +- Trace IDs are generated inside the bridge and are not propagated + back into the page via `X-Trace-Id` or similar, so network-level + evidence cannot be joined with DOM evidence. +- `bridge.traces` stores the ring buffer in memory; extension reloads + wipe history. + +## 5. Contract completeness + +- Contract v1 covers **40** methods. Real worker call sites still use + some legacy names that are only implemented via `tools/aliases.js` + and **not** exposed through `bridge.contract`. A worker calling + `ensure_contract()` will accept v1 but can still invoke a + non-catalogued legacy method at runtime. +- `dom.evaluate` accepts arbitrary JS. It is listed as non-mutating + but cannot be enforced. Callers must treat it as effectively mutating. + +## 6. Test coverage + +- 60 Node tests cover contract metadata, session lifecycle, evidence + bundle shape and stealth assessment math. There is **zero** end-to-end + coverage against a real Chrome instance, a real website, or a + real Cloudflare-protected target. +- No fuzzer against malformed RPC envelopes, yet the bridge is the + trust boundary between worker and browser. + +## 7. What this PR does **not** fix + +- Anti-bot evasion quality. +- Worker orchestration logic (lives in the worker repo). +- Real target-resolver robustness. +- Production readiness of the opt-in new stack on the worker side. + +## Rollout contract + +Until the gaps above are closed, consumers must assume the new surface +is **best-effort additive tooling**, not a replacement for +site-specific hardening. The flat legacy names (`click`, `type`, +`nav_to`, ...) keep working and remain the supported path for +heypiggy's in-production flows. diff --git a/docs/REWRITE_TRACKER.md b/docs/REWRITE_TRACKER.md new file mode 100644 index 0000000..c6ac5eb --- /dev/null +++ b/docs/REWRITE_TRACKER.md @@ -0,0 +1,67 @@ +# Rewrite Tracker — HeyPiggy worker + OpenSIN-Bridge + +> Last updated: 2026-04-20 +> Master: [worker #67](https://github.com/OpenSIN-AI/A2A-SIN-Worker-heypiggy/issues/67) +> Base SHA: `bf1cd8c` on `main` (worker repo) + +## Why this file exists + +Worker repo `OpenSIN-AI/A2A-SIN-Worker-heypiggy` issue #67 froze the previous +implementation as broken and split the rewrite into nine sub-issues. Several of +them have a **bridge-side counterpart** that has to land in this repo so the +worker can be rewritten against a stable contract instead of the old organic +surface. + +This file is the index. Each row links the worker issue to the concrete +bridge artifacts that satisfy its bridge-side requirements. + +## Coverage matrix + +| Worker issue | Theme | Bridge artifacts | Status | +| ------------ | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | +| #68 | Architecture RFC | (worker-only) — the bridge contract v1 (`docs/BRIDGE_CONTRACT_V1.md`) is the bridge-side input to the RFC. | done | +| #69 | Bridge contract | `extension/src/contract/v1/index.js`, `extension/src/tools/contract.js`, `docs/BRIDGE_CONTRACT_V1.md`, `scripts/validate-bridge-contract.mjs`, `bridge.contract` RPC | done | +| #70 | Observability / evidence | `extension/src/drivers/evidence.js`, `extension/src/drivers/trace.js`, `extension/src/tools/evidence.js`, `bridge.evidenceBundle` and `bridge.traces` RPCs | done | +| #71 | Session lifecycle | `extension/src/drivers/session-lifecycle.js`, `session.manifest`, `session.invalidate`, `session.lastKnownGood`, `session.health` RPCs | done | +| #72 | Runtime state machine | (worker-only) — bridge exposes the primitives the FSM consumes (snapshot, dom.click, session.health, evidenceBundle, contract). | n/a | +| #73 | Interaction engine | (worker-only) — bridge already exposes actionability via `dom.fullSnapshot` + `dom.click` postcondition fields. Contract documents the failure codes. | n/a | +| #74 | Stealth strategy | `extension/src/tools/stealth.js`, `stealth.assess`, `stealth.detectChallenge`. The existing `stealth-main.js` is unchanged; this adds the assessment surface. | done | +| #75 | Panel plugins | (worker-only) — bridge stays panel-agnostic by design (`BOUNDARIES.md`). | n/a | +| #76 | Validation harness | `scripts/validate-bridge-contract.mjs`, `test/contract/*.test.mjs`, `test/session/*.test.mjs`, `test/evidence/*.test.mjs`, package script `validate:bridge-contract` | done | + +## How to consume from the worker + +```python +# heypiggy/bridge_contract.py +import json +import urllib.request + +def fetch_contract(bridge_http_url: str) -> dict: + body = {"jsonrpc": "2.0", "id": 1, "method": "tools/call", + "params": {"name": "bridge.contract", "arguments": {}}} + req = urllib.request.Request( + f"{bridge_http_url}/mcp", + data=json.dumps(body).encode(), + headers={"Content-Type": "application/json"}, + ) + with urllib.request.urlopen(req, timeout=10) as resp: + envelope = json.loads(resp.read()) + text = envelope["result"]["content"][0]["text"] + return json.loads(text) +``` + +The worker pins the contract revision at boot, validates that every +method/error code it expects is present, and aborts startup if anything is +missing. That is the gate worker issue #69 demands. + +## Stop conditions + +Per worker issue #76, the rewrite can not enable wider rollout until: + +- `pnpm run validate:bridge-contract` passes in CI on this repo +- the worker-side replay harness clears the synthetic flow set +- a live canary run produces a complete `bridge.evidenceBundle` for every + failure (no unclassified errors) + +Until those gates pass the worker stays in the locked-down rewrite branch +established in worker issue #67. diff --git a/extension/src/content/bridge-isolated.js b/extension/src/content/bridge-isolated.js index 30b2b51..3e03153 100644 --- a/extension/src/content/bridge-isolated.js +++ b/extension/src/content/bridge-isolated.js @@ -257,40 +257,125 @@ } // ---- Input synthesis ------------------------------------------------------ + // Track the last synthetic pointer position so consecutive actions share a + // plausible mouse trajectory instead of teleporting across the viewport. + let __lastPointer = { x: Math.floor(window.innerWidth / 2), y: Math.floor(window.innerHeight / 2) } + + function gaussian(mean, stddev) { + // Box-Muller. Clamped so we never return negative delays. + const u1 = 1 - Math.random() + const u2 = 1 - Math.random() + const z = Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2) + return Math.max(0, mean + stddev * z) + } + + // Quadratic Bezier with a single random control point offset perpendicular + // to the start->end vector. Produces a curved, non-linear trajectory that + // does not look like a straight-line snap. + function* bezierPath(x0, y0, x1, y1, steps) { + const dx = x1 - x0 + const dy = y1 - y0 + const dist = Math.hypot(dx, dy) || 1 + const nx = -dy / dist + const ny = dx / dist + const jitter = Math.min(dist * 0.25, 140) * (Math.random() - 0.5) * 2 + const cx = (x0 + x1) / 2 + nx * jitter + const cy = (y0 + y1) / 2 + ny * jitter + for (let i = 1; i <= steps; i++) { + const t = i / steps + const it = 1 - t + yield { + x: it * it * x0 + 2 * it * t * cx + t * t * x1, + y: it * it * y0 + 2 * it * t * cy + t * t * y1, + } + } + } + + async function moveMouse(fromX, fromY, toX, toY, { human = true } = {}) { + const dist = Math.hypot(toX - fromX, toY - fromY) + const steps = Math.max(6, Math.min(32, Math.round(dist / 28))) + if (!human) { + window.dispatchEvent(new MouseEvent("mousemove", { clientX: toX, clientY: toY, bubbles: true })) + __lastPointer = { x: toX, y: toY } + return + } + for (const pt of bezierPath(fromX, fromY, toX, toY, steps)) { + const ev = { + clientX: pt.x, + clientY: pt.y, + bubbles: true, + cancelable: true, + composed: true, + view: window, + } + const target = document.elementFromPoint(pt.x, pt.y) || document.documentElement + target.dispatchEvent(new PointerEvent("pointermove", { ...ev, pointerType: "mouse" })) + target.dispatchEvent(new MouseEvent("mousemove", ev)) + await sleep(gaussian(12, 5)) + } + __lastPointer = { x: toX, y: toY } + } + async function humanClick(el, opts = {}) { el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" }) - await sleep(30) + await sleep(gaussian(80, 30)) const rect = el.getBoundingClientRect() - const x = rect.left + rect.width / 2 - const y = rect.top + rect.height / 2 + // Pick a point slightly off-centre inside the target -- real users do not + // hit geometric centres. + const pad = 6 + const minX = rect.left + Math.min(pad, rect.width / 4) + const maxX = rect.right - Math.min(pad, rect.width / 4) + const minY = rect.top + Math.min(pad, rect.height / 4) + const maxY = rect.bottom - Math.min(pad, rect.height / 4) + const x = minX + Math.random() * Math.max(1, maxX - minX) + const y = minY + Math.random() * Math.max(1, maxY - minY) + + if (opts.human !== false) { + await moveMouse(__lastPointer.x, __lastPointer.y, x, y, { human: true }) + } else { + __lastPointer = { x, y } + } + const init = { bubbles: true, cancelable: true, composed: true, clientX: x, clientY: y, + screenX: x, + screenY: y, button: opts.button === "right" ? 2 : opts.button === "middle" ? 1 : 0, buttons: 1, view: window, } - el.dispatchEvent(new PointerEvent("pointerover", init)) + el.dispatchEvent(new PointerEvent("pointerover", { ...init, pointerType: "mouse" })) el.dispatchEvent(new MouseEvent("mouseover", init)) - el.dispatchEvent(new PointerEvent("pointerdown", { ...init, pointerType: "mouse" })) + await sleep(gaussian(40, 15)) + el.dispatchEvent(new PointerEvent("pointerdown", { ...init, pointerType: "mouse", isPrimary: true })) el.dispatchEvent(new MouseEvent("mousedown", init)) - if (opts.human !== false) await sleep(20 + Math.random() * 40) + if (opts.human !== false) await sleep(gaussian(70, 25)) el.dispatchEvent(new PointerEvent("pointerup", { ...init, pointerType: "mouse" })) el.dispatchEvent(new MouseEvent("mouseup", init)) for (let i = 0; i < (opts.clickCount || 1); i++) { el.dispatchEvent(new MouseEvent("click", init)) - if (i < (opts.clickCount || 1) - 1) await sleep(60) + if (i < (opts.clickCount || 1) - 1) await sleep(gaussian(120, 30)) } if (typeof el.focus === "function") el.focus() } - async function humanType(el, text, { delayMs, replace = false, pressEnter = false } = {}) { + async function humanType( + el, + text, + { delayMs, replace = false, pressEnter = false, typoRate = 0, human = true } = {}, + ) { if (!el) throw err("type target not found", "NOT_FOUND") if (!(el instanceof HTMLElement)) throw err("type target not editable", "NOT_EDITABLE") - el.focus() + if (typeof el.focus === "function") el.focus() + if (document.activeElement !== el) { + // Recover if the site stole focus; otherwise keypresses go to body. + el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" }) + el.focus() + } if (replace) { if ("value" in el) { setNativeValue(el, "") @@ -299,22 +384,68 @@ el.textContent = "" } } - for (const ch of text) { - const down = new KeyboardEvent("keydown", { key: ch, bubbles: true }) - el.dispatchEvent(down) + + // Inter-keystroke timing model: base mean 95ms stddev 40ms, short burst + // after whitespace, occasional long pauses ("thinking"). + const baseMean = typeof delayMs === "number" ? delayMs : 95 + const stddev = Math.max(10, baseMean * 0.35) + + function pushKey(ch) { + const code = ch.length === 1 ? `Key${ch.toUpperCase()}` : ch + el.dispatchEvent(new KeyboardEvent("keydown", { key: ch, code, bubbles: true })) if ("value" in el) { setNativeValue(el, (el.value || "") + ch) el.dispatchEvent(new InputEvent("input", { inputType: "insertText", data: ch, bubbles: true })) } else if (el.isContentEditable) { document.execCommand?.("insertText", false, ch) } - el.dispatchEvent(new KeyboardEvent("keyup", { key: ch, bubbles: true })) - if (delayMs) await sleep(delayMs + Math.random() * 25) + el.dispatchEvent(new KeyboardEvent("keyup", { key: ch, code, bubbles: true })) + } + + function popChar() { + if ("value" in el) { + const v = el.value || "" + setNativeValue(el, v.slice(0, -1)) + el.dispatchEvent(new InputEvent("input", { inputType: "deleteContentBackward", bubbles: true })) + } else if (el.isContentEditable) { + document.execCommand?.("delete") + } + } + + const neighbours = { + a: "sq", + e: "wr", + i: "uo", + o: "ip", + s: "ad", + t: "ry", + n: "bm", + l: "k", + r: "et", + } + + for (const ch of text) { + // Simulate an occasional typo + correction when humanisation is on. + if (human && typoRate > 0 && /[a-z]/i.test(ch) && Math.random() < typoRate) { + const nb = neighbours[ch.toLowerCase()] + if (nb) { + const wrong = nb[Math.floor(Math.random() * nb.length)] + pushKey(ch === ch.toUpperCase() ? wrong.toUpperCase() : wrong) + await sleep(gaussian(baseMean * 1.4, stddev)) + popChar() + await sleep(gaussian(baseMean * 0.8, stddev)) + } + } + pushKey(ch) + let wait = human ? gaussian(baseMean, stddev) : baseMean + if (ch === " ") wait *= 0.7 + if (human && Math.random() < 0.04) wait += gaussian(450, 180) // thinking pause + await sleep(wait) } el.dispatchEvent(new Event("change", { bubbles: true })) if (pressEnter) { - el.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true })) - el.dispatchEvent(new KeyboardEvent("keyup", { key: "Enter", bubbles: true })) + el.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", code: "Enter", bubbles: true })) + el.dispatchEvent(new KeyboardEvent("keyup", { key: "Enter", code: "Enter", bubbles: true })) if (el.form && typeof el.form.requestSubmit === "function") el.form.requestSubmit() } } @@ -359,6 +490,124 @@ async "dom.fullSnapshot"(args) { return buildSnapshot({ ...args, mode: "full" }) }, + async "dom.resolve"({ + role, + name, + nameMatch = "contains", + ancestor, + testId, + attributes, + visibleOnly = true, + limit = 5, + } = {}) { + // Candidate gathering: start from a cheap CSS filter when possible, + // then score each candidate by how many hints it matches. + const candidates = new Set() + if (testId) { + for (const attr of ["data-testid", "data-test", "data-qa"]) { + for (const el of document.querySelectorAll(`[${attr}="${cssEscapeAttr(testId)}"]`)) { + candidates.add(el) + } + } + } + if (role) { + const tags = roleToTags(role) + const byRole = document.querySelectorAll(`[role="${cssEscapeAttr(role)}"]`) + byRole.forEach((el) => candidates.add(el)) + tags.forEach((t) => document.querySelectorAll(t).forEach((el) => candidates.add(el))) + } + if (!candidates.size) { + document.querySelectorAll("a,button,input,textarea,select,[role],[tabindex]").forEach((el) => candidates.add(el)) + } + + const needle = (name || "").toLowerCase().trim() + function matchName(el) { + if (!needle) return 0.5 + const n = accessibleName(el).toLowerCase() + if (!n) return 0 + if (nameMatch === "exact") return n === needle ? 1 : 0 + if (nameMatch === "startsWith") return n.startsWith(needle) ? 0.9 : 0 + if (nameMatch === "regex") { + try { + return new RegExp(name, "i").test(n) ? 0.85 : 0 + } catch { + return 0 + } + } + return n.includes(needle) ? 0.7 + (n === needle ? 0.3 : 0) : 0 + } + + function matchAncestor(el) { + if (!ancestor) return 0.5 + let node = el.parentElement + let hops = 0 + while (node && hops < 12) { + const r = node.getAttribute?.("role") || implicitRole(node) + const n = accessibleName(node).toLowerCase() + if (ancestor.role && r === ancestor.role) return 0.9 + if (ancestor.name && n.includes(String(ancestor.name).toLowerCase())) return 0.85 + if (ancestor.selector && node.matches?.(ancestor.selector)) return 0.95 + node = node.parentElement + hops += 1 + } + return 0 + } + + function matchAttributes(el) { + if (!attributes) return 0.5 + let score = 0 + let total = 0 + for (const [k, v] of Object.entries(attributes)) { + total += 1 + if (el.getAttribute(k) === v) score += 1 + } + return total ? score / total : 0.5 + } + + const scored = [] + for (const el of candidates) { + if (visibleOnly && !isVisible(el)) continue + const effectiveRole = el.getAttribute("role") || implicitRole(el) + const roleScore = !role ? 0.5 : effectiveRole === role ? 1 : 0 + if (role && roleScore === 0) continue + const nameScore = matchName(el) + if (name && nameScore === 0) continue + const score = + roleScore * 0.35 + + nameScore * 0.35 + + matchAncestor(el) * 0.15 + + matchAttributes(el) * 0.1 + + (isVisible(el) ? 0.05 : 0) + scored.push({ el, score }) + } + + scored.sort((a, b) => b.score - a.score) + const top = scored.slice(0, limit).map(({ el, score }) => { + const rect = el.getBoundingClientRect() + return { + ref: mintRef(el), + score: Number(score.toFixed(3)), + role: el.getAttribute("role") || implicitRole(el), + name: accessibleName(el).slice(0, 160), + tag: el.tagName.toLowerCase(), + selector: stableSelector(el), + rect: { + x: Math.round(rect.x), + y: Math.round(rect.y), + w: Math.round(rect.width), + h: Math.round(rect.height), + }, + visible: isVisible(el), + disabled: !!el.disabled, + inViewport: + rect.bottom > 0 && rect.right > 0 && rect.top < innerHeight && rect.left < innerWidth, + } + }) + return { + matches: top, + ambiguous: top.length > 1 && top[0].score - (top[1]?.score ?? 0) < 0.08, + } + }, async "dom.waitForSelector"({ selector, state = "visible", timeoutMs = 10_000 }) { const start = Date.now() while (Date.now() - start < timeoutMs) { diff --git a/extension/src/contract/v1/index.js b/extension/src/contract/v1/index.js new file mode 100644 index 0000000..b5d51af --- /dev/null +++ b/extension/src/contract/v1/index.js @@ -0,0 +1,345 @@ +/** + * OpenSIN-Bridge Contract v1 + * ========================================================================== + * + * Versioned, machine-readable contract between the Bridge (Chrome extension + * + server) and any worker that drives it. Implements the deliverables of + * issue #69 (worker-repo) on the Bridge side: + * + * - command/event schema (namespaces, params, returns, idempotency) + * - idempotent vs non-idempotent action model + * - error taxonomy (`transport_error`, `target_gone`, `navigation_aborted`, + * `anti_bot_challenge`, ...) + * - retry-policy hints separated from business-recovery policy + * + * Design choices: + * + * - The contract is data, not code. It is consumed by: + * 1. `extension/src/tools/contract.js` (exposed as `bridge.contract`) + * 2. `scripts/validate-bridge-contract.mjs` (CI guard) + * 3. The worker-side counterpart in heypiggy `bridge_contract.py` + * - The contract DOES NOT replace the existing tool registry. It declares + * the public surface area the bridge guarantees and how callers should + * classify failures. + * - Adding a new contract version means: bump VERSION, copy this file to + * `extension/src/contract/v2/index.js`, and keep v1 frozen forever. + * + * Stability rules: + * - Removing a method or namespace is a MAJOR change → new version file. + * - Adding a method or making a non-idempotent method idempotent is a + * MINOR change → bump `revision`. + * - Editing wording / docs only is a PATCH change → no revision bump. + */ + +export const VERSION = "opensin.bridge.contract/v1" +export const REVISION = 1 + +/** + * Stable error taxonomy. The string codes are part of the public contract + * and must never be repurposed. Workers branch on these codes to decide + * between retry, recovery, or abort. + */ +export const ERROR_CODES = Object.freeze({ + // ----- transport / RPC envelope ------------------------------------------ + TRANSPORT_ERROR: "transport_error", + RPC_INVALID: "rpc_invalid", + UNKNOWN_METHOD: "unknown_method", + RATE_LIMITED: "rate_limited", + TIMEOUT: "timeout", + + // ----- browser surface ---------------------------------------------------- + TARGET_GONE: "target_gone", // tab closed / detached during call + NAVIGATION_ABORTED: "navigation_aborted", + NAVIGATION_TIMEOUT: "navigation_timeout", + CDP_FAILED: "cdp_failed", + FRAME_DETACHED: "frame_detached", + + // ----- DOM / interaction ------------------------------------------------- + ELEMENT_NOT_FOUND: "element_not_found", + ELEMENT_NOT_ACTIONABLE: "element_not_actionable", + POSTCONDITION_FAILED: "postcondition_failed", + DUPLICATE_ACTION: "duplicate_action", + + // ----- authentication / session ------------------------------------------ + SESSION_INVALID: "session_invalid", + SESSION_STALE: "session_stale", + SESSION_LOCKED: "session_locked", + ORIGIN_NOT_PERMITTED: "origin_not_permitted", + + // ----- adversarial environment ------------------------------------------- + ANTI_BOT_CHALLENGE: "anti_bot_challenge", + CAPTCHA_REQUIRED: "captcha_required", + RATE_LIMIT_REMOTE: "rate_limit_remote", + + // ----- bridge internals -------------------------------------------------- + PRECONDITION_FAILED: "precondition_failed", + UNSUPPORTED: "unsupported", + INTERNAL_ERROR: "internal_error", +}) + +/** + * Retry hints expressed at the contract layer. These are not policies — the + * worker still owns the final decision — but they tell the worker what kind + * of failure it just saw. + * + * safe_retry → idempotent, retry without side effects + * recover_then_retry → re-acquire context (session, frame, tab) first + * abort → never retry, escalate to a higher policy + */ +export const RETRY_HINTS = Object.freeze({ + [ERROR_CODES.TRANSPORT_ERROR]: "safe_retry", + [ERROR_CODES.RPC_INVALID]: "abort", + [ERROR_CODES.UNKNOWN_METHOD]: "abort", + [ERROR_CODES.RATE_LIMITED]: "safe_retry", + [ERROR_CODES.TIMEOUT]: "safe_retry", + + [ERROR_CODES.TARGET_GONE]: "recover_then_retry", + [ERROR_CODES.NAVIGATION_ABORTED]: "recover_then_retry", + [ERROR_CODES.NAVIGATION_TIMEOUT]: "safe_retry", + [ERROR_CODES.CDP_FAILED]: "recover_then_retry", + [ERROR_CODES.FRAME_DETACHED]: "recover_then_retry", + + [ERROR_CODES.ELEMENT_NOT_FOUND]: "abort", + [ERROR_CODES.ELEMENT_NOT_ACTIONABLE]: "safe_retry", + [ERROR_CODES.POSTCONDITION_FAILED]: "abort", + [ERROR_CODES.DUPLICATE_ACTION]: "abort", + + [ERROR_CODES.SESSION_INVALID]: "recover_then_retry", + [ERROR_CODES.SESSION_STALE]: "recover_then_retry", + [ERROR_CODES.SESSION_LOCKED]: "abort", + [ERROR_CODES.ORIGIN_NOT_PERMITTED]: "abort", + + [ERROR_CODES.ANTI_BOT_CHALLENGE]: "recover_then_retry", + [ERROR_CODES.CAPTCHA_REQUIRED]: "abort", + [ERROR_CODES.RATE_LIMIT_REMOTE]: "safe_retry", + + [ERROR_CODES.PRECONDITION_FAILED]: "abort", + [ERROR_CODES.UNSUPPORTED]: "abort", + [ERROR_CODES.INTERNAL_ERROR]: "abort", +}) + +/** + * Translation table from internal extension BridgeError codes (see + * `extension/src/core/errors.js`) to the public contract codes. The bridge + * error taxonomy was grown organically; this table is the single mapping + * point so downstream callers always see a stable contract code. + */ +export const INTERNAL_TO_CONTRACT = Object.freeze({ + TRANSPORT_ERROR: ERROR_CODES.TRANSPORT_ERROR, + TIMEOUT: ERROR_CODES.TIMEOUT, + RATE_LIMITED: ERROR_CODES.RATE_LIMITED, + UNKNOWN_TOOL: ERROR_CODES.UNKNOWN_METHOD, + METHOD_NOT_FOUND: ERROR_CODES.UNKNOWN_METHOD, + INVALID_INPUT: ERROR_CODES.RPC_INVALID, + INVALID_ARGS: ERROR_CODES.RPC_INVALID, + PRECONDITION_FAILED: ERROR_CODES.PRECONDITION_FAILED, + TAB_GONE: ERROR_CODES.TARGET_GONE, + CDP_FAILED: ERROR_CODES.CDP_FAILED, + NAVIGATION_FAILED: ERROR_CODES.NAVIGATION_ABORTED, + NETWORK: ERROR_CODES.TRANSPORT_ERROR, + NOT_FOUND: ERROR_CODES.ELEMENT_NOT_FOUND, + UNAUTHORIZED: ERROR_CODES.SESSION_INVALID, + FORBIDDEN: ERROR_CODES.ORIGIN_NOT_PERMITTED, + UNSUPPORTED: ERROR_CODES.UNSUPPORTED, + NATIVE_HOST_UNAVAILABLE: ERROR_CODES.TRANSPORT_ERROR, + OFFSCREEN_ERROR: ERROR_CODES.INTERNAL_ERROR, + OFFSCREEN_UNAVAILABLE: ERROR_CODES.UNSUPPORTED, + VISION_UNAVAILABLE: ERROR_CODES.UNSUPPORTED, + INTERNAL_ERROR: ERROR_CODES.INTERNAL_ERROR, +}) + +/** + * Public namespaces the bridge advertises. Each entry declares whether its + * methods are read-only (always idempotent) or include mutating operations + * (need the per-method `idempotent` flag). + */ +export const NAMESPACES = Object.freeze([ + { name: "bridge", category: "meta", description: "Contract, capabilities, evidence." }, + { name: "system", category: "meta", description: "Health, version, capabilities." }, + { name: "tabs", category: "browser", description: "Tab lifecycle." }, + { name: "nav", category: "browser", description: "Navigation primitives." }, + { name: "dom", category: "browser", description: "DOM read + interaction." }, + { name: "cookies", category: "state", description: "Cookie jar." }, + { name: "storage", category: "state", description: "Web storage." }, + { name: "net", category: "browser", description: "Network observation + control." }, + { name: "session", category: "state", description: "Session manifest, restore, invalidate." }, + { name: "behavior", category: "evidence", description: "Behavior timeline." }, + { name: "stealth", category: "browser", description: "Stealth posture + assessment." }, +]) + +/** + * Per-method declarations. Only methods that are part of the v1 contract + * appear here; legacy aliases stay outside the contract intentionally. + * + * Schema (per entry): + * name string — fully qualified tool name (`namespace.method`) + * idempotent boolean — true when retrying with same params is safe + * mutates boolean — true when the method changes browser state + * description string — short human description + * params object — JSON-schema-ish hint (optional fields shown as `?`) + * returns object — return shape hint + * raises array — contract error codes the method may raise + */ +export const METHODS = Object.freeze([ + // ----- bridge meta -------------------------------------------------------- + m("bridge.contract", true, false, "Return the active bridge contract.", {}, { contract: "object" }, []), + m("bridge.evidenceBundle", true, false, + "Assemble a forensic evidence bundle (screenshot, DOM snapshot, console, network, command history).", + { tabId: "?number", traceId: "?string", includeScreenshot: "?boolean", maxNetworkEvents: "?number" }, + { bundle: "object" }, + [ERROR_CODES.TARGET_GONE, ERROR_CODES.CDP_FAILED, ERROR_CODES.INTERNAL_ERROR], + ), + + // ----- system ------------------------------------------------------------- + m("system.health", true, false, "Bridge health snapshot.", {}, { ok: "boolean" }, []), + m("system.version", true, false, "Bridge version + protocol info.", {}, { version: "string" }, []), + m("system.capabilities", true, false, "Discovery: list of registered tools.", {}, { tools: "array" }, []), + m("system.ping", true, false, "Liveness probe.", {}, { pong: "boolean" }, []), + + // ----- tabs --------------------------------------------------------------- + m("tabs.list", true, false, "List open tabs.", {}, { tabs: "array" }, []), + m("tabs.get", true, false, "Get a single tab by id.", { tabId: "number" }, { tab: "object" }, [ERROR_CODES.TARGET_GONE]), + m("tabs.create", false, true, "Create a new tab.", { url: "string", active: "?boolean" }, { tab: "object" }, [ERROR_CODES.RPC_INVALID]), + m("tabs.close", false, true, "Close one or more tabs.", { tabIds: "array" }, { ok: "boolean" }, []), + m("tabs.activate", true, true, "Activate a tab. Idempotent: same tabId twice is harmless.", { tabId: "number" }, { ok: "boolean" }, [ERROR_CODES.TARGET_GONE]), + m("tabs.reload", false, true, "Reload tab.", { tabId: "?number" }, { ok: "boolean" }, [ERROR_CODES.TARGET_GONE]), + + // ----- navigation --------------------------------------------------------- + m("nav.goto", false, true, "Navigate the active tab. Not idempotent: SPAs may build different state per navigation.", { url: "string", tabId: "?number", waitUntil: "?string", timeoutMs: "?number" }, { url: "string" }, [ERROR_CODES.NAVIGATION_ABORTED, ERROR_CODES.NAVIGATION_TIMEOUT, ERROR_CODES.RPC_INVALID]), + m("nav.back", false, true, "Go back in history.", { tabId: "?number" }, { ok: "boolean" }, [ERROR_CODES.TARGET_GONE]), + m("nav.forward", false, true, "Go forward in history.", { tabId: "?number" }, { ok: "boolean" }, [ERROR_CODES.TARGET_GONE]), + m("nav.reload", false, true, "Reload tab.", { tabId: "?number" }, { ok: "boolean" }, [ERROR_CODES.TARGET_GONE]), + m("nav.waitForSelector", true, false, "Wait until a selector resolves.", { selector: "string", state: "?string", timeoutMs: "?number" }, { ok: "boolean" }, [ERROR_CODES.TIMEOUT, ERROR_CODES.ELEMENT_NOT_FOUND]), + + // ----- dom (read-only is idempotent, mutating is NOT) --------------------- + m("dom.snapshot", true, false, "Accessibility snapshot of the page.", { mode: "?string", maxNodes: "?number" }, { snapshot: "object" }, [ERROR_CODES.TARGET_GONE]), + m("dom.resolve", true, false, + "Resolve one or more candidate elements by role + accessible name + ancestor hints. Returns ranked matches with stable refs and ambiguity flag.", + { role: "?string", name: "?string", nameMatch: "?string", ancestor: "?object", testId: "?string", attributes: "?object", visibleOnly: "?boolean", limit: "?number" }, + { matches: "array", ambiguous: "boolean" }, + [ERROR_CODES.TARGET_GONE, ERROR_CODES.ELEMENT_NOT_FOUND], + ), + m("dom.waitForSelector", true, false, + "Wait until a selector matches in the given state (attached|visible|hidden|detached). Idempotent polling.", + { selector: "string", state: "?string", timeoutMs: "?number" }, + { ok: "boolean" }, + [ERROR_CODES.TIMEOUT, ERROR_CODES.TARGET_GONE], + ), + m("dom.fullSnapshot", true, false, "Full observation snapshot with screenshot.", { maxNodes: "?number" }, { snapshot: "object" }, [ERROR_CODES.TARGET_GONE, ERROR_CODES.CDP_FAILED]), + m("dom.click", false, true, "Click a referenced or selected element.", { selector: "?string", ref: "?string" }, { ok: "boolean", proofId: "?string" }, [ERROR_CODES.ELEMENT_NOT_FOUND, ERROR_CODES.ELEMENT_NOT_ACTIONABLE, ERROR_CODES.POSTCONDITION_FAILED, ERROR_CODES.DUPLICATE_ACTION]), + m("dom.type", false, true, "Type text into an element.", { selector: "?string", ref: "?string", text: "string" }, { ok: "boolean" }, [ERROR_CODES.ELEMENT_NOT_FOUND, ERROR_CODES.ELEMENT_NOT_ACTIONABLE]), + m("dom.fill", false, true, "Fill many fields at once.", { fields: "object" }, { ok: "boolean" }, [ERROR_CODES.ELEMENT_NOT_FOUND]), + m("dom.select", false, true, "Select option in a