You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(computers): finish needs-you sign-in handoff and outcome delivery - #463
Bots can now ask for sign-in, verification or CAPTCHA from the watched working browser without keeping a tool call open. The conversation card commits before the computer yields; Watch screen takes exclusive owner control, and hand-back supplies a durable outcome to the next eligible work run. This completes ADR 0092 slice 3c on top of merged slices 3a (#460) and 3b (#461).
Adds closed computer_browserneeds_you and origin-only inline English/Thai cards. No credential fields, CDP batch or agent ticket are used by the handoff. Typed chat text cannot decide it. Routine calls use the existing occurrence needs_you state without yielding.
Extends s16-computer-screen/v1 with yield, exact-generation takeover, existing-operation settlement and event-derived resumed/declined/expired outcomes. Card IDs survive owner leases, lapse and controller restart; return to agent clears the row and records the old ID in the same VM transaction. A durable browser-observation high-water mark prevents pre-handoff counters from equalling a new control generation.
Adds authenticated/CSRF-protected s16-computer-needs-you/v1 commands and guarded needsYouCards snapshots. Current origin tier, owner and VM generation are rechecked across waits. Sensitive sites remain in the protected browser. Open owner screens survive snapshot withholding; session/private-content failure and navigation clear them. Closing restores keyboard focus.
Uses an additive bounded conversation table and a preserving VM CHECK migration. Failed insert cannot yield; failed yield voids its card; startup/read repairs a committed crash orphan only after the 60 s deadline when no control/outcome holds its ID.
Delivers validated/hashed screenOutcomes as separate screen-outcome-data-never-authority host data in production MODEL and fallback Schway. Preparation waits for actual VM placement: a different shared pool member cannot receive or consume the signed-in computer's outcome. The carried IDs are saved before sending and marked delivered only in committed conversation result publication. Failed/held/cancelled work, missing evidence, rollback, no-computer work and later-arriving outcomes cannot consume them.
Ships needsYouMinutes / JIT_COMPUTER_NEEDS_YOU_MINUTES (240 default, 30–1440), independent S15 startup validation, packaged validation/environment/plan support and fixed per-card expiry. Only an untouched yield expires; owner control never does because of card expiry.
S14 remains the default and S15 stays opt-in. Installed policy and retained private scope keep their meaning. There is no new execution loop/provider protocol, upstream source import, dependency or live deployment. Automatic continuation after hand-back remains #459; this slice starts the next run through ordinary ingress.
Validation
Current tested/reviewed head: 9dc946a83d242de859dc2044b699da754b1316f6.
bun run typecheck, bun run upstream:check, git diff --check: pass.
bun test --timeout 30000 over screen/browser/actions/needs-you/composition, MODEL, CHAT, config, model-choice/Bots, artifact lineage and VM-policy suites: 488 passed across 12 files at the final head, including the two-shared-member placement regression.
JIT_S16_BROWSER_TESTS=1 bun test --timeout 30000 over needs-you, computer-screen, model-choice and Bots browser suites: 8 passed across 4 files at 4c481780, English 1440 and Thai 390, keyboard control/focus, refresh continuity and session-loss withholding. After the placement fix, the native needs-you English/Thai flow was rerun at 9dc946a8: 2 passed.
Native non-IdP Chromium fixture: owner input signs in, reload preserves the browser session, fresh-generation production browser observation reads the signed-in page without typed input. Other deterministic cases cover actual SIGKILL after committed insertion, clock expiry, restart/settlement, exact MODEL prompts with only continue, provider failure, completed MODEL with result_unavailable, publication rollback and late outcomes.
All 15 required/current-head CI checks passed (full deterministic shards on x64 and arm64, typecheck and Greptile). Greptile approved exact 9dc946a8; all inline threads are resolved. The local full run started at 4c481780 and completed after the placement fix: 5,056 passed, 81 skipped, 0 failed. Exact-final-head CI and the 488 affected tests provide the final-candidate evidence.
Fixture evidence does not certify guest Chromium/SSH/sudo, real accounts, native devices or owner-host adoption. Those remain #431 and the separate cutover gates. Contract/recovery details and exact test paths are recorded in ADR 0092 and the computer browser/screen contracts.
@greptileai please review exact head 4c48178043b6e247d69b3967e3a3a3894ed7718a. Focus on two-store insert/yield ordering and crash-orphan repair; card ID retention and atomic screen outcome events; browser/screen generation collisions; ticket/owner settlement; source selection and committed publication-only delivery on both MODEL and fallback paths; routine holds; async owner/tier/VM guards; and client refresh/session-loss/focus continuity. The PR body and ADR 0092 record concrete validation and the separate live-evidence boundary. Required CI and current-head review remain necessary before merge.
[Medium impact] The reviewed head appears safe to merge; the wrong-computer outcome delivery is fixed.
Summary
Adds the needs-you browser handoff, owner screen control, and durable outcome delivery.
Bots can ask owners to take over the working browser for sign-in help.
Eligible work receives recorded screen outcomes and consumes them when its result commits.
Owners can set how long an untouched computer handoff waits before it expires.
Diagram
sequenceDiagram
participant Bot
participant Conversation
participant Screen
participant Owner
participant NextRun as Next run
Bot->>Conversation: Commit needs-you card
Conversation->>Screen: Yield computer
Owner->>Screen: Take control and sign in
Owner->>Screen: Hand back
Screen->>Screen: Save outcome event
NextRun->>NextRun: Open tools and confirm placed computer
NextRun->>Conversation: Select outcomes for that computer
Conversation-->>NextRun: Supply labelled outcome data
NextRun->>Conversation: Commit result and mark carried cards delivered
@greptileai please re-review exact head 9dc946a83d242de859dc2044b699da754b1316f6 after the actual-VM placement fix. The inline finding has a regression-backed reply and the contract/ADR now select outcomes only after placement. Restoring the old guard reproduces delivery on the wrong shared member. Local affected suites: 488 passed; typecheck passed. Fresh current-head CI/review remain required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bots can now ask for sign-in, verification or CAPTCHA from the watched working browser without keeping a tool call open. The conversation card commits before the computer yields; Watch screen takes exclusive owner control, and hand-back supplies a durable outcome to the next eligible work run. This completes ADR 0092 slice 3c on top of merged slices 3a (#460) and 3b (#461).
Closes #410
Closes #442
Refs #392, #431, #459
Behavior and contracts
computer_browserneeds_youand origin-only inline English/Thai cards. No credential fields, CDP batch or agent ticket are used by the handoff. Typed chat text cannot decide it. Routine calls use the existing occurrenceneeds_youstate without yielding.s16-computer-screen/v1withyield, exact-generation takeover, existing-operation settlement and event-derived resumed/declined/expired outcomes. Card IDs survive owner leases, lapse and controller restart; return to agent clears the row and records the old ID in the same VM transaction. A durable browser-observation high-water mark prevents pre-handoff counters from equalling a new control generation.s16-computer-needs-you/v1commands and guardedneedsYouCardssnapshots. Current origin tier, owner and VM generation are rechecked across waits. Sensitive sites remain in the protected browser. Open owner screens survive snapshot withholding; session/private-content failure and navigation clear them. Closing restores keyboard focus.screenOutcomesas separatescreen-outcome-data-never-authorityhost data in production MODEL and fallback Schway. Preparation waits for actual VM placement: a different shared pool member cannot receive or consume the signed-in computer's outcome. The carried IDs are saved before sending and marked delivered only in committed conversation result publication. Failed/held/cancelled work, missing evidence, rollback, no-computer work and later-arriving outcomes cannot consume them.needsYouMinutes/JIT_COMPUTER_NEEDS_YOU_MINUTES(240 default, 30–1440), independent S15 startup validation, packaged validation/environment/plan support and fixed per-card expiry. Only an untouched yield expires; owner control never does because of card expiry.S14 remains the default and S15 stays opt-in. Installed policy and retained private scope keep their meaning. There is no new execution loop/provider protocol, upstream source import, dependency or live deployment. Automatic continuation after hand-back remains #459; this slice starts the next run through ordinary ingress.
Validation
Current tested/reviewed head:
9dc946a83d242de859dc2044b699da754b1316f6.bun run typecheck,bun run upstream:check,git diff --check: pass.bun test --timeout 30000over screen/browser/actions/needs-you/composition, MODEL, CHAT, config, model-choice/Bots, artifact lineage and VM-policy suites: 488 passed across 12 files at the final head, including the two-shared-member placement regression.JIT_S16_BROWSER_TESTS=1 bun test --timeout 30000over needs-you, computer-screen, model-choice and Bots browser suites: 8 passed across 4 files at4c481780, English 1440 and Thai 390, keyboard control/focus, refresh continuity and session-loss withholding. After the placement fix, the native needs-you English/Thai flow was rerun at9dc946a8: 2 passed.continue, provider failure, completed MODEL withresult_unavailable, publication rollback and late outcomes.9dc946a8; all inline threads are resolved. The local full run started at4c481780and completed after the placement fix: 5,056 passed, 81 skipped, 0 failed. Exact-final-head CI and the 488 affected tests provide the final-candidate evidence.Fixture evidence does not certify guest Chromium/SSH/sudo, real accounts, native devices or owner-host adoption. Those remain #431 and the separate cutover gates. Contract/recovery details and exact test paths are recorded in ADR 0092 and the computer browser/screen contracts.