Repository navigation
fix(limrun): make Android provider text entry replace the field for fill #3362
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
2e3a458
fix(limrun): make Android provider text entry replace the field for fill
thymikee c2d19ee
fix(limrun): keep the text-entry module off the provider's eager impo…
thymikee e4e95f7
test(limrun): pin the session adb text seam's replace routing
thymikee f8f1196
test(android): pin that a silent provider clear fails the empty fill
thymikee File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| import { expect, test, vi } from 'vitest'; | ||
| import type { AndroidTextInjectionRequest } from '@agent-device/platform-android/mechanics'; | ||
| import { createLimrunAndroidTextInjector } from './android-text-entry.ts'; | ||
|
|
||
| type ClientCall = 'tap' | 'pressKey:a+ctrl' | 'pressKey:del' | `setText:${string}`; | ||
|
|
||
| function sessionWithClient() { | ||
| const calls: ClientCall[] = []; | ||
| const client = { | ||
| tap: vi.fn(async (target: { x: number; y: number }) => { | ||
| calls.push('tap'); | ||
| return target; | ||
| }), | ||
| pressKey: vi.fn(async (key: string, modifiers?: string[]) => { | ||
| calls.push( | ||
| `pressKey:${key}${modifiers?.length ? `+${modifiers.join('+')}` : ''}` as ClientCall, | ||
| ); | ||
| }), | ||
| setText: vi.fn(async (_target: { x: number; y: number } | undefined, text: string) => { | ||
| calls.push(`setText:${text}`); | ||
| return { textLength: text.length }; | ||
| }), | ||
| }; | ||
| return { injector: createLimrunAndroidTextInjector(client), client, calls }; | ||
| } | ||
|
|
||
| function fillRequest( | ||
| overrides: Partial<AndroidTextInjectionRequest> = {}, | ||
| ): AndroidTextInjectionRequest { | ||
| return { action: 'fill', target: { x: 540, y: 220 }, text: 'jane@example.com', ...overrides }; | ||
| } | ||
|
|
||
| // The instance's setText inserts at the focused field's cursor, so a fill that only calls | ||
| // setText concatenates over the old value (#3358). The injector must clear through the | ||
| // instance's own input channel before the text goes in. | ||
| test('a fill focuses the target, clears the selection, and only then sets the text', async () => { | ||
| const { injector, client, calls } = sessionWithClient(); | ||
|
|
||
| await injector(fillRequest()); | ||
|
|
||
| expect(calls).toEqual(['tap', 'pressKey:a+ctrl', 'pressKey:del', 'setText:jane@example.com']); | ||
| expect(client.tap).toHaveBeenCalledWith({ x: 540, y: 220 }); | ||
| // The text follows the selection the injector just established, so it goes to the field the | ||
| // injector tapped, not to whatever the instance would re-resolve from the coordinates. | ||
| expect(client.setText).toHaveBeenCalledWith(undefined, 'jane@example.com'); | ||
| }); | ||
|
|
||
| // `fill ""` is the clear request (#2063), and the instance rejects an empty setText ("text is | ||
| // required"), so the clear stands on its own and no text is sent. | ||
| test('an empty fill clears the field without asking setText to set empty text', async () => { | ||
| const { injector, client, calls } = sessionWithClient(); | ||
|
|
||
| await injector(fillRequest({ text: '' })); | ||
|
|
||
| expect(calls).toEqual(['tap', 'pressKey:a+ctrl', 'pressKey:del']); | ||
| expect(client.setText).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| // `type` appends by contract: neither the focus tap nor the clear may ride that action. | ||
| test('a type sends text to the target with no tap and no clear', async () => { | ||
| const { injector, client, calls } = sessionWithClient(); | ||
|
|
||
| await injector({ action: 'type', text: 'appended' }); | ||
|
|
||
| expect(calls).toEqual(['setText:appended']); | ||
| expect(client.setText).toHaveBeenCalledWith(undefined, 'appended'); | ||
| expect(client.tap).not.toHaveBeenCalled(); | ||
| expect(client.pressKey).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| test('a fill without coordinates still clears the focused field before typing', async () => { | ||
| const { injector, client, calls } = sessionWithClient(); | ||
|
|
||
| await injector(fillRequest({ target: undefined })); | ||
|
|
||
| expect(calls).toEqual(['pressKey:a+ctrl', 'pressKey:del', 'setText:jane@example.com']); | ||
| expect(client.tap).not.toHaveBeenCalled(); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| /** | ||
| * Text entry on a Limrun Android instance. The instance's `setText` inserts at the focused | ||
| * field's cursor and rejects empty text, so it cannot honor `fill`'s replace contract alone: | ||
| * the injector owns the replacement — focus, select-all, delete — and hands `setText` only the | ||
| * text that replaces the selection. Selecting through the instance's own key events keeps the | ||
| * clear on the same input channel as the typing that follows it; a second channel racing the | ||
| * selection (adb keyevents) could lose it. `type` keeps appending: no focus, no clear. | ||
| */ | ||
| import type { AndroidTextInjectionRequest } from '@agent-device/platform-android/mechanics'; | ||
|
|
||
| type LimrunAndroidTextClient = { | ||
| tap: (target: { x: number; y: number }) => Promise<unknown>; | ||
| pressKey: (key: string, modifiers?: string[]) => Promise<unknown>; | ||
| setText: (target: { x: number; y: number } | undefined, text: string) => Promise<unknown>; | ||
| }; | ||
|
|
||
| export function createLimrunAndroidTextInjector( | ||
| client: LimrunAndroidTextClient, | ||
| ): (request: AndroidTextInjectionRequest) => Promise<void> { | ||
| return async (request) => { | ||
| if (request.action !== 'fill') { | ||
| await client.setText(request.target, request.text); | ||
| return; | ||
| } | ||
| if (request.target) { | ||
| await client.tap(request.target); | ||
| } | ||
| // Select what the focused field holds, then remove it: the field is now empty and the | ||
| // caret is the insertion point. `del` is the SDK's backspace name, the delete the issue | ||
| // reproduction verified against the stock Latin IME. | ||
| await client.pressKey('a', ['ctrl']); | ||
| await client.pressKey('del'); | ||
| // The empty fill is the clear request (#2063), which the clear above just completed; | ||
| // the SDK's setText refuses an empty text, so the clear request never sends one. | ||
| if (request.text.length > 0) { | ||
| await client.setText(undefined, request.text); | ||
| } | ||
| }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.