Skip to content

refactor(ios-runner): charge the text-entry budget the plan the runner posts - #2977

Open
thymikee wants to merge 2 commits into
mainfrom
fix/2955-text-entry-plan-budget
Open

thymikee wants to merge 2 commits into
mainfrom
fix/2955-text-entry-plan-budget

Conversation

@thymikee

@thymikee thymikee commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Second commit

705ef4bc0 answers the review on #2955's own terms: one step executor serves both synthesized routes. runSynthesizedTextPlan takes the plan, the text, a post closure and the route's warmup read-back, slices the next characters per step, and takes every wait the plan carries. isSpaced and warmupSplit are gone; the phase log's burst/warmup/paced naming derives from the steps themselves. RunnerTests+TextTyping.swift loses 38 net lines and the fill route no longer re-derives anything from delaySeconds.

Closes #2955. The delivery budget from #2904 was a parallel projection: projectedSeconds re-derived the spaced split, the warmup peel and the warmup wait — all decided elsewhere — and under-charged even its own model, counting a select-and-type post as one synthesize call when the bridge runs two (Cmd-A record + text record).

SynthesizedTextPlan now carries the posts a command makes (character counts, whether a post selects first, the wait after it). The budget charges the sum over those steps, and both synthesized routes execute that same array: the loop sleeps the plan's own pauses instead of re-deriving delaySeconds > 0 (twice more, at that). The type route decides the budget once, outside the per-chunk closure, and its budget refusal and synthesizer .fallback arm share one application-wide typing path. The pace moved from the ObjC bridge (whose getter Swift read back through a second source of truth) into TextEntryTiming and is passed to the bridge, so no call site can type at a pace its own budget did not charge. synthesizedAcknowledgeWindowSeconds was production dead weight; it moved into the tests. The fixture's never-asserted edits= counter is gone.

Numbers: fill's admitted length at --delay-ms 0 moves 214 → 212 — honesty about the select-all record the old hint overpromised (at 80 ms both models say 57). The recovery hint now fits the host's 400-char diagnostic bound it previously overflowed by 60, silently losing its last sentence via redactDiagnosticData.

Validation

Head 705ef4bc0. pnpm check:affected --run passed locally (xctest-selection: 0 dark tests; packaged-runner-swift ok) and pnpm check:fallow --base origin/main reports no issues in the changed files.

Local XCUITest evidence on head 705ef4bc0:

  • iOS simulator lane: all 35 text-entry tests across RunnerTests+TextEntryPolicyTests, RunnerTests+SynthesizedTextEntryTests and RunnerTests+TextTypingTests passed (TEST EXECUTE SUCCEEDED), including the two new executor contract tests.
  • macOS host lane: the 21 of those 35 that compile on macOS all passed (TEST EXECUTE SUCCEEDED); the other 14 are os(iOS)-guarded.
  • The pace test's app-side observation and the over-budget refusal, from the runner log on head:
    AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=synthesized-first-responder-replacement
    AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=synthesized-first-responder-replacement reason=delivery-budget-refused chars=213 budgetChars=212
    AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=xctest-application-fallback reason=delivery-budget chars=1 commandChars=213
    AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=xctest-application-fallback reason=delivery-budget chars=212 commandChars=213
    
    testSynthesizedReplacementPacesAnAppOwnedFieldAtItsAcknowledgeWindow passed on that first line's route (16.4 s); testOverBudgetTypeWithoutResolvableElementTypesApplicationWide passed reading the field back after the second pair (16.7 s).

Regression evidence (mutation discipline, head 43ea03ca0):

  • synthesizeCallCount → 1 → testSynthesizedPlanChargesAReplacingPostTwoSynthesizeCalls red at both asserts.
  • peelsWarmupCharacter: false (budget ignores the peel) → testOverBudgetTypeWithoutResolvableElementTypesApplicationWide red: route synthesized-first-responder ≠ xctest-application-fallback — proving the peel charge is what makes the over-budget branch reachable.
  • New pins are policy floors, not formula copies: spaced marginal cost ≥ pace+overhead+delay (full delay; 1e-9 only absorbs float accumulation between the two subtracted sums); plan steps cover the text exactly once (lengths 2/3/11/240 × delays 0/0.08); the executor hands each step its own slice, gives a warmup step the read-back with exactly its own characters, and reports only delivered characters when a post stops the plan; append chunks admitted at fill's own advertised budget; hint ≤ 400 chars.

One regression the executor consolidation found and fixed: reading withElement's frame after a post that refused the text turned testBareDelayedTypeFailsWhenTappedInputDisappearsMidCommand into a snapshot error, because the input is gone by then. A stopped plan now answers with its typed failure before touching the target.

Risk: admission tightened by 2 chars for fill at delay 0 (refuses 213–214 that previously shipped and could time out mid-command). type .none burst admission unchanged. Docs need no change; commands.md:491 describes the hint's shape, which is preserved.

Review in cubic

…r posts

The delivery budget added in #2904 was a parallel projection: projectedSeconds
re-derived the spaced split, the warmup peel and the warmup wait, all decided
elsewhere, and two pre-merge fix commits were drift this shape caused. It also
under-charged the plan it modeled: a post that replaces selects with its own
synthesize record (RunnerSynthesizedTextEntry.m), which the estimate counted as
one call.

SynthesizedTextPlan now carries the posts a command makes (characters, whether
the post selects first, the wait after it), the budget charges the sum over
those steps, and both synthesized routes execute that same array — the loop
sleeps the plan's own pauses instead of re-deriving delaySeconds > 0 twice more.
The type route decides the budget once, outside the per-chunk closure, and its
budget refusal and synthesizer .fallback arm share one application-wide typing
path. The pace moved from the ObjC bridge (whose getter Swift read back) into
TextEntryTiming and is passed to the bridge, so no call site can type at a pace
its own budget never charged. The test-only acknowledge window moved into the
tests, and the fixture's never-asserted edits= counter is gone.

fill's admitted length at --delay-ms 0 moves 214 -> 212 (honesty about the
select-all record, which the old hint overpromised); the recovery hint now fits
the host's 400-character diagnostic bound it previously overflowed by 60, losing
its last sentence on the wire.
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.85 MB 4.85 MB +224 B
Package (unpacked) 4.85 MB 4.85 MB +224 B
Package (download) 1.45 MB 1.45 MB -46 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.6 ms 26.6 ms +0.1 ms
CLI --help 79.9 ms 79.4 ms -0.5 ms

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed 43ea03c. I found no blocking defect. Both synthesized routes now run the step array the budget charges, and the refusal shares the .fallback typing path.

Could one step executor serve both routes? It would take a plan and a post closure, slice the characters, and sleep pauseAfterSeconds for every step, including the warmup step. That would remove the stored isSpaced and warmupSplit and the two slice loops, and bring the change closer to the −80 lines estimated in #2955 (runner production code grows by about +90 now). The open question is whether the element-backed warmup read-back can be a per-step hook, or must stay outside the plan. If it must stay outside, please say why.

Related, not blocking: the warmup step's pause is charged from the plan, but waitForWarmupValue decides its own wait. They agree today only because the budgeted route has no expected warmup text. A few comments also narrate review history (for example TextEntry.swift#L39); keep the invariant and drop the history.

Smoke Tests, Repo Guards and Coverage were still running at review time, with no failures. Smoke Tests exercises this change directly.

Next step: answer the executor question above. For evidence, please attach the simulator log lines from the head commit for the pace test (route=synthesized-first-responder-replacement) and the over-budget test (route=xctest-application-fallback reason=delivery-budget).

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

@okwasniewski

Copy link
Copy Markdown
Contributor

Downstream report from a CI fleet pinning agent-device releases, before this lands.

On main (#2904) runSynthesizedReplacementRoute answers TEXT_INPUT_SYNTHESIS_BUDGET_EXCEEDED when SynthesizedDeliveryBudget.exceeds says the text does not fit: 214 chars at --delay-ms 0, 57 at 80 ms. This PR keeps that refusal and moves the admitted length to 212. For us any long fill in a user test on a channel-penalized simulator becomes an engine refusal, and the test author cannot tell from the test why a 250-char address failed.

Ask, since the budget model is being rewritten here anyway: make the budget a pace, never a refusal.

  • SynthesizedTextPlan already carries the posts a command makes. Let it carry as many posts as the text needs: type in chunks sized to the budget, and let the caller's command deadline be the only thing that ends a long entry early.
  • If a chunk cannot be posted before the deadline, fail with a typed reason that names how many characters landed (observedText prefix length), not a refusal before the first character.
  • The type route already does not refuse: it falls through to xctest-application-fallback and types the whole value. fill refusing what type accepts is the inconsistency callers hit.

Happy to take this as a follow-up PR on top of this one if you prefer to keep the scope here; wanted the ask on record before the admitted length is pinned again.

…utor

Both synthesized routes now run the plan through runSynthesizedTextPlan: the
executor slices the next characters per step, takes the wait the plan carries,
and hands a warmup step to the route's own read-back. The plan records no
shape flag; the burst/warmup/paced phase naming derives from the steps.

A post that stops the plan answers with its typed failure before the target
reads the last element, so a vanished input cannot turn a focused failure into
a snapshot error.
@thymikee

Copy link
Copy Markdown
Member Author

705ef4bc0 takes the shared executor.

One runSynthesizedTextPlan now serves both routes: it takes the plan, the text, a post closure and the route's warmup hook, slices the next characterCount characters per step, and takes every wait the plan carries. isSpaced and warmupSplit are gone — the phase log's type-all / type-first + warmup + type-remaining / type-delayed naming derives from the steps (pacesEveryCharacter, plus whether a warmup step has fired). Net −38 lines in RunnerTests+TextTyping.swift; the fill route no longer re-derives anything from delaySeconds.

The warmup read-back is a per-step hook and must stay outside the plan, for the reason the charge already encodes: the plan owns that a wait is owed and its budgeted cost (one poll), while what the route waits on depends on a resource the plan cannot see. An element-backed route polls the field until it reads back initial + peeled character, up to warmupValueTimeout — that read needs resolveTextEntryElement, editableTextValue and expectedTextEntryValue(mode:initialText:), i.e. the command's repair mode, its initial text and its target. A spaced or burst plan has no such value (it is nil exactly when the route has no element, which is the route the budget exists for), and then the hook is the plan's one charged poll. So the executor hands the hook the characters that post made and skips the charged pause; the charge stays in the plan, the observation stays in the route. If it were a plan field, the plan would have to carry initialText and a repair mode to describe a wait it cannot itself evaluate.

The consolidation also caught a real ordering regression, now pinned: reading withElement's frame after a post that refused the text made testBareDelayedTypeFailsWhenTappedInputDisappearsMidCommand fail with a snapshot error, because the input has vanished by then and the element the last post returned is unreadable. A stopped plan answers with its typed failure before touching the target.

Cubic's three: the counter comment now distinguishes gating from asserting and names min-gap-ms as message-only (the burst counters do drive the pacing assertion); the spaced marginal-cost floor takes the full delay, with 1e-9 only absorbing float accumulation between the two subtracted sums — a dropped charge is two orders of magnitude larger. The third is intended and unchanged here: type cannot refuse on budget the way fill does, because .append recovery is exactly "fill ≤ N, then type the rest", so a refusing type would make the hint's own second step fail; over-budget type text goes application-wide and reports unverified. That is #2906's measurement question (does the paced per-character type route outlive the watchdog?), and the unified route=xctest-application-fallback reason=%@ chars=%d commandChars=%d line is what makes it answerable from CI logs.

Also dropped the review-history sentence on the hint comment, keeping the 400-char bound as the constraint.

Simulator log lines from this head, as asked:

AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=synthesized-first-responder-replacement
AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=synthesized-first-responder-replacement reason=delivery-budget-refused chars=213 budgetChars=212
AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=xctest-application-fallback reason=delivery-budget chars=1 commandChars=213
AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=xctest-application-fallback reason=delivery-budget chars=212 commandChars=213

testSynthesizedReplacementPacesAnAppOwnedFieldAtItsAcknowledgeWindow (16.4 s) and testOverBudgetTypeWithoutResolvableElementTypesApplicationWide (16.7 s) both passed on that run. All 35 text-entry tests passed on the iOS simulator; the 21 of them that compile on macOS passed on the host lane.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextTyping.swift">

<violation number="1" location="apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextTyping.swift:315">
P3: Do not emit a full-text `type-delayed` phase after the plan stops early; it misstates delivery telemetry for mid-command failures. Skip this phase when `planFailure` is set, or log the posted prefix length.</violation>
</file>

<file name="apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedTextEntry.swift">

<violation number="1" location="apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedTextEntry.swift:263">
P2: This warmup branch replaces the plan’s charged pause instead of applying it. The read-back can return immediately, allowing the remaining slice to post faster than the budgeted plan; apply `pauseAfterSeconds` after the callback or include the callback duration in admission.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment on lines +263 to +267
if step.warmsUpField {
waitAfterWarmupCharacter(String(characters[postedCount - step.characterCount..<postedCount]))
} else if step.pauseAfterSeconds > 0 {
sleepFor(step.pauseAfterSeconds)
}

@cubic-dev-ai cubic-dev-ai Bot Sep 26, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This warmup branch replaces the plan’s charged pause instead of applying it. The read-back can return immediately, allowing the remaining slice to post faster than the budgeted plan; apply pauseAfterSeconds after the callback or include the callback duration in admission.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedTextEntry.swift, line 263:

<comment>This warmup branch replaces the plan’s charged pause instead of applying it. The read-back can return immediately, allowing the remaining slice to post faster than the budgeted plan; apply `pauseAfterSeconds` after the callback or include the callback duration in admission.</comment>

<file context>
@@ -225,6 +219,56 @@ extension RunnerTests {
+      }
+      postedCount = nextCount
+      didPostStep(step)
+      if step.warmsUpField {
+        waitAfterWarmupCharacter(String(characters[postedCount - step.characterCount..<postedCount]))
+      } else if step.pauseAfterSeconds > 0 {
</file context>
Suggested change
if step.warmsUpField {
waitAfterWarmupCharacter(String(characters[postedCount - step.characterCount..<postedCount]))
} else if step.pauseAfterSeconds > 0 {
sleepFor(step.pauseAfterSeconds)
}
if step.warmsUpField {
waitAfterWarmupCharacter(String(characters[postedCount - step.characterCount..<postedCount]))
}
if step.pauseAfterSeconds > 0 {
sleepFor(step.pauseAfterSeconds)
}
Fix with cubic

repairMode: repairMode
)
)
if synthesizedTypePlan.pacesEveryCharacter {

@cubic-dev-ai cubic-dev-ai Bot Sep 26, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Do not emit a full-text type-delayed phase after the plan stops early; it misstates delivery telemetry for mid-command failures. Skip this phase when planFailure is set, or log the posted prefix length.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextTyping.swift, line 315:

<comment>Do not emit a full-text `type-delayed` phase after the plan stops early; it misstates delivery telemetry for mid-command failures. Skip this phase when `planFailure` is set, or log the posted prefix length.</comment>

<file context>
@@ -235,138 +235,100 @@ extension RunnerTests {
-        repairMode: repairMode
-      )
+    )
+    if synthesizedTypePlan.pacesEveryCharacter {
+      // Paced delivery is one paced burst: the phase covers every post and names the whole text
+      // rather than its last character.
</file context>
Suggested change
if synthesizedTypePlan.pacesEveryCharacter {
if synthesizedTypePlan.pacesEveryCharacter && planFailure == nil {
Fix with cubic

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Synthesized text-entry budget: cost the executed plan instead of a parallel projection

2 participants