Objective
For steps where the user has already stated the value in their spoken utterance (e.g. "pay 2400 rupees" while on the amount step), fill the field automatically via the accessibility action API instead of only pointing at it and asking the user to type it themselves.
Rationale
The repo audit's central product criticism: today the app narrates, it does not automate. The user still types the amount and the account number themselves after being pointed at the field - the accessibility permission's risk (reading the full foreground screen) is not currently repaid by a proportional benefit. Auto-fill, where the value is unambiguous, is the single change most likely to make the core value proposition defensible against "this is TalkBack with extra steps."
Acceptance criteria
Implementation steps
- Extend the planner's tool schema with an optional
extracted_value field (additive - the frozen contract permits new optional fields, never removals)
- Add a deterministic fallback extractor (regex over digits/currency words) for the keyless path, since auto-fill should work without a Sarvam key too, consistent with every other feature in this app
- Implement the accessibility
ACTION_SET_TEXT call in ScreenReaderService or a new narrowly-scoped helper - do not expand ScreenReaderService's read-only contract without discussing it first, since "no gestures" is a stated design principle
- Update the spoken instruction to reflect what was actually done ("I've entered 2400 rupees" vs "Enter the amount here")
- Add unit tests for the value extractor and manual device verification that the real field updates
Estimated effort
High (this is a genuine scope expansion of what the app does, not a bug fix - budget a week including the discussion in step 3 about whether ScreenReaderService's "no gestures" principle needs to become "read-only except for explicit, user-narrated value entry")
Risks
- This is the one change in this list that most looks like "adding a sixth capability" to a frozen five-piece architecture. It should go through the discussion CONTRIBUTING.md asks for before code is written, not be merged as a fait accompli
- Auto-filling the wrong value into a real financial form (even the demo one) is a much worse failure than a wrong highlight - this needs a higher confidence bar than the planner's existing 0.5 floor
- Third-party apps (Uber/Ola/Rapido) should very deliberately NOT be in scope for v1 of this - see the separate RideApps matching issue
Testing checklist
Objective
For steps where the user has already stated the value in their spoken utterance (e.g. "pay 2400 rupees" while on the
amountstep), fill the field automatically via the accessibility action API instead of only pointing at it and asking the user to type it themselves.Rationale
The repo audit's central product criticism: today the app narrates, it does not automate. The user still types the amount and the account number themselves after being pointed at the field - the accessibility permission's risk (reading the full foreground screen) is not currently repaid by a proportional benefit. Auto-fill, where the value is unambiguous, is the single change most likely to make the core value proposition defensible against "this is TalkBack with extra steps."
Acceptance criteria
expects_value: true, the value is set on the targetEditTextviaAccessibilityNodeInfo.performAction(ACTION_SET_TEXT, ...)and the user is told what was filled in (not just what to do)book_taxi, which touches real ride apps)pay_billtask; explicitly out of scope for third-party apps until # is resolved, since a wrong auto-fill in a real ride app is a worse failure mode than a wrong auto-fill in a demo screenImplementation steps
extracted_valuefield (additive - the frozen contract permits new optional fields, never removals)ACTION_SET_TEXTcall inScreenReaderServiceor a new narrowly-scoped helper - do not expandScreenReaderService's read-only contract without discussing it first, since "no gestures" is a stated design principleEstimated effort
High (this is a genuine scope expansion of what the app does, not a bug fix - budget a week including the discussion in step 3 about whether ScreenReaderService's "no gestures" principle needs to become "read-only except for explicit, user-narrated value entry")
Risks
Testing checklist