onfi-test is an Android accessibility-driven screen dump collector built by OnFinance. It registers an AccessibilityService that watches a chosen target app, captures materially changed screens (screenshot + deterministic hierarchy XML + metadata JSON), and uploads artifacts directly to S3 using on-device AWS Signature V4 signing — no broker required.
- Select a target app from the Setup screen.
- Enable the accessibility service in Android settings (the app prompts you automatically).
- Tap Start Capture — the target app launches automatically.
- Use the target app normally. The service watches for meaningful screen changes and for each one:
- normalizes the full accessibility tree (framework-aware: Flutter, React Native, WebView, MAUI, native)
- takes a screenshot via
AccessibilityService.takeScreenshot(...) - writes
screenshot.png,hierarchy.xml, andmetadata.json - queues artifacts for S3 upload
- stores session/screen state in Room
- A draggable floating OnFinance button appears on screen — tap it to stop capture from any app.
- Artifacts are uploaded to S3 directly from the device. S3 bucket and path are shown live in the UI.
- Auto-launch: target app opens automatically when capture starts.
- Smart package following: starts capturing the target app; automatically follows cross-app navigation (OAuth flows, payment SDKs, Chrome Custom Tabs) and captures those too.
- Framework-aware hierarchy: relaxed traversal rules for Flutter (depth 40, no visibility filter), WebView/Ionic (depth 60, 800 nodes), and native Android (depth 20, standard filters). All limits are currently uncapped (
Int.MAX_VALUE) to capture full trees. - Floating stop button: draggable OnFinance icon overlay (20dp, 50% opacity) shown over all apps during capture. Tap to stop. Requires "Display over other apps" permission.
- Reactive accessibility status: UI updates the moment you toggle the service in Android Settings — no app restart needed.
- Direct S3 upload: AWS Signature V4 presigned PUT URLs generated on-device from
BuildConfigcredentials. No broker middleman. - Secure windows: screenshots blocked by secure windows are marked
SECURE_WINDOW_BLOCKEDand the session continues. - Duplicate suppression: perceptual screen hash prevents re-uploading unchanged screens.
The hierarchy XML is extracted from Android's AccessibilityNodeInfo tree — whatever the OS exposes. Accuracy depends on the target app's framework:
| Framework | Screenshot | Hierarchy XML | Notes |
|---|---|---|---|
| Kotlin/Java native | Full | Full | 1:1 view mapping |
| React Native | Full | Good | Near-native; custom components may lack descriptions |
| Xamarin.Android | Full | Good | Native renderer path |
| MAUI / Xamarin.Forms | Full | Partial | Renderer-dependent |
| Flutter | Full | Shallow–good | Synthetic SemanticsNode tree; quality depends on Dart Semantics() widget usage |
| WebView (Ionic, Cordova) | Full | ARIA nodes only | Chrome exposes ARIA-annotated DOM; plain divs invisible |
| Unity / OpenGL | Full | None | GLSurfaceView is opaque to accessibility API |
Framework is detected per-session from native library names in the APK (libflutter.so, libreactnativejni.so, libmono*.so, libil2cpp.so) via PackageManager, with fallback to className patterns in the accessibility tree.
Fill in local.properties (gitignored) before building:
s3.accessKey=AKIA...
s3.secretKey=...
s3.bucket=your-bucket-name
s3.region=ap-south-1These become BuildConfig.S3_* fields. Without them the app runs in LOCAL_ONLY mode — captures are saved on-device only.
S3 upload path structure:
captures/<packageName>/<buildId>/<deviceId>/<sessionId>/screens/<seq>/
| Permission | Purpose |
|---|---|
INTERNET |
S3 upload |
POST_NOTIFICATIONS |
Persistent capture notification |
SYSTEM_ALERT_WINDOW |
Floating stop button overlay |
BIND_ACCESSIBILITY_SERVICE |
Core capture capability |
| Area | Version |
|---|---|
| Gradle wrapper | 9.3.1 |
| Android Gradle Plugin | 9.1.0 |
| Kotlin | 2.2.21 |
| Compose UI | 1.7.6 |
| Material 3 | 1.3.1 |
| AndroidX Activity Compose | 1.9.3 |
| Hilt | 2.57.1 |
| Room | 2.7.2 |
| DataStore | 1.1.1 |
| Coroutines | 1.10.2 |
| Kotlinx Serialization | 1.8.1 |
| OkHttp | 4.12.0 |
All versions are centralized in gradle/libs.versions.toml.
- JDK: 21 (Gradle daemon toolchain)
- Android SDK: API 36 —
minSdk 31(Android 12+) - Application ID:
com.onfinanceai.onfitest local.properties: must containsdk.dir=...(created by Android Studio)
# Build debug APK
./gradlew assembleDebug
# Install on connected device
./gradlew installDebug
# Launch
adb shell am start -n com.onfinanceai.onfitest/com.onfinanceai.onfinance_accessibility.app.MainActivity
# Run unit tests
./gradlew testDebugUnitTest
# Run a single test class
./gradlew testDebugUnitTest --tests "com.onfinanceai.onfinance_accessibility.session.SessionStateMachineTest"
# Run instrumentation tests (requires device/emulator)
./gradlew connectedDebugAndroidTestaccessibility/ OnFinanceAccessibilityService — receives events, triggers capture
capture/ CaptureOrchestrator — debounce, dedup, framework-aware tree normalization
session/ SessionCoordinator — state machine (Idle→Monitoring→Completed), upload queue
FloatingStopOverlay — WindowManager overlay for in-app stop button
CaptureNotificationManager — persistent notification with Stop action
processor/ XmlDumpSerializer, ContrastAnalyzer, LocalScreenAnalyzer, ScreenRuleRepository
network/ S3DirectTransport + AwsV4Signer — on-device presigned PUT, no SDK
storage/ Room DB (sessions/screens/artifacts) + DataStore (target app preference)
ui/setup/ Single-screen UI: Idle → Active → Completed states
app/ MainActivity, InstalledAppsRepository, Hilt application
di/ AppModule — Hilt wiring
<filesDir>/dump-sessions/<session-id>/
session/
app.json — package, bucket, prefix, session metadata
device.json — manufacturer, model, SDK, device ID
manifest.json — session manifest
integrity/
checksums.json — artifact hash inventory
screens/
000001/
screenshot.png
hierarchy.xml — full accessibility tree, framework-aware
metadata.json — per-screen metadata
000002/
...
session/SessionStateMachineTest — state transitions
session/ScreenLogcatFormatterTest — logcat formatting
dump/DumpArtifactsTest — S3 path and manifest logic
processor/ContrastAnalyzerTest — WCAG contrast math
processor/XmlDumpSerializerTest — deterministic XML generation
processor/RawHierarchySerializerTest — raw serialization
processor/ResultMergerTest — dedup findings
network/DumpTransportTest — upload transport
androidTest/SmokeInstrumentedTest — basic instrumentation smoke
Accessibility enabled but nothing captures
- Confirm the session is started and a target app is selected
- Switch into the target app after starting capture
- Check for
SECURE_WINDOW_BLOCKEDin notification — secure apps block screenshots
Floating button not appearing
- Grant "Display over other apps" in Android Settings → Apps → onfi-test → Special app access
- The session still works without it; only the overlay is skipped
S3 upload degraded
- Check
local.propertieshas valids3.accessKey,s3.secretKey,s3.bucket,s3.region - Without valid credentials, app runs in
LOCAL_ONLYmode — artifacts saved on-device only
Hierarchy XML is sparse for target app
- If target is Flutter: quality depends on
Semantics()widget coverage in the Dart code - If target is WebView-based: only ARIA-annotated elements appear
- Screenshot is always complete regardless of hierarchy quality
Gradle toolchain error
- Install JDK 21 locally and point Android Studio /
JAVA_HOMEto it
local.properties missing
- Create it with
sdk.dir=/path/to/android/sdk— Android Studio usually does this automatically
Filesystem paths use com/onfinanceAI/... (capital AI) while Kotlin package declarations use com.onfinanceai... (all lowercase). Both refer to the same package — existing inconsistency in the repo.