Gemma 4 Agentic - On-Device AI Assistant

A powerful, private, and fully on-device AI assistant built with Flutter and Gemma 4. This application leverages multimodal vision and agentic tool-calling to perform real-world actions directly on your mobile device.

In our latest feature/a2ui-agentic-upgrade branch, we transitioned the core codebase from a prototype MVP into a production-grade, AI-native platform, incorporating cutting-edge paradigms from Google I/O 2026.

---

🚀 Key Flagship Features

• Google A2UI (Agent-to-User-Interface) Generative UI:

  • Instead of rendering raw Markdown or basic text blocks, the assistant streams declarative JSON specifications directly matching the open A2UI standard.
  • The client app securely parses, validates, and maps these specifications to stunning native, HSL-gradient glassmorphic components (ContactCard, WebResultsCard, AlarmStatusWidget, WeatherCard, ProductCarousel).
  • Dynamic GenUI Card Generation: Absolutely zero product categories or search queries are hardcoded. The search engine retrieves live DuckDuckGo results, and the local Gemma model uses in-context A2UI instruction templates to dynamically build custom native cards (e.g. carousels for shoes, shirts, pants, or electronics) on the fly.
  • Strict Security Sandbox: High-level whitelist validation prevents dynamic code evaluation or runtime script injection. Hallucinated widgets or bad keys are immediately caught by A2uiController and mapped to ContentUnavailableCard, preventing silent errors. Fully whitelisted and defined strict contract validators for 'WeatherCard'.

• Closed-Loop Agentic Tool Calling:

  • Upgraded from hardcoded text-replacement to a circular execution loop: Model Prompt ➔ Tool Trigger ➔ Local Device Execution ➔ System Context Feeding ➔ Model Conversational Summary.
  • Executing local platform tools (contacts list search, structured search API, alarms, app launcher) feeds the environmental outcome back as a system-role message in the model's active memory history, keeping conversations natural and adaptive.

• Real-Time Data Integrations & Anti-Mock Layers:

  • Live Weather Integration: Replaced simulated meteorological metrics with real-time fetching from the open wttr.in API over HTTP. Upgraded parsing logic with a 5-regex parser capable of identifying temperatures in multiple variations (e.g., +33°C, 33° C, 33C, 33 degrees Celsius).
  • Hardware Clock Interceptor: Accelerates time/date queries for the local region (India/present) by intercepting the query, bypassing the network, and querying the device's physical system clock to render the time instantly in a validated WebResultsCard.
  • Genuine Product Aggregation: Replaced static shopping placeholders with live DuckDuckGo search source results, extracting genuine page titles, store URLs, and using regular expressions to grab real-world prices (\$(\d+(?:\.\d{2})?)).

• Under-the-Hood Engine Stability:

  • EGL Thread ANR Safe-Guard: Prevented startup Application Not Responding (ANR) lockups caused by compiling heavy GPU/OpenCL LiteRT-LLM shaders on the main rendering thread. Model compilation is now deferred to a thread-safe lazy-loading trigger on the first user message.
  • Syntax Escaping & Diagnostics: Resolved Dart syntax issues like missing_identifier compile-time compiler exceptions by escaping raw pricing symbols (r"$"), ensuring 100% warning-free static analysis (flutter analyze with 0 issues).

• Decoupled State Machine Architecture:

  • Eliminated the monolithic 1,000+ line UI spaghetti file by separating the codebase into clean, reusable widgets (SidebarDrawer, ChatInputArea, MessageBubble, DeveloperPanel).
  • Centralized all AI states, audio streams, voice prompts, and life-cycles inside a unified state machine called ChatController extending ChangeNotifier to guarantee synchronization.

• Model Capability Failsafes (ModelCapability):

  • Prior to user interaction, the loader evaluates active model specifications (isMultimodal, maxContextLength).
  • UI components reactively adapt; for instance, the camera picker is securely locked and disabled when a text-only model like gemma-4-E2B-it is loaded, protecting the underlying LiteRT engine from memory allocation exceptions.

• Latency Telemetry Panel:

  • A hidden debugger panel that tracks Time to First Token (TTFT) using a 20-frame simple moving average (SMA).
  • Includes a Thermal Throttling Alert that triggers if latency continually degrades, allowing developers to optimize model weights for target mobile hardware.
  • Tracks Action Success Rate (ASR) to measure agentic reliability.

• 🎨 Premium Obsidian & Champagne Gold UI/UX:

  • OLED Smear Prevention: Global scaffold canvas background is set strictly to #121214 (Zinc-950) to slightly activate OLED pixels during scrolling list views, preventing physical "purple smudge" smearing against low-brightness #18181B Obsidian cards.
  • High-Yield Gold Restraint: Enforces strict neutral body typography (#F4F4F5 and #D4D4D8) for long-session readability, reserving the #D4AF37 Champagne Gold accent exclusively as a scarce utility token for active indicators (selected highlights, alarms, weather icon paths, and CTA triggers).
  • Tactile, SnackBar-Reactive Inputs: Replaced silent, dead button states with always-interactive camera, voice, and text inputs. They provide animated feedback and trigger descriptive Snackbar toasts if the model is uninitialized, microphone permission is denied, or vision is unsupported by the active model.
  • Premium Selected Highlights: Upgraded active drawer items with a clean, left-hand vertical gold border highlight instead of gaudy text coloring, serving as an elegant bounding indicator of selected states.
  • Unified Color Space: Purged all legacy slate-blue (#1E293B) backgrounds in alert dialogs, bringing all dialog sheets into the standard #18181B Obsidian container styling.

---

🏗️ Technical Architecture

This application follows a strict Feature-First Architecture divided into decoupled layers:

lib/features/chat/
├── data/
│   ├── history_service.dart     # SQLite/SharedPrefs conversation records
│   └── web_search_service.dart    # High-speed structured query engine
├── domain/
│   └── models/
│       ├── chat_history.dart      # Conversation schema
│       ├── chat_message.dart      # Chat bubble model
│       └── model_capability.dart  # AI capability rules manifest
├── logic/
│   ├── chat_controller.dart     # Centralized State Machine Manager (Notifier)
│   ├── chat_service.dart        # Gemma LiteRT binding & session manager
│   ├── tool_handler.dart        # Platform Intent executor (Phone, Alarm)
│   └── a2ui_controller.dart     # Declarative JSON whitelist schema validator
└── presentation/
    ├── screens/
    │   └── chat_screen.dart     # Modular, lightweight UI Shell (<320 lines)
    ├── components/
    │   ├── a2ui_renderer.dart   # Secure composer building dynamic cards
    │   ├── contact_card.dart    # Dial shortcut launcher
    │   ├── weather_card.dart    # Meteorological gradient visualizer
    │   ├── alarm_status_widget.dart # Alarm scheduling reporter
    │   └── content_unavailable_card.dart # Hallucination failsafe card
    └── widgets/
        ├── sidebar_drawer.dart  # slide-out drawer history launcher
        ├── chat_input_area.dart # Camera and Speech-to-Text active inputs
        ├── message_bubble.dart  # Custom chat text bubble composer
        └── developer_panel.dart # Latency moving average tracker

---

🏁 Getting Started

1. Clone the repository & Checkout Branch:

   git clone https://github.com/Davilash/gemma-on-device-flutter.git
   cd gemma-on-device-flutter
   git checkout feature/a2ui-agentic-upgrade

2. Install dependencies:

   flutter pub get

3. Run Static Diagnostics: Ensure the clean code matches all analysis rules before launching:

   flutter analyze

4. Download the Model: Upon first launch, the app scans for local models. If none are found, follow the download dialog to retrieve the reasoning-optimized model (gemma-4-E2B-it ~390MB).

---

🏁 How to Build

Compile directly using local gradle packaging rules:

cd android
./gradlew assembleDebug

The APK will be generated at: android/app/build/outputs/apk/debug/app-debug.apk

---

Built with ❤️ using Flutter & Google Gemma.