gemmaDemo

An iOS app that runs Google Gemma 4 E2B entirely on-device using LiteRT-LM. No internet required after the initial model download.

Features:

• Text chat with streaming responses and KV cache session reuse
• Vision — analyze images from your photo library using the on-device model
• Audio — record speech via microphone and transcribe/analyze it locally

Requirements:

• iPhone 13 Pro or later (6+ GB RAM)
• iOS 17.0+
• Xcode 16+

Setup

1. Clone and open

git clone https://github.com/TejPotu/gemmaDemo.git
cd gemmaDemo
open gemmaDemo.xcodeproj

2. Add the LiteRTLMSwift package

In Xcode: File > Add Package Dependencies...

• URL: https://github.com/mylovelycodes/LiteRTLM-Swift.git
• Version rule: Branch > main
• Add LiteRTLMSwift to the gemmaDemo target

3. Fix code signing for bundled dylibs

The package ships pre-signed dylibs that iOS will reject. Two settings are needed:

a) Add a Run Script build phase

Target > Build Phases > + > New Run Script Phase. Drag it to the bottom. Paste:

find "$CODESIGNING_FOLDER_PATH/Frameworks" -type f \( -name '*.dylib' -o -name '*.framework' \) -print0 | while IFS= read -r -d '' item; do
    /usr/bin/codesign --force --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --timestamp=none "$item"
done

Uncheck "Based on dependency analysis".

If EXPANDED_CODE_SIGN_IDENTITY doesn't work, replace it with your literal signing identity. Find it with: security find-identity -v -p codesigning

b) Disable User Script Sandboxing

Target > Build Settings > search User Script Sandboxing > set to No

4. Upgrade the C library for vision support

The LiteRTLM-Swift package bundles a v0.10.x C library that doesn't support Gemma 4's vision encoder. Replace it with v0.11.0 binaries from flutter_gemma:

# Download v0.11.0 iOS arm64 binaries
curl -L -o /tmp/litertlm-ios.tar.gz \
  "https://github.com/DenisovAV/flutter_gemma/releases/download/native-v0.11.0-a/litertlm-ios_arm64.tar.gz"
mkdir -p /tmp/litert-ios && cd /tmp/litert-ios && tar xzf /tmp/litertlm-ios.tar.gz

# Find the framework in DerivedData (adjust the hash if needed)
FRAMEWORK=$(find ~/Library/Developer/Xcode/DerivedData/gemmaDemo-*/SourcePackages/checkouts/LiteRTLM-Swift/Frameworks/LiteRTLM.xcframework/ios-arm64/CLiteRTLM.framework -maxdepth 0 2>/dev/null)

# Replace binaries
chmod u+w "$FRAMEWORK/CLiteRTLM" "$FRAMEWORK/libGemmaModelConstraintProvider.dylib"
cp /tmp/litert-ios/libLiteRtLm.dylib "$FRAMEWORK/CLiteRTLM"
cp /tmp/litert-ios/libGemmaModelConstraintProvider.dylib "$FRAMEWORK/libGemmaModelConstraintProvider.dylib"

# Fix install name
install_name_tool -id @rpath/CLiteRTLM.framework/CLiteRTLM "$FRAMEWORK/CLiteRTLM"

Then edit LiteRTLMEngine.swift in the SourcePackages checkout to enable all backends:

DerivedData/gemmaDemo-*/SourcePackages/checkouts/LiteRTLM-Swift/Sources/LiteRTLMSwift/LiteRTLMEngine.swift

Find this line (~line 117):

litert_lm_engine_settings_create(path, backendStr, backendStr, backendStr)

Make sure all three backends are backendStr (not nil). If vision or audio is nil, that modality is disabled.

Note: This step must be re-applied after a clean build or package re-resolve, since Xcode manages SourcePackages.

5. Add required capabilities

a) Increased Memory Limit

Target > Signing & Capabilities > + Capability > Increased Memory Limit

The model needs ~4 GB to load. Without this entitlement the system will kill the app.

b) Microphone Usage Description

Target > Info > add Privacy - Microphone Usage Description with a value like:

Record audio for on-device transcription and analysis with Gemma 4.

6. Build and run

Clean Build (Shift+Cmd+K), then build and run on a physical device. The first launch will prompt you to download the model (~2.6 GB).

Architecture

File	Purpose
AppViewModel.swift	Core state: engine lifecycle, chat sessions, vision/audio inference, mic recording
ContentView.swift	Root view: routes between download, engine loading, and main tab views
DownloadView.swift	Model download UI with progress, pause/resume, and cancel
ChatView.swift	Multi-turn text chat with streaming token display
VisionView.swift	Image picker + vision analysis
AudioView.swift	Microphone recording + audio analysis
gemmaDemoApp.swift	App entry point, injects AppViewModel into environment

Troubleshooting

Problem	Solution
Library not loaded: libGemmaModelConstraintProvider.dylib	Run Script phase missing or sandboxing enabled (see step 3)
Vision Encoder must have exactly one signature but got 3	C library not upgraded (see step 4)
litert_lm_engine_create returned NULL	Missing increased-memory-limit entitlement (see step 5a)
Operation not permitted in build log	Disable User Script Sandboxing (step 3b)
App killed during model load	Device has insufficient RAM (need iPhone 13 Pro+)

Credits

• LiteRT-LM by Google AI Edge — on-device LLM inference engine
• LiteRTLM-Swift by mylovelycodes — Swift wrapper for the C API
• flutter_gemma by Sasha Denisov — source of the v0.11.0 pre-built iOS binaries that fix vision support
• Gemma 4 model from litert-community on HuggingFace

License

Apache 2.0