diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 6d8052f..a6b0f78 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -30,9 +30,9 @@ jobs: run: | echo "y" | $ANDROID_SDK_ROOT/cmdline-tools/latest/bin/sdkmanager "ndk;29.0.13113456" - - name: Clone llama.cpp (pinned to b8390) + - name: Clone llama.cpp (pinned to b9574) run: | - git clone --depth 1 --branch b8390 https://github.com/ggerganov/llama.cpp.git $GITHUB_WORKSPACE/../llama.cpp-master + git clone --depth 1 --branch b9574 https://github.com/ggerganov/llama.cpp.git $GITHUB_WORKSPACE/../llama.cpp-master - name: Remove prebuilt native libs (compiled from source in CI) run: rm -rf lib-llama-android/src/main/jniLibs diff --git a/.gitignore b/.gitignore index bc060d7..ad1888f 100644 --- a/.gitignore +++ b/.gitignore @@ -41,3 +41,6 @@ CLAUDE.md # Build artifacts build_log.txt + +# Local Claude Code config (never publish) +.claude/ diff --git a/README.md b/README.md index 60b0ff6..b865307 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,6 @@ [![Min SDK](https://img.shields.io/badge/Min%20SDK-API%2030-green.svg)](https://developer.android.com/about/versions/11/highlights) [![Build](https://github.com/Helldez/HearoPilot-App/actions/workflows/build.yml/badge.svg)](https://github.com/Helldez/HearoPilot-App/actions/workflows/build.yml) - - Get it on Google Play - - **On-device audio transcription and AI insights for Android** HearoPilot combines a local Speech-to-Text engine with a local Large Language Model to deliver @@ -19,7 +15,7 @@ server. ## Features - **Real-time transcription** — streaming STT via Sherpa-ONNX (NeMo Parakeet TDT 0.6B Int8) -- **On-device AI insights** — contextual analysis via local LLM, fully on-device +- **On-device AI insights** — contextual analysis via llama.cpp (Gemma 3 1B Q6_K) - **100% offline** — privacy-first; no network calls during recording - **Four recording modes** — Simple Listening, Short Meeting, Long Meeting, Real-Time Translation - **25 UI languages** — full i18n including localized LLM system prompts @@ -133,7 +129,7 @@ AudioRecord (16 kHz mono, PCM 16-bit, AudioSource.MIC) | Parameter | Value | Rationale | |---|---|---| -| `AudioSource.MIC` | — | Delivers raw PCM; `VOICE_RECOGNITION` activates HAL noise reduction on some devices that degrades transducer accuracy | +| `AudioSource.MIC` | — | Delivers raw PCM; `VOICE_RECOGNITION` activates HAL noise reduction on some devices (e.g. OnePlus) that degrades transducer accuracy | | Recording thread priority | `URGENT_AUDIO` | Prevents audio buffer drops under CPU load | | ADPF hint (VAD/ASR thread) | 50 ms target | Signals scheduler to prefer big cores on big.LITTLE SoCs; prevents ASR parking on efficiency cores | | VAD window size | 512 samples | Standard Silero-VAD frame size | @@ -172,6 +168,9 @@ models to echo the wrapper keywords instead of translating. ### llama.cpp Optimizations (native layer — `ai_chat.cpp`) `libai-chat.so` is compiled from C++ source on every build (CMake, `externalNativeBuild`). +The prebuilt `.so` was removed in commit `2380ef7`; always run `./gradlew assembleDebug` after +changing any C++ parameter to pick up the new binary. + | Optimization | Value / Setting | Rationale | |---|---|---| | Context size | 4096 tokens | Covers worst-case LONG_MEETING (~3968 tokens) while halving KV cache memory vs 8192 | @@ -209,7 +208,7 @@ Token budget flows from `SyncSttLlmUseCase` → `LlmRepository` → `LlamaAndroi ### Indicative Targets -- STT latency: < 200 ms per segment (high-end ARM device) +- STT latency: < 200 ms per segment (Snapdragon 888) - LLM inference: 2–8 s per insight (varies by chip and mode) - Memory footprint: < 800 MB active (STT + LLM loaded) @@ -254,7 +253,7 @@ git lfs install # 2. Clone (LFS objects are downloaded automatically) git clone https://github.com/Helldez/HearoPilot-App.git -cd HearoPilot +cd HearoPilot-App # 3. Configure Firebase # Copy the example file and fill in your own Firebase project credentials. @@ -285,6 +284,32 @@ adb install app/build/outputs/apk/debug/app-debug.apk --- +## Project History + +Key milestones from MVP to the current release: + +- Initial MVP — on-device STT + LLM, Clean Architecture +- Foreground Service for screen-off recording +- Room database for persistent sessions +- Custom `ModelDownloadManager` with partial-file resume +- VAD configurability (threshold, min silence, max speech) +- GC-free audio buffer + min audio context gate +- i18n: 25 languages with fully localized LLM system prompts +- Material Design 3 + dark mode +- Four recording modes (Simple, Short Meeting, Long Meeting, Real-Time Translation) +- R8 full-mode + ProGuard +- Stateless LLM inference + power optimizations +- LLM system prompt KV cache reuse +- Adaptive LLM unload/reload for Long Meeting (kswapd ANR fix) +- Global transcription search (full-text, highlighted snippets) +- Inline editing of segments, insights, and tasks +- Dual LLM variant support (Q8\_0 / IQ4\_NL) with auto device-tier detection +- Foreground services for model download and LLM batch processing +- Adaptive conservative-thread logic with persisted learning +- `libai-chat.so` compiled from source via CMake (no prebuilt) + +--- + ## Localization Supported locales (25): `en`, `bg`, `cs`, `da`, `de`, `el`, `es`, `et`, `fi`, `fr`, `hr`, `hu`, @@ -296,6 +321,34 @@ consistent JSON parsing. --- +## Troubleshooting + +**App crashes on launch** +→ Check that `RECORD_AUDIO` permission is granted +→ Run `./gradlew.bat clean assembleDebug` and verify zero errors + +**STT produces no output** +→ Confirm the STT model is downloaded (check Settings) +→ Test on a physical device — emulator audio capture is unreliable +→ `adb logcat | grep SherpaOnnx` + +**AI Insights not generated** +→ Confirm the LLM model is downloaded +→ Device needs ~1.5 GB free RAM with the LLM model loaded +→ Check the status chip in the TopBar shows "Ready" +→ `adb logcat | grep "LlamaAndroid\|SyncSttLlm"` + +**Download stuck or slow** +→ Downloads resume from where they stopped — just retry +→ Ensure stable Wi-Fi; models are large (STT ~670 MB, LLM ~1 GB) + +**Transcription inaccurate** +→ Speak clearly at ~30 cm from the microphone +→ Reduce background noise +→ Do **not** use `VOICE_RECOGNITION` audio source (OnePlus HAL issue) + +--- + ## Roadmap - [x] Global transcription search (full-text, highlighted snippets) @@ -304,6 +357,7 @@ consistent JSON parsing. - [ ] Export sessions (TXT / JSON / PDF) - [ ] Speaker diarization - [ ] Home screen widget for quick record +- [ ] Optional end-to-end-encrypted cloud sync - [ ] Kotlin Multiplatform (iOS support) --- diff --git a/app/build.gradle.kts b/app/build.gradle.kts index 32bb94b..55d9e68 100644 --- a/app/build.gradle.kts +++ b/app/build.gradle.kts @@ -20,8 +20,8 @@ android { applicationId = "com.hearopilot.app" minSdk = 30 targetSdk = 35 - versionCode = 20260327 - versionName = "1.25.00" + versionCode = 20260609 + versionName = "1.32.01" testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner" vectorDrawables { diff --git a/app/src/main/ic_launcher-playstore.png b/app/src/main/ic_launcher-playstore.png index 1ef0e76..d070168 100644 Binary files a/app/src/main/ic_launcher-playstore.png and b/app/src/main/ic_launcher-playstore.png differ diff --git a/app/src/main/jniLibs/arm64-v8a/libonnxruntime.so b/app/src/main/jniLibs/arm64-v8a/libonnxruntime.so index 9b1bab7..b73c766 100644 --- a/app/src/main/jniLibs/arm64-v8a/libonnxruntime.so +++ b/app/src/main/jniLibs/arm64-v8a/libonnxruntime.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:c7c0e6b0f9c364a4df4bfc9a3a3576b691899e96f4d9beb60f3b3a7ce4bf572c -size 15988232 +oid sha256:4d2318b3849abb8862133d3068fc7e807ed8b2671cc6d83657fff2fcb9e1caad +size 25831632 diff --git a/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so b/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so index 51befae..f0e2bc7 100644 --- a/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so +++ b/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:136c16ef29ba19d8681a811027c7d7bdc3a427e2df156e00e3cf624c133149cb -size 4705424 +oid sha256:ec09f16e2e5043898d4dd18ec5c316310a39b7ca74ce4db0a33a3b945790e2ab +size 4327992 diff --git a/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so b/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so index 3716399..e5fe060 100644 --- a/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so +++ b/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:659d75245e369c79bd2ec03614ef4a15bf6043e51d1d0484ec4e581aee538bcf -size 410976 +oid sha256:e330684b081ded3d3d694df44abce6de313ccee82e119121d2ea342e5ce0e3d2 +size 419800 diff --git a/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so b/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so index da28dd6..d6b73c6 100644 --- a/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so +++ b/app/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:5d120ad84111b2799b6425369b847ca1bea0a9b911659d26eeef2244e66b7267 -size 5008104 +oid sha256:fc072f201dc1923ee98b594eb61c796b538ef087f7f18d08dcfdf0565167a8bd +size 4623192 diff --git a/app/src/main/res/drawable/ic_launcher_background.xml b/app/src/main/res/drawable/ic_launcher_background.xml index 5662308..57b0262 100644 --- a/app/src/main/res/drawable/ic_launcher_background.xml +++ b/app/src/main/res/drawable/ic_launcher_background.xml @@ -1,8 +1,11 @@ - + + android:type="radial" + android:startColor="#1F1640" + android:endColor="#15102A" + android:centerX="0.5" + android:centerY="0.5" + android:gradientRadius="60dp"/> diff --git a/app/src/main/res/drawable/ic_launcher_foreground.xml b/app/src/main/res/drawable/ic_launcher_foreground.xml index 0ae698f..7ca731f 100644 --- a/app/src/main/res/drawable/ic_launcher_foreground.xml +++ b/app/src/main/res/drawable/ic_launcher_foreground.xml @@ -1,19 +1,90 @@ - - - + android:viewportWidth="4057" + android:viewportHeight="4057"> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/app/src/main/res/drawable/ic_launcher_monochrome.xml b/app/src/main/res/drawable/ic_launcher_monochrome.xml new file mode 100644 index 0000000..c632528 --- /dev/null +++ b/app/src/main/res/drawable/ic_launcher_monochrome.xml @@ -0,0 +1,69 @@ + + + + + + + + + + + + + + + + + + + + + + diff --git a/app/src/main/res/drawable/logo_hearo_pilot_3.png b/app/src/main/res/drawable/logo_hearo_pilot_3.png deleted file mode 100644 index 91137ec..0000000 Binary files a/app/src/main/res/drawable/logo_hearo_pilot_3.png and /dev/null differ diff --git a/app/src/main/res/drawable/logo_hearo_pilot_new.xml b/app/src/main/res/drawable/logo_hearo_pilot_new.xml index 57d38fe..cf46e35 100644 --- a/app/src/main/res/drawable/logo_hearo_pilot_new.xml +++ b/app/src/main/res/drawable/logo_hearo_pilot_new.xml @@ -1,23 +1,78 @@ - + android:viewportWidth="4057" + android:viewportHeight="4057"> + + + + - - - - - + android:pivotX="3438" + android:pivotY="3438" + android:scaleX="1.2" + android:scaleY="1.2"> + + + + + + + + + + + + + + + + + + diff --git a/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml b/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml index bbd3e02..80faec8 100644 --- a/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml +++ b/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml @@ -2,4 +2,5 @@ - \ No newline at end of file + + diff --git a/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml b/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml index bbd3e02..04091bd 100644 --- a/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml +++ b/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml @@ -2,4 +2,5 @@ + \ No newline at end of file diff --git a/app/src/main/res/mipmap-hdpi/ic_launcher.webp b/app/src/main/res/mipmap-hdpi/ic_launcher.webp index d45f8d6..4e18c96 100644 Binary files a/app/src/main/res/mipmap-hdpi/ic_launcher.webp and b/app/src/main/res/mipmap-hdpi/ic_launcher.webp differ diff --git a/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webp index f8bc7c4..c8d4f9e 100644 Binary files a/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webp and b/app/src/main/res/mipmap-hdpi/ic_launcher_foreground.webp differ diff --git a/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp index f1e214e..320209f 100644 Binary files a/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp and b/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp differ diff --git a/app/src/main/res/mipmap-mdpi/ic_launcher.webp b/app/src/main/res/mipmap-mdpi/ic_launcher.webp index 99ebf2f..a3255ac 100644 Binary files a/app/src/main/res/mipmap-mdpi/ic_launcher.webp and b/app/src/main/res/mipmap-mdpi/ic_launcher.webp differ diff --git a/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webp index 638a2ee..5bfa64d 100644 Binary files a/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webp and b/app/src/main/res/mipmap-mdpi/ic_launcher_foreground.webp differ diff --git a/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp index 881db17..57b095c 100644 Binary files a/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp and b/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp differ diff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher.webp index 4bb0471..3b59b96 100644 Binary files a/app/src/main/res/mipmap-xhdpi/ic_launcher.webp and b/app/src/main/res/mipmap-xhdpi/ic_launcher.webp differ diff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webp index f7f8bad..9b9f473 100644 Binary files a/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webp and b/app/src/main/res/mipmap-xhdpi/ic_launcher_foreground.webp differ diff --git a/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp index d69efef..93f7e2d 100644 Binary files a/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp and b/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp differ diff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp index c7cb7b7..8723190 100644 Binary files a/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp and b/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp differ diff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webp index f1d6ee8..31b6e6e 100644 Binary files a/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webp and b/app/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.webp differ diff --git a/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp index ec8b070..30d0ab9 100644 Binary files a/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp and b/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp differ diff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp index fed7c25..bff66aa 100644 Binary files a/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp differ diff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webp index d84893d..3b95829 100644 Binary files a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webp and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.webp differ diff --git a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp index 38feb17..510c628 100644 Binary files a/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp and b/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp differ diff --git a/data/src/main/java/com/hearopilot/app/data/config/ModelConfig.kt b/data/src/main/java/com/hearopilot/app/data/config/ModelConfig.kt index 8a0b5d9..00bea6a 100644 --- a/data/src/main/java/com/hearopilot/app/data/config/ModelConfig.kt +++ b/data/src/main/java/com/hearopilot/app/data/config/ModelConfig.kt @@ -27,7 +27,7 @@ private val STT_FILES = listOf( ) /** - * Q8_0 model configuration — higher accuracy, ~1 GB. + * Q8_0 model configuration — ~1 GB. * Recommended for flagship devices (≥ 10 GB RAM, Cortex-A78 / Oryon CPU or newer). * * LLM : Gemma 3 1B Q8_0 (ggml-org, HuggingFace) @@ -48,7 +48,7 @@ object DefaultModelConfig { } /** - * IQ4_NL model configuration — ~35 % smaller (~650 MB), lower CPU load and battery use. + * IQ4_NL model configuration — ~650 MB, lower CPU load and battery use. * Recommended for mid-range and older devices (Cortex-A77 / Snapdragon 865 era and below). * * LLM : Gemma 3 1B IQ4_NL (unsloth, HuggingFace) diff --git a/domain/src/main/java/com/hearopilot/app/domain/model/LlmModelVariant.kt b/domain/src/main/java/com/hearopilot/app/domain/model/LlmModelVariant.kt index 442e14b..058241b 100644 --- a/domain/src/main/java/com/hearopilot/app/domain/model/LlmModelVariant.kt +++ b/domain/src/main/java/com/hearopilot/app/domain/model/LlmModelVariant.kt @@ -8,13 +8,13 @@ package com.hearopilot.app.domain.model */ enum class LlmModelVariant { /** - * 8-bit quantization — higher accuracy, ~1 GB on disk. + * 8-bit quantization — higher accuracy, ~5 GB on disk. * Suited to flagship devices with ≥ 10 GB RAM and a modern CPU (Cortex-A78 / Oryon or newer). */ Q8_0, /** - * 4-bit non-linear quantization — ~35 % smaller (~650 MB), lower CPU load and battery use. + * 4-bit non-linear quantization — ~3 GB on disk, lower CPU load and battery use. * Recommended for mid-range and older devices (e.g. Snapdragon 865 / Cortex-A77 era). */ IQ4_NL, diff --git a/domain/src/main/java/com/hearopilot/app/domain/usecase/llm/InsightOutputParser.kt b/domain/src/main/java/com/hearopilot/app/domain/usecase/llm/InsightOutputParser.kt index f051fc5..7ec6983 100644 --- a/domain/src/main/java/com/hearopilot/app/domain/usecase/llm/InsightOutputParser.kt +++ b/domain/src/main/java/com/hearopilot/app/domain/usecase/llm/InsightOutputParser.kt @@ -71,28 +71,42 @@ object InsightOutputParser { } /** - * Strip ... reasoning blocks emitted by hybrid thinking models - * (e.g. Qwen3.5) before parsing the JSON payload. + * Strip reasoning/thinking blocks before parsing the JSON payload. * - * NOTE: this is a *defensive fallback* only. The primary suppression happens at - * the C++ layer in ai_chat.cpp (processUserPrompt) by pre-filling an empty think - * block so the model never generates reasoning tokens in the first place. - * If that workaround is ever removed or a future model bypasses it, this strip - * prevents the raw block from surfacing as insight content. + * Supported formats: + * - Qwen3.5: ... + * - Gemma 4: <|channel>thought\n...\n * - * This must NOT be treated as the main solution: reasoning tokens still consume - * the token budget even when stripped here, degrading the quality of the JSON - * that follows them. + * NOTE: this is a *defensive fallback* only. The primary suppression for Qwen3.5 + * happens at the C++ layer in ai_chat.cpp by pre-filling an empty think block. + * Gemma 4 thinking is stripped here because its Jinja2 template inserts <|think|> + * automatically; a C++ suppression equivalent is not yet implemented. + * + * Reasoning tokens still consume the token budget even when stripped here, + * degrading the quality of the JSON that follows them. */ fun stripThinkingBlock(text: String): String { - val start = text.indexOf("") - if (start == -1) return text - val end = text.indexOf("", startIndex = start) - if (end == -1) { - // Unclosed think block (model was cut off mid-reasoning): discard everything. - return text.substring(0, start).trim() + // Qwen3.5: ... + val qwenStart = text.indexOf("") + if (qwenStart != -1) { + val qwenEnd = text.indexOf("", startIndex = qwenStart) + return if (qwenEnd == -1) { + text.substring(0, qwenStart).trim() + } else { + (text.substring(0, qwenStart) + text.substring(qwenEnd + "".length)).trim() + } + } + // Gemma 4: <|channel>thought\n...\n + val gemmaStart = text.indexOf("<|channel>thought") + if (gemmaStart != -1) { + val gemmaEnd = text.indexOf("", startIndex = gemmaStart) + return if (gemmaEnd == -1) { + text.substring(0, gemmaStart).trim() + } else { + (text.substring(0, gemmaStart) + text.substring(gemmaEnd + "".length)).trim() + } } - return (text.substring(0, start) + text.substring(end + "".length)).trim() + return text } /** diff --git a/lib-llama-android/src/main/cpp/CMakeLists.txt b/lib-llama-android/src/main/cpp/CMakeLists.txt index 034a699..5cd4e68 100644 --- a/lib-llama-android/src/main/cpp/CMakeLists.txt +++ b/lib-llama-android/src/main/cpp/CMakeLists.txt @@ -45,12 +45,15 @@ target_compile_definitions(${CMAKE_PROJECT_NAME} PRIVATE target_include_directories(${CMAKE_PROJECT_NAME} PRIVATE ${LLAMA_SRC} ${LLAMA_SRC}/common + ${LLAMA_SRC}/vendor ${LLAMA_SRC}/include ${LLAMA_SRC}/ggml/include ${LLAMA_SRC}/ggml/src) target_link_libraries(${CMAKE_PROJECT_NAME} llama - common + # The common utils library was renamed from `common` to `llama-common` + # upstream (llama.cpp ~b9xxx); it transitively pulls in llama-common-base. + llama-common android log) diff --git a/lib-llama-android/src/main/cpp/ai_chat.cpp b/lib-llama-android/src/main/cpp/ai_chat.cpp index feb68d8..1cc4725 100644 --- a/lib-llama-android/src/main/cpp/ai_chat.cpp +++ b/lib-llama-android/src/main/cpp/ai_chat.cpp @@ -49,6 +49,13 @@ static common_sampler * g_sampler; // cannot rewind the recurrent memory state to an earlier position. static bool g_model_has_mrope = false; +// Whether the loaded model's chat template should emit extended thinking tokens. +// Set to false for models whose template supports enable_thinking (currently Gemma 4) +// so the Jinja template omits <|think|> from the system turn and automatically +// appends the thinking-suppression prefix in the generation prompt. +// For all other models the parameter is a no-op in their template. +static bool g_model_enable_thinking = true; + extern "C" JNIEXPORT void JNICALL Java_com_arm_aichat_internal_InferenceEngineImpl_init(JNIEnv *env, jobject /*unused*/, jstring nativeLibDir) { @@ -176,6 +183,15 @@ Java_com_arm_aichat_internal_InferenceEngineImpl_prepare(JNIEnv * /*env*/, jobje g_model_has_mrope = (rope_type == LLAMA_ROPE_TYPE_MROPE || rope_type == LLAMA_ROPE_TYPE_IMROPE); LOGi("prepare: rope_type=%d has_mrope=%s", (int) rope_type, g_model_has_mrope ? "true" : "false"); + // Detect models whose chat template supports enable_thinking (currently Gemma 4). + // For these models we disable thinking via enable_thinking=false so the Jinja template + // suppresses reasoning tokens automatically, keeping the full output budget for JSON. + char arch_buf[32] = {}; + const int arch_len = llama_model_meta_val_str(g_model, "general.architecture", arch_buf, sizeof(arch_buf)); + const std::string arch_str = (arch_len > 0) ? std::string(arch_buf) : ""; + g_model_enable_thinking = (arch_str != "gemma4"); + LOGi("prepare: arch=%s enable_thinking=%s", arch_buf, g_model_enable_thinking ? "true" : "false"); + return 0; } @@ -346,11 +362,33 @@ static void shift_context() { } static std::string chat_add_and_format(const std::string &role, const std::string &content) { - common_chat_msg new_msg; - new_msg.role = role; - new_msg.content = content; - auto formatted = common_chat_format_single( - g_chat_templates.get(), chat_msgs, new_msg, role == ROLE_USER, /* use_jinja */ false); + common_chat_msg new_msg{role, content}; + + // Use common_chat_templates_apply directly (rather than common_chat_format_single) so we + // can pass enable_thinking. For most models the parameter is a no-op in their template; + // for Gemma 4 it makes the Jinja template omit <|think|> from the system turn and + // automatically append the thinking-suppression prefix in the generation prompt. + common_chat_templates_inputs inputs; + inputs.use_jinja = true; + inputs.enable_thinking = g_model_enable_thinking; + + // Compute the already-formatted prefix to extract only the new message diff. + std::string fmt_past; + if (!chat_msgs.empty()) { + inputs.messages = chat_msgs; + inputs.add_generation_prompt = false; + fmt_past = common_chat_templates_apply(g_chat_templates.get(), inputs).prompt; + } + + inputs.messages = chat_msgs; + inputs.messages.push_back(new_msg); + inputs.add_generation_prompt = (role == ROLE_USER); + const auto fmt_new = common_chat_templates_apply(g_chat_templates.get(), inputs).prompt; + + const auto diff = fmt_new.substr(fmt_past.size()); + // Preserve leading newline when past text ends with '\n' (mirrors common_chat_format_single). + const auto formatted = (!fmt_past.empty() && fmt_past.back() == '\n') ? "\n" + diff : diff; + chat_msgs.push_back(new_msg); LOGi("%s: Formatted and added %s message: \n%s\n", __func__, role.c_str(), formatted.c_str()); return formatted; @@ -551,12 +589,8 @@ Java_com_arm_aichat_internal_InferenceEngineImpl_processUserPrompt( // support / default to thinking mode. Standard attention-only models // (Gemma3 → NEOX=2, Llama → NORM=0) are unaffected. // - // TODO / future consideration: if a future model uses M-RoPE but does NOT - // have thinking mode (or if thinking mode becomes desirable for long-form - // analysis tasks where quality matters more than token budget), this block - // should be gated on a separate g_model_has_thinking flag detected at - // prepare() time rather than reusing g_model_has_mrope. For now the two - // properties are perfectly correlated for every model we support. + // Note: Gemma 4 also has thinking mode but uses a different token format + // (<|channel>thought ... ) and is handled in the block below. // // Upstream reference: llama.cpp --reasoning off / enable_thinking=false in // the Jinja chat template (models/templates/Qwen-Qwen3-0.6B.jinja:82-84). @@ -568,6 +602,11 @@ Java_com_arm_aichat_internal_InferenceEngineImpl_processUserPrompt( __func__); } + // For models with enable_thinking support (e.g. Gemma 4) the suppression is handled + // transparently in chat_add_and_format via g_model_enable_thinking=false, which + // makes the Jinja template append the thinking-suppression prefix automatically. + // No additional step is needed here. + // Decode formatted user prompts auto user_tokens = common_tokenize(g_context, formatted_user_prompt, has_chat_template, has_chat_template); for (auto id: user_tokens) { diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-base.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-base.so index 9c36ebe..1a81ae1 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-base.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-base.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:16ab986b45411ec94337947b26f3ab8b73d1b79c0dc2471e9923a27dc100ded1 -size 6976240 +oid sha256:c094f42bb4f086c026bf45f0d0e32cb708c5baafaa07fbe114f058eb12052edd +size 6978512 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.0_1.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.0_1.so index d8dd1e4..a68cb89 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.0_1.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.0_1.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:30d603bd4fdf90318c6704c2171de244c36edb9c8b4d80f5937d0fd0cb8f8301 -size 7741608 +oid sha256:a903be8fc4644d8e1d44357e03896a543ea8b5515c4fe156627552c49f5150e1 +size 7751768 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_1.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_1.so index e42dedb..df329cc 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_1.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_1.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:8e3689bc616a1bd1a82f16901752abf047e4cb24038df4b925b5389020e45ae1 -size 8062288 +oid sha256:2300e34bdd07ccf3e0bc2fdc05b3c45372c1662d24dabb4695f089308be866fc +size 8072256 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_2.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_2.so index 666ec3e..258c7ac 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_2.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.2_2.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:782f850434771bab86258b2dc7d296155746f46fe83f75b3952fd0fb2e40582e -size 8062528 +oid sha256:d5802212d18b0852f3ff9625e1151fd572ebe8046b6dcb0212832f74264a991f +size 8072520 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.6_1.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.6_1.so index 7e26059..dbdc523 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.6_1.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv8.6_1.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:d9309193af4b5602fa6d379aa78f9f9f365ca948ec7bc13e79180b2507e5b863 -size 8209544 +oid sha256:796f723f57a215155be113c187a1cfadd481764b59727b07bbdb0180df45ba24 +size 8219696 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.0_1.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.0_1.so index 437385c..cbbb580 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.0_1.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.0_1.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:70cd960ee4fe1854c6cca8b46962f17960adb05523afa10b399ba4022755821e -size 8314856 +oid sha256:655240b1b96b306c29d798af7c3c4bddea3d39273a470818a45440ade93c68fc +size 8325296 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_1.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_1.so index f3f7dcc..28966d4 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_1.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_1.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:63f58a4b4f5367deffa671c2d5a73690c978a82a6dd453f6c6e3f7bc20f9abf8 -size 8444488 +oid sha256:7968f97f16ef6d93a3fe87aa2398c7556a6c64d363adc78fdd4f362f1c79a756 +size 8454824 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_2.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_2.so index fba0dcd..57214fe 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_2.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml-cpu-android_armv9.2_2.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:9f6eefe932e4bf9b66f934d243fe934524f6fdc511c98225e4047bdae31bbcb8 -size 8444512 +oid sha256:482d329b970fd2813d314f7d7808e87e3b233ba6551fc10216dbbb4252fd1b66 +size 8454864 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml.so index 72c862f..b5e19f3 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libggml.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:c32b0ad54c7ce2ded6b79ae4fd2a1698204a8c56b6b45818fbef97ff3b642cf7 +oid sha256:0565045a23fe71d6b526aed27412cfabc798f60779e67677bc3b41d1f7038d46 size 1729496 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libkleidiai.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libkleidiai.so index 234fd71..e86de97 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libkleidiai.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libkleidiai.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:a197970e768b5f99279dc5cdbe9bcdd97b8d1626927ee29857d6a3e87f52c5da -size 1921976 +oid sha256:c0c59c7d365d0b196123ec1d983bcc79c5cb21fccd2e0ffa2dd1b22802d2e3b8 +size 2129240 diff --git a/lib-llama-android/src/main/jniLibs/arm64-v8a/libllama.so b/lib-llama-android/src/main/jniLibs/arm64-v8a/libllama.so index 2e3c305..62df76f 100644 --- a/lib-llama-android/src/main/jniLibs/arm64-v8a/libllama.so +++ b/lib-llama-android/src/main/jniLibs/arm64-v8a/libllama.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:34c3a24bd59271bdddc098716877b86dfb005908b55da14cb094860f0789ae61 -size 34988376 +oid sha256:aa9a7fa378d22de5f7efaa2593eb8fd78f65a473272a18727c159ab8da0d5318 +size 35588960 diff --git a/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineRecognizer.kt b/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineRecognizer.kt index 826b61c..1488768 100644 --- a/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineRecognizer.kt +++ b/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineRecognizer.kt @@ -50,6 +50,10 @@ data class OfflineMedAsrCtcModelConfig( var model: String = "", ) +data class OfflineFireRedAsrCtcModelConfig( + var model: String = "", +) + data class OfflineFunAsrNanoModelConfig( var encoderAdaptor: String = "", var llm: String = "", @@ -66,6 +70,19 @@ data class OfflineFunAsrNanoModelConfig( var hotwords: String = "", ) +data class OfflineQwen3AsrModelConfig( + var convFrontend: String = "", + var encoder: String = "", + var decoder: String = "", + var tokenizer: String = "", + var maxTotalLen: Int = 512, + var maxNewTokens: Int = 128, + var temperature: Float = 1e-6f, + var topP: Float = 0.8f, + var seed: Int = 42, + var hotwords: String = "", +) + data class OfflineWhisperModelConfig( var encoder: String = "", var decoder: String = "", @@ -84,11 +101,23 @@ data class OfflineCanaryModelConfig( var usePnc: Boolean = true, ) +data class OfflineCohereTranscribeModelConfig( + var encoder: String = "", + var decoder: String = "", + var language: String = "", + var usePunct: Boolean = true, + var useItn: Boolean = true, +) + data class OfflineFireRedAsrModelConfig( var encoder: String = "", var decoder: String = "", ) +// For moonshine v1, you need four models. +// For moonshine v2, you need two models. +// - v1: preprocessor, encoder, uncachedDecoder, cachedDecoder +// - v2: encoder, mergedDecoder data class OfflineMoonshineModelConfig( var preprocessor: String = "", var encoder: String = "", @@ -97,10 +126,6 @@ data class OfflineMoonshineModelConfig( var mergedDecoder: String = "", ) -data class OfflineFireRedAsrCtcModelConfig( - var model: String = "", -) - data class OfflineSenseVoiceModelConfig( var model: String = "", var language: String = "", @@ -122,8 +147,11 @@ data class OfflineModelConfig( var omnilingual: OfflineOmnilingualAsrCtcModelConfig = OfflineOmnilingualAsrCtcModelConfig(), var medasr: OfflineMedAsrCtcModelConfig = OfflineMedAsrCtcModelConfig(), var funasrNano: OfflineFunAsrNanoModelConfig = OfflineFunAsrNanoModelConfig(), + var qwen3Asr: OfflineQwen3AsrModelConfig = OfflineQwen3AsrModelConfig(), var fireRedAsrCtc: OfflineFireRedAsrCtcModelConfig = OfflineFireRedAsrCtcModelConfig(), var canary: OfflineCanaryModelConfig = OfflineCanaryModelConfig(), + var cohereTranscribe: OfflineCohereTranscribeModelConfig = + OfflineCohereTranscribeModelConfig(), var teleSpeech: String = "", var numThreads: Int = 1, var debug: Boolean = false, @@ -176,6 +204,11 @@ class OfflineRecognizer( return OfflineStream(p) } + fun createStream(hotwords: String): OfflineStream { + val p = createStreamWithHotwords(ptr, hotwords) + return OfflineStream(p) + } + fun getResult(stream: OfflineStream): OfflineRecognizerResult { return getResult(stream.ptr) } @@ -188,6 +221,8 @@ class OfflineRecognizer( private external fun createStream(ptr: Long): Long + private external fun createStreamWithHotwords(ptr: Long, hotwords: String): Long + private external fun setConfig(ptr: Long, config: OfflineRecognizerConfig) private external fun newFromAsset( @@ -787,6 +822,187 @@ fun getOfflineModelConfig(type: Int): OfflineModelConfig? { tokenizer = "$modelDir/Qwen3-0.6B", ), tokens = "", + numThreads=3, + ) + } + + 47 -> { + val modelDir = "sherpa-onnx-wenetspeech-wu-u2pp-conformer-ctc-zh-int8-2026-02-03" + return OfflineModelConfig( + wenetCtc = OfflineWenetCtcModelConfig( + model = "$modelDir/model.int8.onnx", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 48 -> { + val modelDir = "sherpa-onnx-wenetspeech-wu-u2pp-conformer-ctc-zh-2026-02-03" + return OfflineModelConfig( + wenetCtc = OfflineWenetCtcModelConfig( + model = "$modelDir/model.onnx", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 49 -> { + val modelDir = "sherpa-onnx-zipformer-vi-30M-int8-2026-02-09" + return OfflineModelConfig( + transducer = OfflineTransducerModelConfig( + encoder = "$modelDir/encoder.int8.onnx", + decoder = "$modelDir/decoder.onnx", + joiner = "$modelDir/joiner.int8.onnx", + ), + tokens = "$modelDir/tokens.txt", + modelType = "transducer", + ) + } + + 50 -> { + val modelDir = "sherpa-onnx-fire-red-asr2-ctc-zh_en-int8-2026-02-25" + return OfflineModelConfig( + fireRedAsrCtc = OfflineFireRedAsrCtcModelConfig( + model = "$modelDir/model.int8.onnx", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 51 -> { + val modelDir = "sherpa-onnx-moonshine-tiny-ko-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 52 -> { + val modelDir = "sherpa-onnx-moonshine-tiny-ja-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 53 -> { + val modelDir = "sherpa-onnx-moonshine-tiny-en-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 54 -> { + val modelDir = "sherpa-onnx-moonshine-base-zh-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 55 -> { + val modelDir = "sherpa-onnx-moonshine-base-vi-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 56 -> { + val modelDir = "sherpa-onnx-moonshine-base-uk-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 57 -> { + val modelDir = "sherpa-onnx-moonshine-base-ja-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 58 -> { + val modelDir = "sherpa-onnx-moonshine-base-es-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 59 -> { + val modelDir = "sherpa-onnx-moonshine-base-en-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 60 -> { + val modelDir = "sherpa-onnx-moonshine-base-ar-quantized-2026-02-27" + return OfflineModelConfig( + moonshine = OfflineMoonshineModelConfig( + encoder = "$modelDir/encoder_model.ort", + mergedDecoder = "$modelDir/decoder_model_merged.ort", + ), + tokens = "$modelDir/tokens.txt", + ) + } + + 61 -> { + val modelDir = "sherpa-onnx-qwen3-asr-0.6B-int8-2026-03-25" + return OfflineModelConfig( + qwen3Asr = OfflineQwen3AsrModelConfig( + convFrontend = "$modelDir/conv_frontend.onnx", + encoder = "$modelDir/encoder.int8.onnx", + decoder = "$modelDir/decoder.int8.onnx", + tokenizer = "$modelDir/tokenizer", + ), + tokens = "", + numThreads=3, + ) + } + + 62 -> { + val modelDir = "sherpa-onnx-nemo-parakeet-unified-en-0.6b-int8-non-streaming" + return OfflineModelConfig( + transducer = OfflineTransducerModelConfig( + encoder = "$modelDir/encoder.int8.onnx", + decoder = "$modelDir/decoder.int8.onnx", + joiner = "$modelDir/joiner.int8.onnx", + ), + tokens = "$modelDir/tokens.txt", + modelType = "nemo_transducer", ) } diff --git a/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineStream.kt b/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineStream.kt index ab316c5..96393a7 100644 --- a/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineStream.kt +++ b/lib-sherpa-onnx/src/main/java/com/k2fsa/sherpa/onnx/OfflineStream.kt @@ -4,6 +4,10 @@ class OfflineStream(var ptr: Long) { fun acceptWaveform(samples: FloatArray, sampleRate: Int) = acceptWaveform(ptr, samples, sampleRate) + fun setOption(key: String, value: String) = setOption(ptr, key, value) + + fun getOption(key: String): String = getOption(ptr, key) + protected fun finalize() { if (ptr != 0L) { delete(ptr) @@ -22,6 +26,8 @@ class OfflineStream(var ptr: Long) { } private external fun acceptWaveform(ptr: Long, samples: FloatArray, sampleRate: Int) + private external fun setOption(ptr: Long, key: String, value: String) + private external fun getOption(ptr: Long, key: String): String private external fun delete(ptr: Long) companion object { diff --git a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libonnxruntime.so b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libonnxruntime.so index 9b1bab7..b73c766 100644 --- a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libonnxruntime.so +++ b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libonnxruntime.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:c7c0e6b0f9c364a4df4bfc9a3a3576b691899e96f4d9beb60f3b3a7ce4bf572c -size 15988232 +oid sha256:4d2318b3849abb8862133d3068fc7e807ed8b2671cc6d83657fff2fcb9e1caad +size 25831632 diff --git a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so index 51befae..f0e2bc7 100644 --- a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so +++ b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-c-api.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:136c16ef29ba19d8681a811027c7d7bdc3a427e2df156e00e3cf624c133149cb -size 4705424 +oid sha256:ec09f16e2e5043898d4dd18ec5c316310a39b7ca74ce4db0a33a3b945790e2ab +size 4327992 diff --git a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so index 3716399..e5fe060 100644 --- a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so +++ b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-cxx-api.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:659d75245e369c79bd2ec03614ef4a15bf6043e51d1d0484ec4e581aee538bcf -size 410976 +oid sha256:e330684b081ded3d3d694df44abce6de313ccee82e119121d2ea342e5ce0e3d2 +size 419800 diff --git a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so index da28dd6..d6b73c6 100644 --- a/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so +++ b/lib-sherpa-onnx/src/main/jniLibs/arm64-v8a/libsherpa-onnx-jni.so @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:5d120ad84111b2799b6425369b847ca1bea0a9b911659d26eeef2244e66b7267 -size 5008104 +oid sha256:fc072f201dc1923ee98b594eb61c796b538ef087f7f18d08dcfdf0565167a8bd +size 4623192