From 569f4be688574a5ce6e8b38eeccbe4ce65c72b49 Mon Sep 17 00:00:00 2001 From: Elie Gambache Date: Mon, 19 Oct 2026 21:22:54 +0300 Subject: [PATCH] Benchmarks: a real app's Lucene search on each runtime Signed-off-by: Elie Gambache --- BENCHMARKS.md | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++- README.md | 4 ++- 2 files changed, 87 insertions(+), 2 deletions(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 5c35162..44ce0b5 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -1,6 +1,13 @@ # Benchmarks -Measured on 2026-10-16, on an Apple Silicon Mac (10 cores): +Two sets of measurements: + +- [Java libraries](#java-libraries) and [the bridge](#the-bridge): NJA's own examples, on jdk-mobile's JVM (2026-10-16). +- [A real app](#a-real-app-lucene-search-in-zayit): Zayit's search, a Lucene 10 index of 7M documents, on each runtime + NJA offers: jdk-mobile's JVM, a GraalVM native image, and ART, compared with Lucene patched to run on ART + (2026-10-17). + +Measured on an Apple Silicon Mac (10 cores): - **Android**: the `zayit35` emulator (API 35, arm64, 4 vCPUs, hypervisor-backed), with the release, non-debuggable app `examples/benchmark-app`. That means ART's JIT is not slowed down by debugging. @@ -56,6 +63,82 @@ Each call is measured from the app, round trip included. extra thread. Before this optimization: 4.9k calls/s on Android and 13k on iOS. A call that actually suspends goes through the events: the JVM's coroutine, then the app's polling thread. +## A real app: Lucene search in Zayit + +[Zayit](https://github.com/kdroidFilter/Zayit)'s search runs Lucene 10 through NJA: its façade is SeforimLibrary's +`search-facade`. The index holds 7M documents (Hebrew texts, with int8 384-dimension vectors for semantic search). The +whole search is measured: Lucene through NJA, plus the app's Kotlin around it (SQLite, snippets, highlights, and the ONNX +model for hybrid search). + +- **Android**: the release benchmark app on the `zayit35` emulator (Android 15, arm64). Each value is the median of 3 + runs, with the page cache dropped before each. The GraalVM image was built on Linux, the APK on macOS. +- **iOS**: the release test executable in the iPhone 17 Pro simulator. + +"Hybrid" is lexical and semantic search fused; "lexical" is lexical only. ART here is the app's former approach, an +older Lucene patched to run on ART; NJA's `ART` runtime can't run Lucene 10 (it needs Java 21). + +### Android + +| | Lucene patched on ART | NJA `JVM` (HotSpot) | NJA `GRAALVM` | +|---|---:|---:|---:| +| First search, cold (runtime start included), hybrid | **1.41 s** | 2.11 s | 1.51 s | +| First search, cold, lexical | 0.92 s | 1.55 s | **0.91 s** | +| 5 warm rounds, total, hybrid | 5.82 s | **5.27 s** | 6.01 s | +| 5 warm rounds, total, lexical | 7.29 s | **5.75 s** | 7.78 s | +| Search, first page, hybrid | 83 ms | **75 ms** | 85 ms | +| Search, first page, lexical | 75 ms | **59 ms** | 79 ms | +| Search, next page, lexical | 75 ms | **58 ms** | 76 ms | +| Facets, hybrid | 8.0 ms | 8.1 ms | 7.8 ms | +| Peak memory (PSS), hybrid | **426 MB** | 559 MB | 481 MB | +| Peak memory (PSS), lexical | 337 MB | 332 MB | **329 MB** | +| App per arm64 device (installed) | **42.7 MB** | ~98 MB | 58.1 MB | +| Of which Lucene's runtime | ~3 MB (dex) | ~58 MB (HotSpot, `java.home`) | ~19 MB (`libnja.so`, ~7 MB compressed) | + +- **GraalVM** starts as fast as ART, with about the same speed and memory, and runs the real Lucene 10. +- **HotSpot** is the fastest once warm (C2), but starts ~0.6 s later and is the heaviest. +- The index is read through NIO on Android: memory-mapped, its pages would count in the app's PSS (1.4 GB peak in + hybrid search), which Android's low-memory killer watches. + +### iOS + +HotSpot can't run on iOS (no JIT allowed): NJA's `JVM` runtime there is Zero, an interpreter. + +| | NJA `JVM` (Zero) | NJA `GRAALVM` | +|---|---:|---:| +| First search, cold (runtime start included), hybrid | 4.63 s | **0.82 s** | +| First search, cold, lexical | 4.15 s | **0.52 s** | +| Search, first page, hybrid | 1.18 s | **144 ms** | +| Search, first page, lexical | 1.11 s | **142 ms** | +| Search, next page, lexical | 1.11 s | **127 ms** | +| Facets | 148 ms | **18 ms** | +| Book suggestions | 67 ms | **48 ms** | +| Memory after warmup, hybrid (peak) | 386 MB (422) | 390 MB (430) | +| Memory after warmup, lexical (peak) | 148 MB (163) | **124 MB (155)** | +| Added to the app | ~50 MB (Zero, `java.home` bundled) | **~20 MB** (one object, ~7 MB compressed) | + +- **GraalVM** is about 8× faster than Zero and starts 5–8× sooner, at the same memory (the ONNX model dominates hybrid + search) and less than half the size. +- The search's results are byte-identical to the desktop's JVM, hybrid search included. + +### Lucene alone + +The same façade's searches, without the app around them, warm: + +| | HotSpot (desktop) | `GRAALVM` (iOS simulator) | `GRAALVM` (Android emulator) | +|---|---:|---:|---:| +| Phrase search | 1–2 ms | 1–2 ms | 1–3 ms | +| Facets | 1–3 ms | 1–4 ms | 1–4 ms | +| KNN (50 nearest), warm page cache | ~2 ms | ~1.5 ms | 3–10 ms | +| KNN, cold page cache | 130–250 ms | 130–250 ms | 130–500 ms | +| Memory | – | 61 MB | 63 MB (RSS) | + +A native image runs Lucene at HotSpot's speed. KNN depends on the page cache, not on the runtime. + +### Correctness + +The example's suites (2008 tests) pass on every runtime: `GRAALVM` on the iOS simulator and an Android x86_64 emulator +(CI), `JVM` and `ART` on an Android 15 emulator, and the JVM target. + ## Reproduce The runtimes are jdk-mobile's latest release, which the plugin downloads. diff --git a/README.md b/README.md index 593a77f..bf00709 100644 --- a/README.md +++ b/README.md @@ -147,7 +147,9 @@ cancellation storms, 200k-element flows, a 20 s chaos soak on 24 threads. See [BENCHMARKS.md](BENCHMARKS.md). In short, on Android NJA's HotSpot runs Jackson and Commons Math 1.8–7.8× faster than ART, and Lucene 10 (Java 21, out of ART's reach) at desktop speed; an empty call costs 0.33 µs. On iOS, Zero (an -interpreter) suits occasional calls, not CPU-bound work; an empty call costs 2.4 µs. +interpreter) suits occasional calls, not CPU-bound work; an empty call costs 2.4 µs. A GraalVM native image (`GRAALVM`) runs at HotSpot's +speed on both: on a real app's Lucene search, 8× faster than Zero on iOS, and on par with Lucene patched for ART on +Android, at about its memory. ## jdk-mobile