Skip to content

android: one window per host, GL fallback, backend pinning and IME fixes - #26

Merged
tuyc merged 4 commits into
mainfrom
tuyc/android-multi-window
Oct 9, 2026
Merged

tuyc merged 4 commits into
mainfrom
tuyc/android-multi-window

Conversation

@tuyc

@tuyc tuyc commented Oct 8, 2026 •

Copy link
Copy Markdown

Fixes and features found while embedding gpui-mobile in a production Android app (an AI chat page hosted in a regular Activity, stacked any number of times: page → AI → page → AI). Four commits, one per problem; each builds on its own.

1. One GPUI window per host, not one per process (b9faada)

Platform::open_window always answered with the primary window, so every host after the first rendered into the first host's surface. Hosts are now keyed by a HostId: each gets its own AndroidWindow, kept across that host's surface teardowns so a returning Activity finds its own window. The window is named via a pending slot before GPUI's open_window runs; on_open_window receives the HostId; Resumed/Paused act on their own host only (stacking Activities interleaves A.onPause → B.onResume → A.onStop, so a global resume woke A back up).

2. Load on API < 29 (d1d67bd)

ndk_sys::android_get_device_api_level() only exists in libc from API 29, and Rust has no __INTRODUCED_IN guard, so the .so failed to dlopen on Android 8.x (reproduced on an OPPO R11s). Read ro.build.version.sdk instead. Verified by diffing the .so's strong undefined symbols against the NDK API 26 stubs: all 231 now resolve.

3. Fall back to GL when Vulkan cannot build our pipelines (e870c46)

wgpu picks an adapter by creating a device and configuring a surface, which says nothing about shader compilation. An Adreno 512 (Android 8.1) passes that probe, then loses the device while WgpuRenderer::new builds its pipelines — every frame after that was black, and re-attaching after Home panicked. A GL without unaligned buffer bindings rejects path_rasterization, which reached wgpu's default error handler and panicked the render thread.

  • Fresh contexts are built on an explicit backend set and the first renderer is created inside Validation/OutOfMemory/Internal error scopes; any error or device loss drops the context and tries GL next, and if GL fails too open_window fails cleanly.
  • host::set_gpu_backend(GpuBackend::Gl) lets a host skip Vulkan once it knows the device needs GL (R11s: 689 ms → 68 ms); host::on_gpu_outcome reports Vulkan / Gl / GlAfterVulkanFailed, and Failed once per host whose window cannot be opened or re-attached (re-attach failures used to only log, leaving a black page).
  • Failed hosts are remembered until destroyed and their later surfaces ignored (surfaceChanged follows surfaceCreated, so every failure ran twice).
  • Stop forcing Mailbox when replacing a surface: replace_surface doesn't check the caps (unlike WgpuRenderer::new) and GL only offers Fifo.

4. Hide only the keyboard on back (91ca188)

Back used to send IME event 4, which drain() turns into Escape — hiding the keyboard over a search overlay also closed it and dropped the query. Back now sends a new event 5: composition finished, keyboard remembered, no Escape. GPUI only requests the keyboard on focus gain and the platform cannot blur, so a tap on the focused field's row (judged by its caret) brings the keyboard back, as on a native EditText. Event 4 (IME "done") is unchanged; GpuiInputActivity sends 5 on back.

Compatibility

  • Android-only code paths; iOS and desktop are untouched.
  • on_open_window now takes Fn(HostId, &mut App), and surface_created / surface_destroyed / resumed / paused take a HostId — hosts using android::host need a one-line change.
  • Hosts with their own IME proxy keep working: event 5 is new and optional.

Testing

  • Rebased on main (gpui-pre 0.3.7). Locally: cargo check --all-targets, cargo test --lib (46 passed), cargo fmt --all -- --check, cargo check --target aarch64-apple-ios / aarch64-apple-ios-sim, and Android aarch64 checks of the crate and the example. Each of the four commits builds for Android and passes cargo fmt --check on its own.
  • On devices, through the host app: OPPO R11s (Android 8.1, Adreno 512) — Vulkan fails, GL is used, the chat renders, Home/return cycles and new pages render; second launch goes straight to GL. Emulator (API 35/36): multi-window isolation, back/keyboard sequence on the history search, forced GL failure reported once.

About the iOS CI failures

Not from this branch. Cargo.lock is not tracked, so CI resolves libc 0.2.190 (released 2026-10-02), whose _dyld_* functions no longer resolve for aarch64-apple-ios with backtrace 0.3.76. A clean main fails the same way with libc 0.2.190 and passes with 0.2.189; main's last green run was 2026-09-29, before that release.

🤖 Generated with Claude Code

涂阳川 and others added 4 commits October 8, 2026 17:53
The host path opened a single window and re-attached every incoming surface to
it. That is right for one Activity, and wrong the moment a host stacks two: both
Activities' `SurfaceView`s end up driving the same panel, so walking back to the
first shows whatever the second left behind. A host that wants
ordinary → AI → ordinary → AI cannot build it on that.

Surfaces now arrive tagged with a `HostId` the host assigns per Activity, and
each distinct id gets a window of its own. A returning id re-attaches to the
window it left; a new one opens another. `host_destroyed` drops a window when
its Activity finishes for good.

Several things made this more than bookkeeping:

`Platform::open_window` answered every request with `primary`, since GPUI asks
for "a window" with no way to name a surface. A second `cx.open_window` would
have rendered the new panel into the first host's surface. The platform now
carries a pending slot the host sets just before asking GPUI to open.

The one-shot `launch` closure could build exactly one window. UI construction
moves to `on_open_window(Fn(HostId, &mut App))`, called once per host with the
id passed in rather than read back out of a global; `launch` keeps the
process-wide setup it always did. Later windows go through
`ApplicationHandle::update`.

`Resumed` / `Paused` act on their own host only. Stacking two Activities
interleaves as `A.onPause -> B.onResume -> A.onStop`, so a global resume woke A
back up right after it paused, purely because B's surface had not arrived yet to
take over as attached.

Input and drawing follow the *attached* window rather than `primary`. Android
tears down a covered Activity's surface, so at most one host is attached at a
time — which is also why the memory a second window adds is a renderer and an
atlas, not a swapchain.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`ndk_sys::android_get_device_api_level()` only entered libc in Android 10 (API
29). Rust has no counterpart to the NDK headers' `__INTRODUCED_IN` guards, so
building against a lower platform still emits a *strong* undefined reference
rather than falling back to the header's inline implementation. Every consumer
of this crate therefore failed to `dlopen` below API 29 — reproduced on an OPPO
R11s (Android 8.1): `cannot locate symbol "android_get_device_api_level"`. The
only caller is the high-refresh-rate path, whose whole purpose is to degrade
gracefully on old devices.

Read `ro.build.version.sdk` instead: available since API 1 and needs no new
symbol. Verified against the NDK's API 26 stub libraries — all 231 strong
undefined symbols now resolve, where before exactly one did not.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
wgpu picks an adapter by creating a device and configuring a surface, which
says nothing about whether the driver can build our pipelines — and on Android
it often cannot. An Adreno 512 (Android 8.1, OPPO R11s) passes that probe, then
logs `Shader compilation failed` while `WgpuRenderer::new` builds its pipelines
and loses the device: every frame after that was black, and re-attaching the
window after Home panicked on the dead device. A GL without unaligned buffer
bindings rejects `path_rasterization` the same way, except that the validation
error reaches wgpu's default handler and panics the render thread, because
`WgpuRenderer::new` builds its pipelines before installing its own handler.

Fresh contexts are now built by us on an explicit backend set, and the first
renderer is created inside Validation / OutOfMemory / Internal error scopes. Any
scope error or device loss counts as a failed attempt: the context is dropped,
releasing the window, and GL is tried next; if GL fails too, `open_window`
fails cleanly instead of panicking.

The host can remember the answer instead of failing on Vulkan every launch
(689 ms on the R11s, against 68 ms going straight to GL):

- `host::set_gpu_backend(GpuBackend::Gl)` skips Vulkan for the next fresh
  context.
- `host::on_gpu_outcome` reports once per fresh context what it ended up on —
  `Vulkan`, `Gl` or `GlAfterVulkanFailed` — and `Failed` once per host whose
  window cannot be opened, or cannot be re-attached when the host comes back.
  Re-attach failures used to only log, leaving the page black with nothing
  telling the host to switch away.
- Failed hosts are remembered until destroyed and their later surfaces ignored:
  `surfaceChanged` follows `surfaceCreated`, so every failure ran — and was
  reported — twice.

Also stop forcing `Mailbox` when replacing a surface. `replace_surface` applies
a preferred present mode without checking the surface caps (unlike
`WgpuRenderer::new`), and GL only offers `Fifo`: the swap left the surface
unconfigured and the next frame panicked in `get_current_texture_view`. `None`
keeps the mode negotiated when the renderer was created.

Verified on the R11s: Vulkan fails, GL is used, the chat renders; three
Home/return cycles and new pages render without errors; the second launch goes
straight to GL. On the emulator with GL forced, the rejected pipeline reports
`Failed` once instead of panicking.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
IME back used to send event 4, which `drain()` turns into an Escape keystroke,
so hiding the keyboard over a search overlay also closed the overlay and dropped
its query. Back now sends a new event 5: the composition is finished and the
dismissed keyboard remembered, with no Escape. Event 4 (the IME's "done") is
unchanged, so hosts with their own proxy keep working.

GPUI asks for the keyboard only when a text input *gains* focus, and the
platform cannot blur it, so after a back-press the field stays focused with no
keyboard and tapping it again would change nothing. A tap landing on the
focused field's row — judged by its caret, since the input handler exposes text
geometry rather than the field's frame — now brings the keyboard back, as on a
native `EditText`. Drags and taps elsewhere do not. `GpuiInputActivity` sends
event 5 on back.

Verified on the emulator: back hides only the keyboard (overlay and query
kept), tapping the field shows it again, then back closes the overlay and back
exits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@tuyc
tuyc force-pushed the tuyc/android-multi-window branch from c55e043 to 91ca188 Compare October 8, 2026 09:54
@tuyc
tuyc merged commit 448b250 into main Oct 9, 2026
3 of 5 checks passed
@tuyc
tuyc deleted the tuyc/android-multi-window branch October 9, 2026 02:09
yuanzhuang-svg added a commit to yuanzhuang-svg/gpui-mobile that referenced this pull request Oct 9, 2026
libc 0.2.190 dropped the iOS `_dyld_*` symbol declarations that
`backtrace`'s symbolize/gimli/libs_macos.rs relies on, so a fresh
resolution breaks `cargo check --target aarch64-apple-ios[-sim]`
(E0425: cannot find `_dyld_image_header` in `libc`). This already
reddened main (longbridge#26); pin to the last good release until a fixed
libc ships.
madcodelife pushed a commit that referenced this pull request Oct 9, 2026
Two commits:

### 1. Bump gpui-pre to 0.3.8
Bumps the `gpui-pre` family (`gpui-pre`, `gpui-pre-wgpu`) from `=0.3.7`
to `=0.3.8`, following the previous bumps (#18 → 0.3.6, #21 → 0.3.7).

**Why:** `gpui-kit` has advanced to `0.7.1`, which requires `gpui-pre
0.3.8`. Downstream consumers that pull **both** `gpui-mobile` and
`gpui-kit` (e.g. ai-chat's `examples/ios` iOS static lib) currently
can't resolve a consistent set: `gpui-kit 0.7.1` wants `gpui-pre 0.3.8`
while `gpui-mobile` pins `=0.3.7`, so the iOS build fails (`cannot find
RoundedFrameCover in gpui_kit::base`, an API that only exists on the
0.7.1 line).

### 2. Pin libc to 0.2.189
`libc 0.2.190` dropped the iOS `_dyld_*` symbol declarations that
`backtrace`'s `symbolize/gimli/libs_macos.rs` relies on, so a fresh
dependency resolution breaks `cargo check --target
aarch64-apple-ios[-sim]` (`E0425: cannot find _dyld_image_header in
libc`). This already reddened `main` (#26) independently of this bump;
pinning to the last good release keeps the apple-ios CI green until a
fixed libc (>0.2.190) ships.

### Verification
- `gpui-pre` / `gpui-pre-wgpu` / `gpui-pre-reqwest-client` `0.3.8` are
published on crates.io.
- Local `cargo check` for `aarch64-apple-ios` and
`aarch64-apple-ios-sim` both finish clean with this branch.
- With this branch patched in, ai-chat's `examples/ios` resolves
`gpui-kit 0.7.1` + `gpui-pre 0.3.8` + `libc 0.2.189` and `cargo check
--target aarch64-apple-ios-sim` finishes with no errors.

---------

Co-authored-by: yuanzhuang-svg <273899003+yuanzhuang-svg@users.noreply.github.com>
Bombatomica64 added a commit to Bombatomica64/gpui-mobile that referenced this pull request Oct 9, 2026
Since longbridge#26 each host Activity has its own window, and going back re-attaches
an earlier one. The bridge kept the last window's callbacks and view, so the
returning window got no screen reader. Each window now keeps its callbacks,
takes the adapter over when it becomes active, and the window it replaces
is told to stop building trees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bombatomica64 added a commit to Bombatomica64/gpui-mobile that referenced this pull request Oct 12, 2026
Since longbridge#26 each host Activity has its own window, and going back re-attaches
an earlier one. The bridge kept the last window's callbacks and view, so the
returning window got no screen reader. Each window now keeps its callbacks,
takes the adapter over when it becomes active, and the window it replaces
is told to stop building trees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant