Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions CONTRIBUTING_IDEAS.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,15 @@ October 2026; "fork" means the fix is on one of the branches listed in
- Not started: flags and `❤️` (VS16) still don't render with the
bundled emoji font; flags come from the system's COLRv1
`NotoColorEmojiFlags.ttf`.
- Not started: safe-area insets. The plan is in
[docs/safe-area-insets.md](docs/safe-area-insets.md).
- Fork: safe-area insets. The host reports the system bars, the cutout
and the keyboard through `host::insets_changed` (gpui-mobile
`android-host-insets`). They become GPUI's `PlatformWindow::insets`, so
`Window::fully_visible_bounds()` works on Android. The lab is edge to
edge, its shell pads by them, Kit inputs reveal above the keyboard
(`input-reveal-on-focus`), and Diagnostics shows them. Upstream draft in
[docs/upstream/safe-area-insets.md](docs/upstream/safe-area-insets.md).
Next: Kit sheets and notifications still lay out against the whole
window, under the bars.
- Not started: system fonts.
8. **Upstream the fork fixes.** 22 branches in
[MOBILE_PATCHES.md](MOBILE_PATCHES.md) are still fork-only: 11 in
Expand Down
14 changes: 7 additions & 7 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ crate-type = ["cdylib"]
# the gpui-mobile fork; [patch.crates-io] swaps in 0.3.8 with our touch fixes. Kit and gpui-mobile come from our forks, pinned by rev
# (see MOBILE_PATCHES.md for what each carries on top of upstream).
gpui = { package = "gpui-pre", version = "=0.3.8", default-features = false }
gpui-kit = { git = "https://github.com/Bombatomica64/gpui-kit", rev = "d47dd0bde03345977db286f9bc8224bf1199cb60", default-features = false, features = ["component", "assets"] }
gpui-mobile = { package = "gpui-pre-mobile", git = "https://github.com/Bombatomica64/gpui-mobile", rev = "2456b8e91ca98f37cea00c8e6a1a3e1b0f8a2410", default-features = false, features = [
gpui-kit = { git = "https://github.com/Bombatomica64/gpui-kit", rev = "e8b28164ac7fa7e1f7d393d441beb738c286bbc8", default-features = false, features = ["component", "assets"] }
gpui-mobile = { package = "gpui-pre-mobile", git = "https://github.com/Bombatomica64/gpui-mobile", rev = "c71a31aaa12660552ca7a772f6fdd890d9c8cf7a", default-features = false, features = [
"clipboard", "share",
# Exercised by the Platform APIs screen; build.sh copies their Java helpers from this checkout.
"file_selector", "image_picker", "local_auth", "permission_handler", "calendar", "contacts",
"location", "notifications", "url_launcher", "maps_launcher", "audio", "media_session",
"microphone", "deeplink",
] }
# Kit's FPS / frame-time HUD (enables gpui's `profiler` frame trace).
gpui-fps = { git = "https://github.com/Bombatomica64/gpui-kit", rev = "d47dd0bde03345977db286f9bc8224bf1199cb60" }
gpui-fps = { git = "https://github.com/Bombatomica64/gpui-kit", rev = "e8b28164ac7fa7e1f7d393d441beb738c286bbc8" }
anyhow = "1"
log = "0.4"
# Already in the tree through gpui; the Platform APIs screen reads picked images' sizes.
Expand Down
20 changes: 18 additions & 2 deletions MOBILE_PATCHES.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,20 @@ Send then hide the keyboard. The purpose half needs the GPUI branch.
[amount keyboard](docs/demos/android-amount-keyboard.png),
[Search key, Enter](docs/demos/android-ime-actions.png).

### Let host-driven apps report safe-area and keyboard insets — branch `android-host-insets`

The host path had no way to pass insets in, so an edge-to-edge host could
not tell GPUI where the system bars, the cutout and the keyboard are: the
Android window returned GPUI's default, zero, `PlatformWindow::insets`.
`host::insets_changed(host, SafeAreaInsets, ime)` takes them in physical
pixels. The render thread stores them on that host's window, or keeps them
until the window opens. The window implements `insets()` and
`on_insets_changed`, so apps read them through GPUI
(`Window::fully_visible_bounds`) and GPUI refreshes the window when they
change. `safe_area_insets()` and `keyboard_height()` return them too, as on
iOS. LabActivity calls it from its `OnApplyWindowInsetsListener`.
Draft upstream PR: [docs/upstream/safe-area-insets.md](docs/upstream/safe-area-insets.md).

### Clear the Java exceptions the accessibility bridge leaves pending — branch `android-a11y-clear-exceptions`

`accesskit_android`'s jni 0.21 API leaves a thrown Java exception pending,
Expand Down Expand Up @@ -140,8 +154,10 @@ press on a focused input calls `Window::request_virtual_keyboard`.
When the IME opens, `adjustResize` shrinks the viewport, but nothing scrolled
the focused input back into view. A focused input calls
`Window::request_autoscroll` for its text area (the caret line in a
textarea) when it gains focus or the viewport size changes. Needs the GPUI
autoscroll fix below to work in plain scroll containers.
textarea) when it gains focus or `Window::fully_visible_bounds()` changes:
the viewport resizing under `adjustResize`, or the keyboard arriving as an
inset on an edge-to-edge window. Needs the GPUI autoscroll fix below to work
in plain scroll containers.

### Tell the keyboard how to assist each input — branches `input-text-input-configuration`, `input-purpose`

Expand Down
2 changes: 1 addition & 1 deletion android/app/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ android {
defaultConfig {
applicationId = "dev.gpui.mobile.lab"
minSdk = 26
// 34 keeps adjustResize semantics (35 forces edge-to-edge).
// 34: LabActivity goes edge-to-edge itself; 35 would force it.
targetSdk = 34
versionCode = 3
versionName = "0.1.2"
Expand Down
3 changes: 1 addition & 2 deletions android/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,13 @@
android:supportsRtl="true"
android:theme="@style/LabTheme">

<!-- Portrait only; configChanges keeps density/keyboard/uiMode changes from
<!-- configChanges keeps rotation/density/keyboard/uiMode changes from
recreating the Activity (recreation is still handled: see LabActivity). -->
<activity
android:name=".LabActivity"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|keyboardHidden|keyboard|navigation|uiMode|density"
android:exported="true"
android:launchMode="singleTask"
android:screenOrientation="portrait"
android:windowSoftInputMode="adjustResize|stateHidden">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
Expand Down
51 changes: 37 additions & 14 deletions android/app/src/main/java/dev/gpui/mobile/lab/LabActivity.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import android.app.Activity;
import android.content.pm.ApplicationInfo;
import android.graphics.Rect;
import android.os.Build;
import android.os.Bundle;
import android.text.Editable;
Expand All @@ -15,12 +14,17 @@
import android.view.SurfaceView;
import android.view.View;
import android.view.ViewGroup;
import android.view.WindowManager;
import android.view.inputmethod.BaseInputConnection;
import android.view.inputmethod.EditorInfo;
import android.view.inputmethod.InputConnection;
import android.view.inputmethod.InputConnectionWrapper;
import android.view.inputmethod.InputMethodManager;
import android.widget.EditText;
import androidx.core.graphics.Insets;
import androidx.core.view.ViewCompat;
import androidx.core.view.WindowCompat;
import androidx.core.view.WindowInsetsCompat;

import java.io.File;

Expand All @@ -29,8 +33,8 @@
*
* A plain Activity owning a SurfaceView: the Rust side keeps one process-lived
* render thread and GPUI App, so recreating this Activity only swaps the surface.
* The window uses adjustResize, so the SurfaceView (and GPUI's viewport) shrinks
* above the software keyboard.
* The SurfaceView fills the window, behind the system bars, display cutout and
* software keyboard; their insets go to Rust, and the app shell pads by them.
*
* The IME proxy is adapted from gpui-mobile's GpuiInputActivity: an invisible
* EditText whose InputConnection forwards composing/commit/delete to Rust.
Expand Down Expand Up @@ -77,19 +81,29 @@ protected void onCreate(Bundle state) {
surface.setOnTouchListener((view, event) -> forwardMotion(event));
setContentView(surface);

// The keyboard shrinks the content via adjustResize; report its height so the
// diagnostics screen can show it next to GPUI's own viewport.
final View root = getWindow().getDecorView();
root.getViewTreeObserver().addOnGlobalLayoutListener(() -> {
Rect visible = new Rect();
root.getWindowVisibleDisplayFrame(visible);
int hidden = root.getHeight() - visible.bottom;
boolean shown = hidden > root.getHeight() / 5;
// Edge to edge: with the decor no longer fitting system windows, adjustResize
// stops shrinking the window for the keyboard, so the IME is an inset too.
WindowCompat.setDecorFitsSystemWindows(getWindow(), false);
if (Build.VERSION.SDK_INT >= 28) {
// Also under the cutout in landscape, rather than letterboxed beside it.
getWindow().getAttributes().layoutInDisplayCutoutMode =
WindowManager.LayoutParams.LAYOUT_IN_DISPLAY_CUTOUT_MODE_SHORT_EDGES;
}
if (Build.VERSION.SDK_INT >= 29) {
// No scrim over 3-button navigation: GPUI paints what is underneath.
getWindow().setNavigationBarContrastEnforced(false);
}
ViewCompat.setOnApplyWindowInsetsListener(surface, (view, insets) -> {
Insets bars = insets.getInsets(
WindowInsetsCompat.Type.systemBars() | WindowInsetsCompat.Type.displayCutout());
int ime = insets.getInsets(WindowInsetsCompat.Type.ime()).bottom;
boolean shown = insets.isVisible(WindowInsetsCompat.Type.ime());
if (shown != keyboardVisible) {
keyboardVisible = shown;
Log.i(TAG, "software keyboard " + (shown ? "shown" : "hidden") + " (" + hidden + " px)");
Log.i(TAG, "software keyboard " + (shown ? "shown" : "hidden") + " (" + ime + " px)");
}
nativeKeyboardInsets(shown, shown ? hidden : 0);
nativeInsets(hostId, bars.top, bars.bottom, bars.left, bars.right, shown ? ime : 0);
return insets;
});

nativeOnCreate(this, Build.VERSION.SDK_INT);
Expand Down Expand Up @@ -207,6 +221,15 @@ private static boolean isNavigationOrShortcut(KeyEvent event) {
}
}

/** Dark status and navigation bar icons over a light app theme. Called from Rust. */
public void gpuiSetLightSystemBars(boolean light) {
runOnUiThread(() -> {
View decor = getWindow().getDecorView();
WindowCompat.getInsetsController(getWindow(), decor).setAppearanceLightStatusBars(light);
WindowCompat.getInsetsController(getWindow(), decor).setAppearanceLightNavigationBars(light);
});
}

// ── IME bridge (called from Rust on the render thread) ─────────────────

/**
Expand Down Expand Up @@ -439,7 +462,7 @@ private void endEdit() {
private static native void nativeHostDestroyed(long host);
private static native void nativeResumed(long host);
private static native void nativePaused(long host);
private static native void nativeKeyboardInsets(boolean visible, int heightPx);
private static native void nativeInsets(long host, int top, int bottom, int left, int right, int ime);
private static native boolean nativeMotion(int action, int actionIndex, int[] ids, float[] xs, float[] ys);
private static native void nativeKey(int keyCode, int action, int metaState);
private static native void nativeIme(long session, int kind, String text, int start, int end);
Expand Down
4 changes: 2 additions & 2 deletions android/app/src/main/res/values/styles.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<resources>
<style name="LabTheme" parent="android:Theme.Material.NoActionBar">
<item name="android:windowBackground">@color/lab_background</item>
<item name="android:statusBarColor">@color/lab_background</item>
<item name="android:navigationBarColor">@color/lab_background</item>
<item name="android:statusBarColor">@android:color/transparent</item>
<item name="android:navigationBarColor">@android:color/transparent</item>
</style>
</resources>
Binary file added docs/demos/safe-area-keyboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/demos/safe-area-landscape.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/demos/safe-area-portrait.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 20 additions & 1 deletion docs/safe-area-insets.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
# Safe-area insets on the host-driven path: what it would take

Status: not started. The pieces are known; they need their own PRs.
Status: done on the fork. Steps 1–2 are gpui-mobile branch
`android-host-insets` (draft upstream PR in
[upstream/safe-area-insets.md](upstream/safe-area-insets.md)); steps 3–5 are
in the lab. Three choices differ from the plan below:

- GPUI does have a safe-area concept: gpui-pre 0.3.8 has
`PlatformWindow::insets()` (`WindowInsets { safe_area, ime }`),
`on_insets_changed` and `Window::fully_visible_bounds()`. gpui-mobile
implements them, and the lab shell pads by `fully_visible_bounds()`. The
plan below missed this.
- The keyboard is passed in as a fifth value (`ime`): it becomes GPUI's `ime`
inset and feeds `gpui_mobile::keyboard_height()`, which iOS already fills.
Shrinking the `SurfaceView` in Java would have rebuilt the swapchain on
every keyboard toggle, and apps would get a different contract on Android
than on iOS. Kit's input reveal had to follow `fully_visible_bounds()`
instead of the viewport size (Kit `input-reveal-on-focus`).
- `safe_area_insets()` reads the active window from the platform, not a
host-path global, so the `android-activity` path gets it as well.

The original plan follows.

## Where things stand

Expand Down
99 changes: 99 additions & 0 deletions docs/upstream/safe-area-insets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Draft: longbridge/gpui-mobile — safe-area and keyboard insets on the host path

Not posted. Branch
[`Bombatomica64/gpui-mobile:android-host-insets`](https://github.com/Bombatomica64/gpui-mobile/tree/android-host-insets),
one commit on upstream `main` (f9fe5a7), 4 files, +155 −6. It applies to
upstream as is; on our fork `main` it only conflicts next to
`configuration_changed` (#30).

---

**Title:** android: Let host-driven apps report safe-area and keyboard insets

### What

A host that draws behind the system bars (edge to edge, which SDK 35 makes
the default) has no way to tell GPUI where the bars, the display cutout and
the software keyboard are. GPUI already models this
(`PlatformWindow::insets()` → `WindowInsets { safe_area, ime }`,
`on_insets_changed`, `Window::fully_visible_bounds()`), but the Android
window returns the default, zero, insets, and
`gpui_mobile::safe_area_insets()` returns zeros on Android.

This adds one entry point to `android::host`, in the style of the other host
calls:

```rust
pub fn insets_changed(host: HostId, safe_area: SafeAreaInsets, ime: f32)
```

- Physical pixels relative to the surface, from
`View.OnApplyWindowInsetsListener` (`Type.systemBars() | Type.displayCutout()`
and `Type.ime()`).
- The render thread stores them on that host's window. If the window does
not exist yet, they are kept until it opens.
- `AndroidPlatformWindow` implements `insets()` and `on_insets_changed`, so
GPUI refreshes the window when they change (an inset change does not
always come with a resize) and `Window::fully_visible_bounds()` excludes
them.
- `safe_area_insets()` returns the active Android window's insets (from
either entry point), and the attached host's keyboard feeds
`keyboard_height()`, so those two calls also match iOS.

A host that keeps its surface between the bars does not call it, and
everything stays zero, as today.

### Host side (Java)

```java
WindowCompat.setDecorFitsSystemWindows(getWindow(), false);
ViewCompat.setOnApplyWindowInsetsListener(surfaceView, (view, insets) -> {
Insets bars = insets.getInsets(
WindowInsetsCompat.Type.systemBars() | WindowInsetsCompat.Type.displayCutout());
int ime = insets.isVisible(WindowInsetsCompat.Type.ime())
? insets.getInsets(WindowInsetsCompat.Type.ime()).bottom : 0;
nativeInsets(hostId, bars.top, bars.bottom, bars.left, bars.right, ime);
return insets;
});
```

and in Rust:

```rust
host::insets_changed(host, SafeAreaInsets { top, bottom, left, right }, ime);
```

An app pads by what GPUI reports:

```rust
let viewport = window.viewport_size();
let visible = window.fully_visible_bounds();
div()
.size_full()
.pt(visible.top())
.pb(viewport.height - visible.bottom())
.pl(visible.left())
.pr(viewport.width - visible.right())
```

### On a device

OnePlus CPH2581 (Android 16, punch-hole cutout), in the
[GPUI Mobile Lab](https://github.com/Bombatomica64/Gpui-android). The app shell
pads by the insets and paints its title bar color underneath them. The
Diagnostics row shows the values GPUI gets, in logical pixels.

| Portrait | Landscape (cutout on the left) | Keyboard open |
| --- | --- | --- |
| ![portrait](../demos/safe-area-portrait.png) | ![landscape](../demos/safe-area-landscape.png) | ![keyboard](../demos/safe-area-keyboard.png) |

### Not in this PR

- GPUI Kit's focused-input reveal followed the viewport size, so with the
keyboard as an inset it no longer scrolled the field into view. That is a
separate Kit change (reveal on `fully_visible_bounds()`), used in the
screenshots.
- Animating with the keyboard (`WindowInsetsAnimation`). The inset arrives
once the keyboard has settled.

Thanks for looking at this.
Loading
Loading