electron shared texture

.codespellrc

@@ -1,2 +1,3 @@
 [codespell]
 skip = **/package-lock.json,CHANGELOG.md
+ignore-words-list = renderd

.github/workflows/test_and_release.yml

@@ -138,6 +138,15 @@ jobs:
           cd js-components
           npm run lint
 
+      - name: Test trame-rca-electron
+        run: |
+          cd electron
+          npm ci
+          npm test
+          npm run lint
+        env:
+          ELECTRON_SKIP_BINARY_DOWNLOAD: 1
+
   test-npm-build:
     runs-on: ubuntu-latest
 

README.rst

@@ -128,3 +128,52 @@ up outside the trame-rca directory, checkout the `wheel` branch, and install it
     cd ../trame-rca
 
     python ./examples/05_video/vtk_cone_simple_video.py
+
+Shared textures with Electron
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In an Electron app, VTK can render into a GPU texture that Electron shows as is: without encoding
+and without copying to the CPU. The Electron side is the ``@kitware/trame-rca-electron`` npm package in
+``./electron``: it starts the trame server and loads its page. Supported on macOS (IOSurface),
+Windows (D3D11 NT handles) and Linux under Wayland (dma-buf).
+
+- macOS: works out of the box (IOSurface).
+- Windows: the GL driver needs ``WGL_NV_DX_interop2``, and VTK must render on the GPU
+  Chromium uses. Set ``D3D_ADAPTER=<index>`` if that isn't DXGI adapter 0.
+- Linux: Wayland only for now (dma-buf). Electron duplicates the server's file descriptors
+  with ``pidfd_getfd``, which needs ``kernel.yama.ptrace_scope`` at 1 or less:
+  ``sudo sysctl -w kernel.yama.ptrace_scope=1``. Set ``RENDER_NODE=/dev/dri/renderD129``
+  if VTK picks another GPU than the compositor.
+
+.. code-block:: python
+
+    from trame_rca.widgets import init_vtk_for_shared_textures
+    from trame_rca.widgets import rca
+
+    init_vtk_for_shared_textures()  # before creating any VTK render window
+
+    # ... create render_window, then in the UI:
+    view = rca.RemoteControlledArea(display="shared-texture")
+    view_handler = view.create_electron_handler(render_window)
+
+VTK renders into a ring of shared textures, by default this is 2. This allows VTK
+to draw the next frame, while Electron still shows the previous one. If you're on a
+mid-tier to high-end GPU, you normally do not need to use more than 1 texture because
+VTK can render faster. However, on low-end hardware, it's more efficient to render the
+N+k frames while electron is presenting the N'th frame. This hides the cost of each render.
+The  ``create_electron_handler(..., texture_count=n)`` sets the size of this ring texture container,
+and you can change it at runtime with ``view_handler.texture_count = n``. A value of 1
+waits for Electron after every frame.
+
+You can try our code example:
+
+.. code-block:: console
+
+    pip install trame trame-vuetify
+    pip install "trame-rca[electron]"
+
+    cd examples/06_electron/electron
+    npm i
+    npm start
+
+See ``electron/README.md`` for the Electron API and what packaging an app needs.

electron/.eslintrc.cjs

@@ -0,0 +1,14 @@
+/* eslint-env node */
+module.exports = {
+  root: true,
+  env: {
+    node: true,
+    browser: true,
+    es2022: true,
+  },
+  extends: ['eslint:recommended'],
+  parserOptions: {
+    ecmaVersion: 'latest',
+    sourceType: 'module',
+  },
+};

electron/.gitignore

@@ -0,0 +1 @@
+node_modules/

electron/README.md

@@ -0,0 +1,53 @@
+# @kitware/trame-rca-electron
+
+Shows trame-rca `display="shared-texture"` views in Electron. The trame server renders VTK
+into a GPU texture, and this package hands it to the page without encoding, or copying to the CPU.
+
+This package and the `trame-rca` python package speak a versioned protocol (see
+`src/main/protocol.js` and `trame_rca/electron/comm.py`): use matching releases.
+
+## Use
+
+```js
+import { app, BrowserWindow } from 'electron/main';
+import { launchTrame, webPreferences } from '@kitware/trame-rca-electron';
+
+app.whenReady().then(() => {
+  const win = new BrowserWindow({ webPreferences: webPreferences() });
+  launchTrame(win, { command: 'python3', args: ['app.py'], onExit: () => app.quit() });
+});
+```
+
+`launchTrame(win, options)` spawns `command` with `args` plus `--server --port 0 --emp-port <port>`,
+and loads the page in `win` once trame serves it. Options:
+
+- `command`, `args`, `cwd`, `env`: the trame server to run. Any executable works: a python
+  interpreter and a script, a frozen backend, etc.
+- `onExit(code, signal)`: the server exited.
+- `onStats({ view, fps, frames, info })`: once a second, per shared-texture view that drew frames.
+- `log`: defaults to `console`.
+
+It returns `{ process, close }`. `close()` kills the server; it is called when `win` closes.
+
+`webPreferences(extra)` sets the preload, and turns off the sandbox: the preload needs
+`sharedTexture`, and Electron doesn't run ES module preloads in a sandbox. If the window needs its
+own preload, `import '@kitware/trame-rca-electron/preload'` from it: it must then be an ES module
+preload too, with the `.mjs` extension.
+
+On the python side, see `examples/06_electron` in trame-rca.
+
+## Packaging an app
+
+Packaging (electron-builder or Forge, a frozen python backend, signing) belongs to each app.
+What this package needs from a packaged app:
+
+- `koffi` is a native module: unpack it from the asar archive (e.g. electron-builder's
+  `asarUnpack: ["**/node_modules/koffi/**"]`).
+- Pass the packaged backend as `command`. It must be spawned by `launchTrame`, as a direct child
+  of the main process: on Linux, duplicating its dma-buf file descriptors needs ptrace access to
+  it (`kernel.yama.ptrace_scope` at 1 or less).
+- The frozen backend must include `vtkmodules` and the `serve` directories of `trame_rca` and of
+  every other trame package it uses.
+- macOS shares the texture as a global IOSurface, which does not work in an App Sandbox (so not on
+  the Mac App Store).
+- Linux: X11 is not supported yet, Electron must run under Wayland.