Skip to content

README.md

Latest commit

 

History

History
134 lines (98 loc) · 8.7 KB

README.md

File metadata and controls

134 lines (98 loc) · 8.7 KB
Raw

DataLab Web

Full-Web reimplementation of the DataLab scientific data-processing platform — the entire computation engine and processing catalog run inside the browser.

DataLab Web overview, with Eze-sur-mer in the background

DataLab Web embeds the Sigima computation engine in Pyodide (CPython compiled to WebAssembly, JupyterLite-style) and pairs it with a dedicated React / TypeScript user interface modelled on the desktop Qt DataLab application. Plotting is delegated to Plotly.js since Qt-based PlotPy is not available in the browser.

🚀 Try it now — no install required

👉 https://datalab-platform.com/web/

The latest release is deployed automatically to GitHub Pages. Open the link in any modern browser (Chrome, Edge, Firefox, Safari) and the full DataLab application — Sigima, NumPy, SciPy, scikit-image, h5py — runs locally inside the browser tab. No server, no account, no upload: your data never leaves your machine.

First load downloads Pyodide and installs Sigima via micropip (~30–60 s). Subsequent loads are cached by the browser. On Microsoft Edge the first load may be much slower — see doc/troubleshooting.md for the one-line fix.

DataLab Web demo — creating a 2D sinc image, extracting a cross-section profile, measuring its full width at half-maximum, and running Canny edge detection from the command palette

Features

DataLab Web mirrors a large portion of the desktop application surface:

  • Signal & image panels — 1D curves and 2D arrays with a rich set of synthetic generators, full Plotly visualisation, cross-hair markers, contrast adjustment, cross profiles and stats area tools.
  • Processing & analysis — operations, transforms, filters, fitting, FFT/PSD, stability analyses, measurements and profile extraction, exposed automatically through the menu bar by introspecting Sigima's catalog. Compatible parameterised processings offer an optional non-publishing live preview before anything is added to the workspace.
  • ROI & object tree — segment / rectangular / circular / polygonal regions of interest, plus a multi-group workspace with drag & drop, metadata editor, statistics card and computation history.
  • Macros & notebooks — embedded Python editor and multi-tab notebook panel, each running in dedicated Web Workers, with .ipynb import/export and bidirectional macro ⇄ notebook conversion. See doc/notebooks.md.
  • Plugins — Qt-compatible PluginBase API: the same plugin source runs in DataLab desktop and DataLab Web. See doc/plugins.md.
  • I/O — HDF5 browser (via h5py in Pyodide), text import wizard and per-directory save dialog.
  • UI niceties — light / dark theme, persisted layout, pop-out result panel, contextual help, full internationalisation.

Architecture overview

flowchart LR
    UI["<b>React / TypeScript UI</b><div style='text-align:left;min-width:260px'>• Signal &amp; image panels<br/>• Plotly.js plots<br/>• Menus / dialogs<br/>• Macro editor<br/>• Plugin manager</div>"]
    Py["<b>Pyodide (CPython + WASM)</b><div style='text-align:left;min-width:420px;white-space:nowrap'>• numpy / scipy / scikit-image<br/>• h5py<br/>• sigima (computation engine)<br/>• bootstrap.py (object store + JS-friendly helpers)</div>"]
    UI -->|"runtime.ts bridge"| Py
Loading

For the full picture — layer view, component view, worker protocols, the source-tree breakdown and end-to-end flows with diagrams — see doc/architecture.md. The persistence model explains why the HDF5 workspace file is the single durable source of truth.

Comparison with related projects

Project Purpose Runs where
DataLab Reference desktop app (Qt + PlotPy) Native
DataLab-Kernel Jupyter kernel exposing DataLab to notebooks Local Python
DataLab-Web Full browser app, Sigima in WASM (this project) Browser
Sigima Headless computation engine (signals/images) Anywhere Python

Development

Prerequisites: Node.js ≥ 18.

npm install
npm run dev      # Vite dev server on http://localhost:5173
npm run build    # static bundle in dist/ (deployable to any static host)
npm run lint     # ESLint
npm run format   # Prettier

The first dev load downloads Pyodide (~10 MB) and installs Sigima via micropip (30–60 s); subsequent loads are cached. Vite is configured with base: "./" so the build works under sub-paths.

Testing unreleased Sigima changes

Cross-repository integration snapshots are declared once in sigima-dependency.json. publishedRequirement is the exact PyPI version qualified for releases; developmentRef is either null or a full 40-character commit SHA already merged into Sigima. The test and performance workflows turn that immutable SHA into both the CPython requirement and the Pyodide wheel. Install the same configured dependency locally with:

.\.venv\Scripts\python scripts\sigima_dependency.py install

If .env still includes ..\Sigima in PYTHONPATH, that sibling checkout intentionally takes priority for CPython. Remove the entry when qualifying the exact manifest-selected snapshot.

For rapid work on an uncommitted sibling checkout, PYTHONPATH makes the local Sigima sources available to CPython, but the browser still requires a wheel. Build that checkout directly:

cd ..\Sigima
python scripts\run_with_env.py python -m build --wheel --outdir dist
cd ..\DataLab-Web

Then add the generated wheel to the ignored .env file using Vite's /@fs/ URL syntax (absolute path, forward slashes):

VITE_SIGIMA_INSTALL_SPEC=/@fs/C:/Dev/Sigima/dist/sigima-X.Y.Z-py3-none-any.whl

Replace X.Y.Z with the generated filename, then use the usual npm run dev or Playwright commands. The override applies to the main runtime and the macro and notebook workers. Rebuild the wheel and restart Vite after each Sigima change. This ignored .env override takes priority over the published requirement but does not change the versioned CI snapshot. Remove the line to return to the exact PyPI requirement; /@fs/ URLs are for local development only and must not be used for release builds.

Any non-null developmentRef deliberately blocks DataLab-Web releases. Once the target Sigima version is published and qualified without a local override, set the field to null; the generalized snapshot mechanism remains available for the next coordinated change.

Documentation

Topic Guide
Documentation index doc/README.md
Architecture (layers, workers, flows, source tree) doc/architecture.md
Persistence model doc/persistence.md
Notebooks doc/notebooks.md
Plugins doc/plugins.md
Internationalisation doc/i18n.md
Testing strategy & running the suites doc/testing-strategy.md
Releasing & distribution (app + SDK tarballs) doc/releasing.md
Troubleshooting doc/troubleshooting.md
Roadmap doc/roadmap.md

Use of Generative AI

DataLab-Web is funded under an NLnet grant and complies with the NLnet policy on the use of Generative AI. GenAI is used as a development aid only; high-level review, architectural decisions and scientific validation remain under exclusive human responsibility, and AI-assisted commits carry an Assisted-by: <Model> <Version> trailer. The full rules live in CONTRIBUTING.md.

License

BSD 3-Clause, same as DataLab and Sigima.