Compile the full engine to WebAssembly - #59
Merged
Conversation
The complete C ABI — bake, chart, compose, style, raster — builds as one wasm reactor module, so a browser chartplotter can bake charts and serve tiles with no server. Lua and libtess2 keep their setjmp error paths: those objects compile per-file with -mexception-handling plus clang's sjlj lowering, and the vendored wasi-libc runtime (src/portray/wasm_sjlj_rt.c) supplies the helpers. The target's own feature set stays default, so Zig's wasi-libc build never compiles its own (broken in zig 0.16) copy. Portability gates: filemap reads instead of mmap on wasi, the pmtiles reader lock is a no-op there, every thread fan-out comptime-gates on builtin.single_threaded, SQLite drops to THREADSAFE=0, and capi's time(3) binding uses time_t (64-bit on wasi). bindings/wasm/engine-smoke.mjs drives the real pipeline under node's WASI: bake one S-57 cell, open the archive from bytes, fetch a vector tile, render a PNG view.
…demo wasi-shim.mjs is a dependency-free wasi_snapshot_preview1 host for the browser: a read-only in-memory file tree preopened at one path, the clock, randomness, and stdout/stderr to the console — the full import surface of tile57-engine.wasm. tile57.mjs wraps the exports one-to-one (linear-memory allocation, C strings, out-params, tile57_error decode). demo.html fetches S-57 cells over HTTP, bakes them, composes them, and renders every pan/zoom view inside the page. The shim path is exercised end to end under node (same code, no node:wasi): bake two cells, compose, tile, PNG.
The engine's zip bake writes per-chart archives, so the shim's file tree now supports create, write, rename, and directory ops. bakeZip wraps tile57_bake_zip: one call bakes a dropped exchange set into the tree.
gpu-renderer.mjs draws the engine's GPU scenes with WebGPU. Its WGSL is a port of the reference shaders in shaders/ over the same vertex, quad, and uniform layouts. The engine batches the ranges (tile57_gpu_batch); the renderer uploads the buffers once per scene and redraws from uniforms alone, so pan and zoom stay live between scene rebuilds. tile57.mjs grows the GPU surface: scene build and decode, batch, the sprite and SDF-glyph atlas bakes, the colortables, and the ABI layout check. demo.html is now a full-screen chartplotter: drop .000 cells or an exchange-set .zip, drag to pan, wheel to zoom, HUD, zoom and fullscreen controls over the map. WebGPU when the browser has it; PNG views otherwise (?png=1 forces the fallback). A pre-module script catches drops from first paint — an uncancelled drop navigates the tab while the engine is still compiling. The shim resolves absolute paths against the preopen root: the engine passes some opens that way, and the zip bake's archive writes hit it.
A bake holds the CPU for seconds and a district zip for minutes; on the main thread the page froze and no loader could even animate. The engine now lives in engine-worker.mjs and the page drives it one call per RPC message, so the map, the HUD, and the loader stay live while it works. The loader shows determinate progress: a dropped zip is listed (tile57_zip_list), then extracted (tile57_zip_extract) and baked one cell at a time — N of M with the cell name, for plain cells and zip members alike. zip_extract creates no directories, so the worker pre-creates each out path's parents in the shim tree (MemFS.mkdirs). The GPU renderer no longer holds an engine handle: it takes the atlas assets at create and plain draw-ready data per scene, so the engine can stay in the worker while the device and buffers live on the page. When WebGPU is unavailable the HUD now names the reason for the PNG fallback instead of switching silently.
navigator.gpu exists only in a secure context; a LAN IP over plain http hides it with the same symptom as a browser without WebGPU.
The docs workflow builds the engine (ReleaseSmall: one third the download, this workload runs just as fast), stages the demo app under static/demo-app/, and src/pages/demo.jsx embeds it at /demo — GitHub Pages is https, so WebGPU is available to it. A Live Demo item joins the navbar, and engine changes now redeploy the site so the demo always runs the current engine. CI builds the wasm engine too, so a break surfaces on every push, not just docs deploys. Demo polish: the drop hint links NOAA's free ENC downloads and says charts never leave the page, arrow keys pan and +/- zoom, an engine that fails to start reports itself instead of spinning, and a batch's failures land in one status line.
The chrome themes itself from the engine's own colour tables (a new palette worker op): NODTA for the map surround, UIBCK panels with UIBDR borders, UINFD/UINFF text, UINFB accents, UINFR for failure lines, and the cursor readout in CURSR. The CSS fallbacks carry the same day values, so the page looks right before the engine is up.
zig fmt re-columns the sjlj flag table. The time binding uses std.c.time_t only where the target defines it — Windows leaves it void in std.c, and the wasi fix had broken both Windows cross-compiles.
One wasm instance is single-threaded, so parallel baking means more instances: bake-pool.mjs spins up N extra engine workers (sized from the machine's cores, ?workers=N overrides) and a batch's cells fan out across them, several at a time. A cell bake is pure — bytes in, archive bytes out — so pool slots share nothing; the primary engine worker keeps the charts, the compositor, and rendering, and archives open there as bakes finish. A dropped zip stays in the primary (it lists and extracts); each cell's extracted files shuttle out through a new readFile op to a pool slot, so extraction streams while bakes run wide. Pool slots close when the batch ends — an engine instance holds linear memory wasm never returns, and respawning one costs far less than keeping it. worker-rpc.mjs carries the shared RPC protocol for both the primary and the pool.
…inish wasm i32 values cross into JS SIGNED, so once the engine's memory grew past 2 GiB every returned pointer looked negative — each DataView write threw, ops unwound mid-call, and a district-sized drop collapsed into a cascade of range errors and follow-on OOMs. The wrapper now masks every allocation (walloc: checked and >>> 0), and the shim masks every numeric argument on the way in — the engine passes pointers into WASI too. The GPU renderer draws the bare NODTA ground when no scene is uploaded instead of touching missing buffers. Memory now stays flat through a big batch: a new remove op drops files from an engine's tree, pool slots free each cell's files as its bake ends, the primary frees a zip cell's extracted files the same way, and the zips themselves free when the batch closes.
Browsers deprecate the legacy 'try'; wasm-use-legacy-eh=false makes the sjlj lowering emit try_table over exnref instead, which also needs the reference-types feature — enabled per file the same way as exception-handling, since zig's module-derived feature flags land last and must be won back with raw -Xclang pairs.
…enes Baked archives go to disk as they finish and only the view's charts stay in the engine — the fix for a district-sized drop exhausting the primary's 4 GiB linear memory at compose_open. chart-library.mjs stores each archive in the browser's origin-private file system with a metadata sidecar (the tile57_info the bake worker now returns alongside the bytes), so a page load catalogs the library without opening one archive, and re-drops of known cells skip the bake. Without OPFS the session keeps archives in page memory instead. The 🗑 control clears the library. The page now holds a catalog, not open handles. Each rebuild selects the charts whose bounds intersect the view at a compilation scale within ~11x of the view's (nearest few as fallback, so overscale beats blank sea), opens the missing ones from the library, composes exactly that subset, and evicts least-recently-used charts beyond a 32-chart cap. The composite is keyed by the selected set and reused while the set is stable. Scenes also refresh DURING a gesture now: rebuilds run in the worker, so a wheel-zoom or drag kicks a throttled rebuild and a sharper scene lands mid-gesture instead of waiting for the debounce to settle.
Over the cap the selection order decides who draws, and it now keeps the most GENERAL charts first: they cover the view in the fewest cells, so a z3 view over a harbor-cell library fills with the overview band instead of 32 arbitrary harbor cells. Under the cap the order is moot — the engine's partition gives detailed charts precedence on overlap. The out-of-window fallback competes every intersecting chart instead of an arbitrary eight, and ?open=N tunes the cap.
… at 64 The library catalog needs no engine: the page lists OPFS before the wasm even downloads, so a returning visit shows its chart count and stored bytes immediately (and the HUD keeps showing them through engine startup instead of flashing 'no charts'). The library listing now carries each archive's on-disk size, the HUD shows the running total, and newly baked charts add to it. The residency cap default rises to 64 open charts (?open=N still overrides).
One overview chart can span an ocean; the bounds union then centres far off the charts that matter. Charts with a footprint over ~8x the median stay out of the fit (they still draw), so a loaded library lands centered over its detailed cluster.
The vertex shaders draw each vertex at the world copy nearest the camera — seamless across the antimeridian when zoomed in, but in a wide view the seam meridian falls onto geometry and a primitive straddling it wraps its vertices to opposite copies, tearing into full-width horizontal streaks. Once the viewport spans a large share of a world the renderer now disables the wrap (everything draws in its home copy). Stored bytes come from navigator.storage.estimate() — one instant call. The per-file sizing crawled at thousands of charts and read zero sizes on Firefox, showing '1 kB' for a full library. clear() is one recursive directory remove instead of thousands of per-file deletes.
The scale window alone left holes: at z4 only the overview band qualified (21 charts open under a 64 cap) while areas those charts miss stayed blank until deeper zoom pulled the next band in. Slots left after the suitable window now go to out-of-window charts nearest the view's scale — an area only a detailed chart covers shows that chart overscaled, never a hole. The camera persists in localStorage on every settled rebuild, and a load with a saved view lands exactly there; the cluster fit only runs for a first visit (and after a drop, which still jumps to what was added).
Rotation rides the GPU-scene contract as designed: scenes stay north-up in world space, the MVP turns the camera, and the shaders turn exactly the map-aligned offsets by the same angle (rot_sin/rot_cos were already wired). Shift-drag rotates about the screen centre, a two-finger pinch zooms, twists, and pans about its midpoint, the compass control resets north-up (its needle tracks the heading, shown in the HUD), and the rotation persists with the saved view. All camera math — cursor readout, pan, anchored zoom, the PNG placement — now goes through rotation-aware screen/world transforms. A fast pan release flicks: the camera keeps its velocity and decays exponentially, redrawing live and settling into a rebuild. Scene and PNG requests cover the rotated viewport's bounding box inflated by a prefetch margin (?margin=K, default 1.6), and chart selection uses the same box — a pan, zoom-out, or turn shows real chart from the standing scene while a sharper one streams in. The PNG fallback re-places its last north-up render as one CSS similarity transform, which gives it the same live pan/zoom/rotate preview.
Half the length: the demo sits at the top behind the standard Annapolis view (the same view the quick-start png command draws), the body says what you can build rather than which calls exist, and the API tables and code samples live where the docs already cover them.
bindings/wasm merges into bindings/js: the JS modules, the demo, the wasm build entry, and the old style-only package now live in one directory, like bindings/go. The package's main export is the full engine (index.mjs re-exports every module and createEngine stands the engine up in a browser or node); the style-only engine moves to the tile57/style subpath (style.js, style.d.ts). The full-engine wasm is a build artifact (npm run build), not a committed binary. README: the API list names C, Zig, JavaScript, Go, and the CLI; the screenshot drops to 140 kB; no em dashes.
The demo becomes a real chartplotter host, organized as focused modules under bindings/js/demo/ (view, camera, gestures, chart store, importer, mariner model, settings panel, pick) with demo.html as a slim shell. The chrome follows the chartplotter and lookout-marine shells: the map is the UI, round buttons float in the corners, one bottom-centre data card carries the readout, bake progress, and messages (no more modal loader), panels are caret popovers, toasts stack above the card, and the ui tokens re-skin everything for day, dusk, and night. A first visit gets a welcome card, and the docs workflow bundles a NOAA sample harbor (US5MD12M) it can load with one tap. Mariner settings run end to end: tile57.mjs marshals tile57_mariner (the node test proves the layout against the engine's own defaults and a night render differing from day), the worker passes settings through the png and gpu-scene ops, and the settings panel carries the options the struct takes: detail level, depth contours in the mariner's unit, the soundings tri-state, symbol and text choices, dates. The scheme control cycles day, dusk, night: ui tokens flip, the sprite atlas re-bakes for the palette, and scenes rebuild with the new settings. A tap picks. The C query API reports through a callback no JS host can provide, so wasm_root grows tile57_wasm_query: the callback lives in the module and the features return as one JSON array. The worker pairs each feature with its decoded tile57_s57_report; pick-model.mjs ranks the set (a port of lookout-marine's pick.zig rules) and the report card presents it lookout's way, chips for the set, the operative fact as the title, the raw S-57 one fold away.
The S-52 canon (safety contour 10 m) suits SOLAS drafts; a sailboat draws about 2 m, and a 10 m safety contour paints most of a harbor as unsafe. The demo defaults to shallow 2 m, safety 3 m, deep 10 m, safety depth 3 m; stored settings still win.
The recreational convention on US charts; stored values stay metric under the hood and the mariner's own setting still wins.
The rebuild reads gpu.halo for the SDF text pass and the clear colour; an un-awaited setScheme left both one scheme behind, so labels held the old palette until the next camera move rebuilt again.
The pick cursor; the grabbing hand shows only during an actual drag.
Annapolis and the bay approaches at 1:40,000. The previous cell was a thin harbor slice that did not even contain the Annapolis point.
The harbor at 1:12,000 (US5MD1MC), the Severn and the bay approaches at 1:40,000 (US5MD13M, US5MD12M), and the band-3/4 context (US4MD1DD, US3EC08M) so zooming out still shows chart. One NOAA cell was a slice; this set quilts the whole area at every zoom.
The FullLightLines bake portrays a sectored light's legs twice (sect 0, the 25 mm stubs; sect 1, the full-length pass) and filters them in the MapLibre style. The engine's own render paths resolve display variants in resolve.zig, which gated bnd and pts but not sect, so both passes drew and full sector lines showed whichever way the switch stood. The composed replay also dropped the sect tag on decode. Validated on US5MD1LC: Thomas Point Shoal Light draws stubs with the switch off and full legs with it on.
US5MD1MD carries the eastern half of the deep-draft anchorage and was missing, so the sample cut it at the harbor cell's seam; the bundle now holds the central 1:12,000 row (MB, MC, MD), the Thomas Point cell (LC), and the 1:40,000/45,000/200,000 context. The anchorage view now renders byte-identical to the full Maryland library. The README lists quilting as its own capability.
The pick report follows lookout-marine's callout: the pick's objects stay in sight as a left column (the main data in each row, the object on show held selected, the chart's M_* notes pinned at the column's floor), and the detail holds the decoded report with the provenance line and the S-57 source fold at ITS floor, so the controls keep their place while the rows scroll. A copy control puts the raw payload on the clipboard. Esc closes the settings drawer first, then the report.
The text and pictures a chart's features point at now survive import: the zip path stores every non-cell sibling (the TXTDSC notes, PICREP pictures) in the library beside the chart's archive (OPFS, with a session fallback), and a file row in the pick report opens its content inline - text as a scrollable block, web-displayable pictures as an image, TIFF named as such. Clearing a chart clears its files. The book, copy, and document glyphs become inline SVGs: the exotic codepoints read as tofu on fonts that lack them, which is what the notes rows and data-quality entries were showing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
zig build wasm-enginebuilds the complete C ABI — bake, chart, compose, style, raster — as one wasm32-wasi reactor module. A browser can bake charts and serve tiles with no server.Lua and libtess2 keep their setjmp/longjmp error paths. Those objects compile per file with the wasm exception-handling feature plus clang's sjlj lowering, and a vendored copy of wasi-libc's sjlj runtime supplies the helpers. The target's own feature set stays default: with the feature enabled target-wide, zig 0.16 compiles its own copy of that runtime into libc, and that compile crashes. Every current browser and node implements the instructions this emits.
Engine changes for the target:
builtin.single_threadedat comptime and fall back to their serial paths.THREADSAFE=0on wasm.time(3)binding usestime_t;c_longis 32-bit on wasm32 and wasm-ld rejects the mismatch against libc.JS bindings, in
bindings/wasm/:wasi-shim.mjs— a dependency-free WASI host for the browser, with a writable in-memory file tree.tile57.mjs— a wrapper over the exports: allocation, C strings, out-parameters, error decode.gpu-renderer.mjs— WebGPU over the engine's draw-ready GPU scenes. The WGSL is a port of the reference shaders inshaders/; buffers upload once per scene and pan/zoom redraws from uniforms alone.engine-worker.mjs— the engine in a Web Worker, one call per RPC message, so the page stays live through long bakes.demo.html— a full-screen chartplotter: drop.000cells or an exchange-set.zip, drag to pan, wheel to zoom, per-cell bake progress. WebGPU when the browser has it, PNG views otherwise.engine-smoke.mjs— the pipeline under node's WASI host.Verified on real NOAA cells, under node and under the browser shim: bake US5BDRAB and US5BDRBB from S-57, compose them, fetch a vector tile, render correct S-52 views.
zig build,zig build wasm, andzig build testare unchanged and pass.