Render MapLibre-style at the drawn scale: tile57/3 tiles, runtime sounding runs, exchange-set bakes - #58
Merged
Merged
Conversation
Every symbol in a baked library draws at the engine's SYMBOL_SCALE, but the MapLibre sheet rasterized at the 0.08 catalogue scale — 8x the area anything displayed. Bake at the drawn scale (mln_drawn_scale) and rebase the style's icon-size so everything samples 1:1: the day sheet falls 105 MB -> 19 MB RGBA and every entry fits under MapLibre's 1024 px image cap. Sounding depths are comma-joined glyph runs no prebaked sheet can enumerate. tile57_render_symbol_run renders one run to RGBA on demand (exportAlloc convention, so tile57_free stays the one deallocator); the host answers MapLibre's missing-image events with it. renderSymbolRun applies the same drawn scale so runtime digits match the sheet. glyphpbf wrote fields 5 and 6 unsigned; they are zigzag in the spec, so any negative glyph bearing corrupted the whole PBF walk downstream (the 'Channel' -> 'anne' text). A TestReader regression walks the encoded ranges.
MapLibre evaluates every style layer's filter against every feature of that layer's source-layer at each tile layout. With all decorated lines in one 'lines' source-layer, 85 of 100 layers scanned every line feature of every tile — about a thousand expression nodes per feature, the largest single cost in a zoom profile, and why content trailed the camera on a library crossing. The bake now routes each complex-linestyle feature into its own 'lines-ls-<STYLE>' source-layer, so a style's two or three layers see only their own features and a tile without the style doesn't carry the source-layer at all. The name keeps the 'lines' prefix: replay and query dispatch on startsWith and portray them unchanged. The compositor buckets the split layers by name through seam composes, and the style drops the merged spacing-bucket machinery — symbol spacing is exact per style again. endScene also folds, per feature, everything the style otherwise re-derives per feature per layer (augmentV3): vz/oz, the display zooms the SCAMIN and overscale gates compare against ['zoom'] in one node; ep, the effective DrawingPriority with the danger-over-sounding deviation baked in, which collapses the point partition's class-membership clause; lsk, the line z-order sort; and mq/iso/lt flags replacing the M_QUAL/ISODGR01/LIGHTS string compares. Found wiring it: the category gate read a 'cat' property no tile has ever carried (tiles bake display_category), so every feature coalesced to STANDARD and the base/other toggles and data-quality overlay were silently inert on the MapLibre path. The gate reads the real properties now. Schema tile57/3: a /2 archive still renders under the new style but gates and sorts nothing, so a library needs a rebake. Note for zip sources: the bake preserves the archive's inner directory prefix, so 'ENC_ROOT/' nests — worth flattening some day.
tile57_bake_zip is the archive twin of tile57_bake_tree: find the cells, name every output, bake. Nothing is unpacked — each cell inflates as its turn comes, so importing an 800 MB set never costs the disk a second copy. Naming was the reason for adding it. Working out where each chart goes had been reinvented three times (this CLI, the macOS shell, an Android path) and two of the three carried the archive's own wrapper directory into the output: NOAA's All_ENCs.zip puts every cell under ENC_ROOT/, so -o ~/Charts/ENC_ROOT wrote ~/Charts/ENC_ROOT/ENC_ROOT/ — a second library beside the real one, which a host that opens the parent then composes together with it, one vintage silently winning per tile. The prefix is computed from what the cells actually share rather than assumed to be one level, so an archive holding two districts keeps them apart. The CLI now takes the rule from the engine (archiveRootPrefix) instead of its own copy. Incremental, as bakeTree already was: an archive newer than the zip is skipped. A national set is hours of work on a tablet and will be interrupted — backgrounded, battery, or the mariner stopping to sail — and without this every resume started from the first cell and never finished. The count is reported before the first chart bakes rather than when one finishes. The first cell out of a national set is a huge overview cell, so a host had no denominator for minutes and could only show a spinner; it looked stuck while working. fileModNs uses statFile instead of open+fstat+close, which over Android's FUSE storage was three round trips per chart before any progress was reported at all. The CLI's own tests are collected now. Nothing but the executable referenced tools/*.zig, and an executable does not collect tests, so the test already written there had never run.
tg_portray_run stood up a whole VM per call: a fresh lua_State, the standard libraries, thirty-odd host bindings, then require of the five framework modules — which COMPILES them. A bake runs it up to three times per chart (base, plain boundaries, simplified symbols), so a 7,224-cell exchange set compiled the framework about 21,000 times. Lua's code generator was the largest single item in an Android bake profile, ahead of every piece of chart work. Two steps, because the obvious one is wrong on its own. Caching the compiled bytecode per thread and undumping it is safe and needs no other change. Keeping the VM as well is what removes newstate/openlibs/register, and that alone broke six golden Part-9 streams: PortrayalAPI holds featureCache/informationCache/spatialCache at module level keyed by ID, and our IDs are feature indices, which repeat from one chart to the next — the second chart read the first chart's features back out of the cache. Dropping the modules from package.loaded and requiring them again per pass rebuilds those caches, and with the bytecode cached it costs no compilation. Thread-local throughout: a lua_State is not thread-safe and the bake runs a worker per core. douglasPeucker counts its kept points and allocates once instead of appending one at a time, which regrew the list log2(n) times per line — over every line in every tile of every cell, with Android's hardened allocator charging for each. Measured over a 1,200-chart bake on an 8-core tablet these are worth 1.00-1.06x, which is not what the profile shares suggested: memset fell 20% -> 12% and libc 5.7% -> 2.6%, but shares redistribute when the denominator shrinks. Kept because they cut real work — a phone with fewer cores and a battery pays for all of it.
…logue The cache was built for five framework modules and the catalogue is not five modules: the framework requires a rule module per feature class on demand, and sixty charts touch 98 of them. At sixteen entries it cached the framework, recompiled every rule, and missed 3,741 times against 2,031 hits. Sized properly that inverts to 329 misses — one compile per module per thread, which is the floor. It makes no difference. Measured over identical bakes: 60 charts on macOS 0.96x, 1,200 charts on an 8-core tablet 378s against 372s. Both inside run-to-run noise, and the properly-sized cache is no faster than the broken one. The compile cost is real and visible in a profile — Lua's code generator was the largest single item on Android — but it is not on the critical path, and freeing those cycles buys no throughput. So the VM reuse goes. It was the half that carried risk: PortrayalAPI keeps ID-keyed caches at module level and our IDs repeat between charts, so keeping a state read one chart's features back for the next — six golden Part-9 streams changed. Reloading the framework per pass fixed that, and paid for it with a full GC and five module re-executions per pass. None of that is worth carrying for an effect no measurement can find. The bytecode cache stays: it is twenty lines, cannot leak state, and costs one dump per module per thread. The measured speedup on Android is the worker count, not this.
…can build A 32-bit x86 target has no 64-bit atomic add, which the OpenCPN plugin build hits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch fixes how charts render through the MapLibre path and the GPU scene, and adds the bake and compose pieces those fixes needed.
Rendering
screen.
Tile format
rebaked.
Bake
Build
ABI additions: tile57_bake_zip, tile57_compose_set_px_per_tile, tile57_render_symbol_run, tile57_bake_sprite_mln takes a pixel ratio and scheme, and the GPU scene vertex and quad carry per-vertex color, paint-order depth, and the text run flip/tangent fields.