Skip to content

Codebase audit: crash safety, data-loss notices, UI responsiveness & hygiene #2008

Open

Description

@giswqs

Tracking issue for non-trivial reliability, data-integrity, responsiveness, and code-health findings from a full-workspace audit against AGENTS.md and docs/maintaining.md.


1. Crash, Hang & Memory Bounds (Never crash)

  • GeoTIFF decimation & mask u32 overflow / infinite loop (crates/data/src/raster/geotiff.rs:1086-1090, 1126-1148, crates/data/src/raster/mask.rs:112-116): Widen tile coordinate loop arithmetic (x0, y0, x, y, w, h, k) to u64 so dimensions near u32::MAX cannot panic in debug builds or wrap x += k into an uncancellable loop in release builds. (Fixed in fix(data): GeoTIFF decimation and Zarr LZ4/blosc reads bounded against hostile sizes #2219.)
  • Empty-geometry clamp panic in mapview::text_graphic_at (crates/ui-egui/src/mapview.rs:2255-2258): Check e.is_empty() before centre.x.clamp(e.min_x, e.max_x) when map.view.clip_distance is set.
  • Negative / NaN rectangle size clamp panic in svg::layout_svg (crates/render/src/svg.rs:80, 142): Clamp w, h, and radius to finite >= 0.0 before radius.clamp(0.0, w.min(h) / 2.0) (matching crates/render/src/layout.rs:236).
  • Signed integer & 32-bit wasm32 arithmetic overflows (crates/data/src/dxf.rs:974, 1148, crates/ui-egui/src/pluginpanel.rs:734, crates/data/src/raster/zarr.rs:688-695, 745-767):
    • Replace i64::abs() on group code 62 in dxf.rs so i64::MIN cannot panic/wrap.
    • Use widened/checked arithmetic for (v - min + 1) and (max - min + 1) in pluginpanel.rs.
    • Check start <= d.len() / checked_add on Blosc block offsets in zarr.rs for 32-bit wasm32, and avoid up-front 1 GiB chunk preallocations (numcodecs_lz4, blosc) on tiny truncated inputs.
  • Format reader memory-amplification bounds (crates/data/src/sheets.rs:104-109, 180-187, 410, crates/data/src/osm.rs:185-191, 288-304, crates/data/src/sqlite.rs:125-133, 173-194, crates/data/src/filegdb.rs:376-428, 895-963):
    • sheets.rs: Validate ZIP expansion even when a preamble precedes PK\x03\x04, cover all archive members case-insensitively (including .rels), and pre-check ODS repeated cell expansion before worksheet_range.
    • osm.rs: Enforce a cumulative decoded-byte budget when cloning strings out of StringTable across nodes/ways/relations.
    • sqlite.rs: Guard against duplicate/overlapping cell pointers within a single B-tree page in scan_table and scan_index.
    • filegdb.rs: Enforce cumulative attribute/geometry byte budgets (matching esripbf.rs) when .gdbtablx contains duplicate row offsets.
  • MVT vector tile & PMTiles directory memory accounting (crates/tiles/src/mvt.rs:9-15, 400-425, 558-559, crates/tiles/src/cache.rs:1864-1869, 1913-1919, crates/tiles/src/pmtiles.rs:699-707):
    • Store multi-point MVT geometries without allocating a separate Vec<[f32; 2]> per point, enforce MAX_ITEMS and a geometry/feature byte budget per tile rather than per layer, and account for Vec/Layer struct overhead in cache::payload_bytes.
    • Bound pmtiles::leaves() by total byte budget rather than 256 leaves of up to 32 MiB (8 GiB), and ensure eviction still bounds growth when entries are Leaf::Loading.
  • Layout clamping & high-PPP blur bounds (crates/map/src/layout.rs:1436-1521, crates/render/src/lib.rs:430-434, crates/render/src/gradient_fill.rs:280):
    • Cap ElementKind::Picture { image } base64 size (matching Report::clamp / Presentation::clamp), Colorbar, Rectangle::radius, and Grid::crs in Layout::clamp, and cache decoded base64 images in render/src/layout.rs:385 & render/src/report.rs:416.
    • Clamp View::pixels_per_point and gradient_fill blur radius so small logical views with huge pixels_per_point cannot stall box_1d.
  • Size-checked file reads before std::fs::read in desktop UI (crates/ui-egui/src/lib.rs:5804, crates/ui-egui/src/pluginsui.rs:585, 601, apps/rustgis/src/open_documents.rs:72-74): Route .rgplugin file installs through store::install_path (or project::read_file) and cap open_documents::File::bytes before reading into memory.
  • Graceful OS thread-spawn failure & PyO3 catch_unwind coverage (crates/ui-egui/src/{repairsource.rs:99, layerchooser.rs:63, localagents.rs:171-206, assistantpane.rs:274, pluginsui.rs:117, sqlpane.rs:203-245, panes.rs:3229, update/install.rs:233}, apps/rustgis-python/src/lib.rs:145-245):
    • Replace the 14 remaining std::thread::spawn calls in ui-egui with std::thread::Builder::new().spawn(...).
    • Wrap render_png, layer_geojson, project_json, _peer_text, _open_peer_text, and load_plugins in rustgis-python with catch_unwind so panics convert to RustGISError and still flush queued notices.

2. Data Integrity, CRS Accuracy & Forward-Version Checks (Never silent)

  • Forward-version checks on saved settings, packages & standalone items (saved files & settings):
    • Check "version" (and refuse/avoid overwriting unreadable or newer files) in Connections::from_json (crates/engine/src/connections.rs:297-331, crates/ui-egui/src/sourcespane.rs:79-105), Sources (crates/engine/src/sources.rs:36-93), and crates/plugins/src/store.rs:48-62.
    • Validate package.json "version" in dl.import (crates/engine/src/cmd/deeplearning.rs:87-93), reject nbformat > 4 in notebook.open (crates/engine/src/cmd/notebook.rs:480-537), write "format" in .rgismap/.rgismpk/.rgislyr writers (crates/engine/src/cmd/items.rs:744, 1531-1547), add version headers/checks to standalone .rgislayout, .rgisreport, .rgisrpttemplate, .rgispres, .rgistask, .rgisramp, .rgisstyle, .rgisloc, .rgisbookmarks, and .rgisnb files, and avoid silently stripping unknown variants in crates/map de_* helpers (fills.rs:397-421, symbology.rs:2482-2506, symbol_by.rs:48-60, label_by.rs:43-55, tiles.rs:163-196, color.rs:338-346).
  • Zarr geographic/planetary CRS preservation & datum warnings (crates/data/src/raster/zarr.rs:1258-1280, 1447-1484, 2179): Keep the Crs resolved by crs_of(store, a) instead of overwriting geographic/planetary WKT (ED50, OSGB36, TOWGS84, Mars 2000) with Crs::LonLat, emit typed_crs_note / datum_warning on PROJ/EPSG attributes, and label 8-byte integer arrays Int64/UInt64. (Fixed in fix(data): rasters keep the coordinate system they give and say what they assume (#2008) #2159.)
  • GeoTIFF & ASCII Grid CRS notices, rotation resampling notices, and Palette / GrayAlpha encoding (crates/data/src/raster/geotiff.rs:1644-1676, 1774-1798, 1866, 1950-1966, crates/data/src/raster/ascii.rs:90, crates/data/src/raster/mod.rs:672, 1250-1258):
    • Record an r.lost notice whenever Crs::LonLat is assumed on CRS-less grids (and do not swallow geo.custom_error), preserve assumed_lonlat / others in raster::read_converted, and notify when north_up resamples/coarsens rotated GeoTIFFs.
    • Encode ColorHint::Palette colour tables and ColorHint::GrayAlpha alpha bands in geotiff::encode (write / write_all), and preserve Band::classes past band 1.
  • Reject geocentric CRS (EPSG:4978) in kernelDensity (crates/analysis/src/tools.rs:4744-4755): Check crs.is_geocentric() alongside crs.is_geographic() (matching projected_crs at tools.rs:4600). (Fixed in fix(analysis): reject geocentric CRS in Kernel Density #2035.)
  • Antimeridian (±180°) cell aggregation in warp::project (Method::Average / Method::Mode) (crates/analysis/src/warp.rs:898-965): Unwrap/split cell corner longitudes across ±180° when warping geographic rasters so antimeridian cells do not span 0..w across interior longitudes.
  • Streamed Shapefile (ShpSource) .dbf column fidelity (crates/data/src/source/shp.rs:447-455, 547-549): Decode FoxPro binary Double ('B') and DateTime ('T'/'@') fields instead of returning Value::Null, and either read .dbt/.fpt memos ('M') or report skipped unsupported columns via SourceInfo::notes.
  • Vector & database provider loss notices (crates/data/src/source/duckdb.rs:523, crates/data/src/wkb.rs:104-112, 230-239, 288-299, crates/data/src/topojson.rs:40-46, 113-124, crates/data/src/dxf.rs:1245-1284, crates/data/src/filegdb.rs:416-427, crates/data/src/source/{gpkg.rs:245, postgis.rs:741, postgis.rs:955, mysql.rs:632, sqlserver.rs:547}):
    • Check ST_HasZ/ST_HasM before ST_Force2D in duckdb.rs and emit zm_note().
    • Recurse into GeometryCollection members in wkb::has_zm (protecting 3D collections during gpkg/edit.rs:489 edits) and report dropped <4-point interior holes and heterogeneous collection members.
    • Report dropped Z/M coordinates in topojson.rs and sloped 3D entities in dxf.rs.
    • Add continue after report.skipped += 1 in filegdb.rs:416, call info.note_columns(...) for skipped binary/extra-geometry columns across gpkg/postgis/mysql/sqlserver/filegdb, and report undecodable rows in postgis.rs::read_editable.
  • Geoprocessing, command & startup Session::notify / ToolResult::messages gaps:
    • kriging & idwInterpolation (crates/analysis/src/kriging.rs:296-305, crates/analysis/src/tools.rs:4347-4470): Report singular-matrix IDW fallbacks, skipped Null/missing-geometry features, and averaged coincident points in ToolResult::messages; preserve crs_incomplete/zm_dropped in merge, joinField, featureToPoint, and exportFeatures.
    • Forward extra["messages"] to Session::notify in workflows::add_made (crates/engine/src/cmd/imagery.rs:1246-1266, workflows.rs:150-181), reject unparseable values in data.contingentValues instead of coercing to wildcard Value::Null (crates/engine/src/cmd/design.rs:265-270, 1726-1730), and notify (or refuse) on truncations/skips in raster.editPixels, raster.georeference, edit.paste, share.webMap, share.locator, report.export, overpass, nldi, routing, and QGIS/GeoLibre project importers.
    • Surface desktop startup project and CLI/file-association open errors via app.session.notify(e, true) in apps/rustgis/src/main.rs:668-670, 703-705.

3. UI-Thread Responsiveness & Cancellation (Never hold the window)

  • Avoid 30-second UI-thread sleep on pending tile legends in Layout view (crates/render/src/layout.rs:79-99, 338, 2012-2019): Skip wait_for_tile_legends when render_pending runs on the UI thread (!wait_shadows), marking pending = true so the page redraws when legend images land; check cancel::requested() during export waits (layout.rs, raster_export.rs:80-136). (Fixed in fix(layout): the Layout view no longer waits for legend pictures, and an export's waits are complete and cancellable (#2008) #2279.)
  • Cancellation checks & test coverage across geoprocessing tools (crates/analysis/src/warp.rs:831, 889-969, crates/analysis/src/tools.rs:5870-6224, crates/tiles/src/vector_export.rs:59-217, crates/analysis/src/network.rs:43-236, crates/analysis/src/dggs.rs:248-408, crates/analysis/src/rasterops.rs:296-300, crates/engine/src/tests.rs:44356-44374):
    • Add Pace / control::check() inside warp::Aggregate::row (and account for Mode sample vectors in threads_for_rows), tabulateIntersection, vectorToPmtiles, network::shortest_route, a5Grid/a5Aggregate, rasterops::resample_to same-grid path, joinField, merge, summaryStatistics, pivotTable, xyTableToPoint, featureVerticesToPoints, and tools/geometry.rs.
    • Add all input-scaling geoprocessing tools and reads::read_ahead / ReduceRun / TileExportRun runners to long_commands_stop_when_cancelled_while_they_run.
  • Background dispatch for desktop exports, prints, raster/Zarr builders & heavy render passes (crates/ui-egui/src/lib.rs:2482-2520, 3861-3870, 4070-4091, crates/engine/src/cmd/{workflows.rs:191-206, exports.rs:40-47, imagery.rs:49-125, zarr.rs:24-29, cube.rs:31, plugins.rs:181-192}, crates/render/src/{raster.rs:469-593, effects.rs:497-646, report.rs:315-491}):
    • Route exports::runs_in_background commands through exportjob::start inside RustGisApp::run so Command Search, Assistant, and MCP exports do not block the UI thread.
    • Offload 300 DPI print generation, multi-frame/package/sharing exports, raster/Zarr/cube builders, cache.fill, and plugin.prepare off the UI thread, and check cancel::requested() inside raster::draw_with, 3D extrusion, thinning, and report pagination.
    • Extend MCP HTTP crates/assistant/src/relay/http.rs:438-471 (submit) to wait up to APPROVAL_WAIT when a request is held for interactive user approval.

4. Architecture, Geometry Edge Cases & Repo Hygiene

  • Parse [target.'cfg(...)'.dependencies] and [dependencies.<pkg>] in cargo xtask layers (xtask/src/main.rs:35-53).
  • Align crate-root #![deny(...)] lint headers in apps/rustgis/src/main.rs:388, apps/rustgis-cli/src/main.rs:30, and xtask/src/main.rs with the full 6-lint workspace block (unwrap_used, expect_used, panic, unimplemented, todo, unreachable).
  • Add ATTRIBUTION.md entries for crates/tiles/src/cubes.json and crates/tiles/src/default_style.json.
  • Validate non-finite coordinates / radii in rustgis_geom and UTF-8 char length in label_by (crates/geom/src/lib.rs:397-400, 756-766, crates/geom/src/world.rs:180-216, crates/map/src/label_by.rs:49): Reject !is_finite() latitudes in Aeqd, non-positive/NaN sphere radius in World, and use query::check_len (char count) in de_label_by.
  • Wrap remaining hardcoded English UI notices in tl! / i18n::fmt (apps/rustgis-mobile/src/lib.rs:361-411, apps/rustgis-web/src/web.rs:216, crates/ui-egui/src/autosave.rs:248-281, 729-741).

Activity

  1. choupeanut commented on Nov 14, 2026

    @choupeanut
    Contributor

    I'll take only the Kernel Density geocentric-CRS finding in section 2. I checked current main (7c377ff), issue comments and open PRs and found no existing claim or implementation for this item. I'll reject geocentric targets before the 2D calculation for both an explicit crs and the input layer's default CRS, add regression coverage and retain working projected-CRS behavior. No dependency, command-parameter or data-model changes; the rest of this audit remains open.

  2. choupeanut commented on Nov 14, 2026

    @choupeanut
    Contributor

    The Kernel Density geocentric-CRS finding is addressed in #2035 (453e308). Both explicit and inherited geocentric systems are refused before projection or raster creation, with a projected-CRS suggestion. The regression covers EPSG:4978 and a custom geocentric PROJ definition, plus successful projected overrides; all 8 Kernel Density tests passed. Full cargo xtask ci passed (6,452 tests passed, 23 ignored), all pre-commit hooks passed, and the analysis crate's wasm32 check passed. The rest of this tracking issue remains open.

  3. choupeanut commented on Nov 14, 2026

    @choupeanut
    Contributor

    I'll take only the rustgis_geom::world::World sphere-radius validation part of section 4. I checked main at 3da536e, comments and open PRs and found no claim or fix for it. I'll make both forward and inverse projection return None for non-positive or non-finite radii, document that contract, and test all five world projections with invalid and valid radii. No API signatures, dependencies or data-model changes. The Aeqd and UTF-8 label-length findings, and the rest of the audit, remain open.

  4. choupeanut commented on Nov 14, 2026

    @choupeanut
    Contributor

    The World sphere-radius validation subitem is addressed in #2038 (6ba2530). Both forward and inverse projection now return None for zero, negative or non-finite radii, before projection arithmetic. All five world projections are covered by invalid-radius regressions and positive-radius round trips. cargo xtask ci passed (6,461 tests passed, 23 ignored), all pre-commit hooks passed, and the geom crate's wasm32 check passed. The Aeqd and UTF-8 label-length findings remain open; this PR does not close the audit.

  5. teaphile commented on Nov 14, 2026

    @teaphile
    Contributor

    I will take only the TopoJSON dropped Z/M notice finding in section 2 (also listed as open in ROADMAP.md). I checked current main, this issue’s comments and open PRs and found no claim or implementation for that part. I plan to retain existing 2D geometry behavior and report affected-feature counts through the existing reader/session notice path. Tests will cover direct Point/MultiPoint coordinates, transformed and untransformed arcs, clean 2D silence, and session propagation. No dependencies, command parameters, data-model or saved-format changes. The remaining audit findings stay open.

  6. deniial00 commented on Nov 15, 2026

    @deniial00
    Member

    I'm taking only section 3's pending Layout-view legend wait and the associated layout/raster export-wait cancellation finding. I checked main at 9b128c4, this issue's comments and cross-references, all-state legend/wait PR searches, and the current scope of #1177, #2131 and #2114; I found no overlapping claim or fix for these waits. Plan: skip legend sleeping for interactive layout rendering and retain pending state; enable desktop legend readiness invalidation using #1177's existing redraw-on-completion predicate; make export legend/tile waits observe cancellation without returning a successful partial export; verify with controlled pending/completion/cancellation regressions, the actual Layout surface and the repository gates. Export tile/shadow completeness stays intact. No new dependencies, command parameters, saved-format or core-model changes; #2131/#2114 features and the rest of this audit remain out of scope.

  7. deniial00 commented on Nov 15, 2026

    @deniial00
    Member

    I'm taking only these section 2 subfindings: (1) Zarr geographic/planetary CRS preservation and the Int64/UInt64 labels; (2) GeoTIFF/ASCII CRS-less and custom-CRS notices, read_converted's lost assumed_lonlat/extra variables, and the north_up resampling notice for rotated GeoTIFFs.

    I checked current main (9b128c4), this issue's comments and all-state PRs, and found no claim or implementation for these. I read #2117 (Zarr spatial:/proj: inheritance) and #2032 (CRS engine swap): I'm not touching their transform, inheritance or engine work.

    Plan:

    1. zarr.rs: keep the geographic or planetary Crs that crs_of resolves (ED50, OSGB36, TOWGS84, Mars 2000) instead of forcing Crs::LonLat, with its notes. Say another body once, whether it came from WKT or a PROJ/EPSG attribute, and label 8-byte integer arrays Int64/UInt64.
    2. geotiff.rs/ascii.rs: one Raster::lost line when a CRS-less grid is read as WGS 84 (with geo.custom_error's reason when there is one), and one line when north_up resamples or coarsens a rotated GeoTIFF.
    3. raster/mod.rs/engine: read_converted keeps assumed_lonlat and the unread variables in lost, so every GDAL read says them through the session notices.
    4. Verification: lossy and clean tests per reader on tiny synthetic files, engine session-notice tests, docs, then cargo xtask ci and the wasm build.

    Out of scope: #2117's and #2032's work, GeoTIFF palette/gray-alpha encoding, codec and decimation bounds, datum grids, a CF Zarr store with no CRS attribute, world-file placement, and any new dependency, parameter or data-model change.

  8. deniial00 commented on Nov 15, 2026

    @deniial00
    Member

    I'm taking only these two section 1 subfindings: (1) GeoTIFF decimation and transparency-mask coordinate/step arithmetic near u32::MAX (Chunks::wanted/place in geotiff.rs, mask::read); (2) the Zarr numcodecs_lz4 and blosc decoders' up-front allocations and block-offset arithmetic (zarr.rs).

    I checked current main (9b128c4), this issue's comments and cross-references, and all-state PR searches for these symbols, and found no claim or implementation for either. I read the code: x0 + cw, div_ceil(k) * k and x += k in Chunks are u32 (x += k wraps into an endless loop in release, panics in debug), mask::read has the same rounding, and numcodecs_lz4/blosc allocate what their header claims before any input is checked, with at + len slice ends unchecked (overflow on wasm32). #2117 touches zarr.rs for spatial metadata, not the codecs; the open CRS claim here leaves codec and decimation bounds out.

    Plan:

    1. geotiff.rs/mask.rs: tile corners, sampling starts, ends and steps in u64 (pixel placement, decimation and the chunk-by-chunk Cancel/progress unchanged), and the mask's output made through raster::filled, so a failed allocation is an error the reader already reports, not an abort.
    2. zarr.rs: LZ4 and blosc refuse a header that claims more output than its input can hold (LZ4's 255:1 ceiling), a block table or offset outside the chunk, and a block size of 0 or a short result, as errors before allocating; slice ends use checked arithmetic.
    3. Tests on synthetic geometry up to u32::MAX and on small lying, truncated and boundary frames (nothing allocates a GiB), plus the existing decimation, mask, blosc and LZ4 round trips; docs/user-guide/data.md if its wording needs it. Checked with cargo xtask ci and the wasm build.

    No dependency, command-parameter or data-model change. Everything else in this audit stays open.

  9. deniial00 commented on Nov 17, 2026

    @deniial00
    Member

    I'm taking only the spreadsheet ZIP/ODS memory-amplification subfinding in section 1. I checked current main (841447d), this issue's comments/cross-references and all-state spreadsheet/ODS/ZIP/preamble PR searches; I found no overlapping claim or fix. Plan: (1) validate every ZIP member, including mixed-case names and .rels, under the existing shared expanded-byte budget and support preambles; (2) pre-scan ODS repetition counts, checked products/sums and cumulative materialized sheet extents under the existing cell limit before opening calamine (ODS loads eagerly); (3) add tiny synthetic regressions through sheet_names, columns and read, preserving ordinary repeats and text/Null notices, and document the limits. No dependencies, public API, command parameters or saved-format changes. The other readers and XLS fixes remain out of scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions