Skip to content

[Feature]: Scale the paged-feature and raster-sample caps with the computer's memory #1318

Closed

Description

@elpidioassinaturas

Before you start

What problem does this solve?

Two fixed caps decide how much data RustGIS holds, whatever the computer:

  • Features read from a paged service: ogc::MAX_PAGED_FEATURES is 100,000 for a WFS, an OGC API Features collection or an ArcGIS layer read page by page. IBGE's soil map of Brazil (BDIA:pedo_area, the national 1:250,000 survey) has 114,658 polygons. It can't be read whole, so a user who wants the country's soils has to fetch it by regions and merge them.
  • Raster samples: raster::MAX_SAMPLES is 120 million on the desktop and 32 million in the browser. Rasterizing that soil map at 250 m, its survey's scale, gives about 300 million cells for Brazil (about 4,400 × 4,300 km). That is out of reach in one piece, while 500 m (about 76 million) fits.

Both caps are right for a small machine. A workstation with 32 or 64 GB could hold several times as much. #830 was opened because reading pages used far more memory than the data, which kept the page cap at 100,000. Now that it is closed, the cap could follow the memory the computer has.

Proposed solution

On the desktop, derive both caps from the memory available when RustGIS starts, with today's values as the floor and a ceiling:

  • MAX_PAGED_FEATURES: for example, today's 100,000 per 8 GB of available memory, up to a ceiling of perhaps 1,000,000.
  • MAX_SAMPLES: likewise, from today's 120 million, keeping the "reading briefly needs about twice this" margin the code already notes.
  • The browser keeps fixed caps, since wasm memory tops out at 4 GB and the page shares it.
  • When a read stops at the cap, the message says what the cap is on this computer, and that a smaller area or a lower resolution fits.

Estimated scope

Small to medium. It needs a way to read available memory on Windows, macOS and Linux (a small dependency, or the system calls), the two constants turned into functions, and the places that read them. A setting to override the cap could come later.

Alternatives considered

  • A setting for each cap: simpler, but a user has to know their machine and the risk of setting it too high.
  • Leave the caps fixed: users split large reads by region and merge them, and rasterize in pieces or at a coarser resolution. It works, but it costs time and steps, and it isn't obvious why a large machine can't do more.
  • Read large services in place ([Feature]: Read large ArcGIS layers live, a view at a time (an ArcGIS FeatureSource) #829): that suits viewing. A whole-dataset step such as rasterizing still needs every feature.

Activity

  1. pick2510 commented on Nov 2, 2026

    @pick2510
    Contributor

    This overlaps step 2 of #1291 ("desktop caps scaled to the machine"), which proposes deriving the analysis tools' fixed limits from the machine's memory. Nobody has started that step yet. Doing MAX_SAMPLES and MAX_PAGED_FEATURES here, in the same way, would keep one approach for both sets of caps.

    Related work already in:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions