Skip to content
Merged
8 changes: 5 additions & 3 deletions crates/data/src/ogc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -464,9 +464,11 @@ pub struct Service {

/// Most layers or collections `service.layers` lists from one document. An OGC API collection
/// can make up to three layers (features, map tiles, vector tiles), so its listing can pass this
/// many layers. The others are still read ([`Service::listed`]), so that they can be added by
/// name: which ones come first is up to the server, and some change their order between answers
/// (#1519).
/// many layers. In an OGC document the others are still read ([`Service::listed`]), so that they
/// can be added by name: which ones come first is up to the server, and some change their order
/// between answers (#1519). Reading stops at [`MAX_READ_LAYERS`], or sooner when the layers'
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
/// request addresses are long ([`read_limit`], [`MAX_COPIED_ADDRESS_BYTES`]); past that none is
/// read. An ArcGIS list is read only as far as it is listed.
const MAX_LISTED_LAYERS: usize = 2_000;
/// Most layers or collections read from one document, which is hostile input.
const MAX_READ_LAYERS: usize = if cfg!(test) { 3_000 } else { 100_000 };
Expand Down
28 changes: 25 additions & 3 deletions crates/engine/src/cmd/embeddings.rs
Original file line number Diff line number Diff line change
Expand Up @@ -148,13 +148,15 @@ fn add(s: &mut Session, p: &Value) -> CmdResult {
});
// Its header read first, so its bands are known; the layer is framed only when it is the
// map's first content (no "zoom" is passed, so add_cog defaults it to that). In the
// browser nothing may wait, and it is read as the map draws.
// browser nothing may wait, nor may a plugin's command in the app (it runs on the
// frame): it is read as the map draws.
let probe = cfg!(not(target_arch = "wasm32")) && !s.leave_downloads;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unusable tile is accepted silently on the plugin path. With leave_downloads set, probe is now false, so the probe_blocking check that returned "{file}: not a cloud-optimised GeoTIFF" no longer runs. add_cog_with then calls the non-blocking cog::info, which returns Ok(None) while the header loads. Nothing reports a failure, and Session::notify is never called. So an unreachable or non-COG AlphaEarth tile yields Ok with no notice. The new test asserts exactly that: r.is_ok() for https://127.0.0.1:9/....

This matches what the browser already does. The AGENTS.md rule "Unusable results are errors" still applies, though. Suggest either a notice when the background header read later fails, or a note in the command's result that the tile was added unchecked.

Confidence: moderate (about 55%). The design choice is deliberate, so this may be a trade-off to confirm rather than a defect.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same trade-off as layer.add on a plugin's command and in the browser: with leave_downloads the frame can't wait for the header, so the tile is added and read as the map draws. It isn't silent: when the header read fails, the tile layer says why (the same failure the layer shows elsewhere), and add_cog puts a warning in the answer when a failure is already recorded. A notice tied to this command's result once the background read fails would apply to every unchecked COG add, so it is left for the release quality pass.

#[cfg(not(target_arch = "wasm32"))]
if !crate::render::rustgis_tiles::cog::probe_blocking(file)? {
if probe && !crate::render::rustgis_tiles::cog::probe_blocking(file)? {
return Err(format!("{file}: not a cloud-optimised GeoTIFF"));
}
let look = crate::render::rustgis_tiles::cog::with_look(file, &look);
super::layer::add_cog(s, &json!({"name": name}), &look, cfg!(not(target_arch = "wasm32")))
super::layer::add_cog(s, &json!({"name": name}), &look, probe)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}
"earth-index" => add_points(s, p, name),
_ => add_tessera(s, p, d, name),
Expand Down Expand Up @@ -313,6 +315,26 @@ mod tests {
assert_eq!(band_label([1, 16, 9]), "2, 17, 10");
}

/// A plugin's command in the app (`leave_downloads`) doesn't read an AlphaEarth tile's header
/// on the frame: the layer is added and read as the map draws.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn alphaearth_added_without_waiting_when_downloads_are_left() {
let url = "https://127.0.0.1:9/annual/2024/10N/x.tiff";
let mut s = Session::new(Project::default());
s.leave_downloads = true;
let r = s.run("embeddings.add", &json!({"dataset": "alphaearth", "url": url}));
assert!(r.is_ok(), "{r:?}");
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
// The tile is added with its AlphaEarth look.
let tile = s.map().tiles.last().map(|t| t.url.clone()).unwrap_or_default();
let (file, look) = crate::render::rustgis_tiles::cog::split(&tile);
assert_eq!(file, url);
assert!(look.aef, "{tile}");
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
// Elsewhere its header is read first, so an unreadable tile is refused.
let mut t = Session::new(Project::default());
assert!(t.run("embeddings.add", &json!({"dataset": "alphaearth", "url": url})).is_err());
}

#[test]
fn points_coloured_by_their_components() {
let mut s = Session::new(Project::default());
Expand Down
2 changes: 1 addition & 1 deletion crates/engine/src/cmd/imagery.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ pub fn commands() -> Vec<C> {
r#"{"layer": raster, "crs"?: EPSG code or "EPSG:n" of the output (default the layer's), "cellSize"?: number or [x, y] in the output's units (default about the layer's pixel there), "origin"?: [x, y] a cell corner falls on (default [0, 0], so grids of the same cellSize line up), "extent"?: polygon layer (its selected or all features' bounds) | [west, south, east, north] in degrees | "view" (default the layer's footprint), "grid"?: raster layer whose coordinate system, cells and extent are taken exactly (instead of crs, cellSize, origin and extent), "resampling"?: "nearest" (default) | "bilinear" | "cubic" | "average" (the mean of the pixels whose centres fall in a cell) | "mode" (their most frequent value, the least of those tied), "name"?: new layer (default "<layer> projected"), "path"?: GeoTIFF to write it to}"#,
),
C::new("raster.rasterize", "Polygons to Raster", "Analyze › Alignment", rasterize).inputs(rasterize_inputs).roles(&[]).effects(crate::policy::Effects::NONE).params(
r#"{"layer": polygon layer (its selected or all features; a later one over an earlier), "field"?: numeric field (its values) or text field (codes 1 to n in sorted order of its values, listed in a table named after the result with classes), "value"?: number every polygon gives when there is no field (default 1), "nodata"?: number of the cells under no polygon (default 0), "crs"?: EPSG code or "EPSG:n" of the output (default the layer's), "cellSize"?: number or [x, y] in the output's units (required without grid), "origin"?: [x, y] a cell corner falls on (default [0, 0], so grids of the same cellSize line up), "extent"?: polygon layer (its selected or all features' bounds) | [west, south, east, north] in degrees | "view" (default the polygons' bounds), "grid"?: raster layer whose coordinate system, cells and extent are taken exactly (instead of crs, cellSize, origin and extent), "name"?: new layer (default "<layer> raster"), "path"?: GeoTIFF to write it to}"#,
r#"{"layer": polygon layer (its selected or all features; a later one over an earlier), "field"?: numeric field (its values) or text field (codes 1 to n in sorted order of its values, listed in a table named after the result with classes), "value"?: number every polygon gives when there is no field (default 1), "nodata"?: number of the cells under no polygon (default 0), "crs"?: EPSG code or "EPSG:n" of the output (default the layer's), "cellSize"?: number or [x, y] in the output's units (required without grid), "origin"?: [x, y] a cell corner falls on (default [0, 0], so grids of the same cellSize line up), "extent"?: polygon layer (its selected or all features' bounds) | [west, south, east, north] in degrees | "view"; default the polygons' bounds, "grid"?: raster layer whose coordinate system, cells and extent are taken exactly (instead of crs, cellSize, origin and extent), "name"?: new layer (default "<layer> raster"), "path"?: GeoTIFF to write it to}"#,
),
// A fit is a measurement of the points, not a project change: no undo step.
C::new("raster.gcpFit", "Control Point Fit", "Analyze › Alignment", gcp_fit).roles(&[]).effects(crate::policy::Effects::NONE).view().params(
Expand Down
16 changes: 13 additions & 3 deletions crates/engine/src/cmd/items.rs
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ fn embedded_map(m: &Map, budget: &mut u64) -> Map {
/// Adds maps from a map file, project or package.
fn import_maps(s: &mut Session, p: &Value) -> CmdResult {
let text = read_text(p)?;
crate::project::check_format(&text, "map file or project")?;
crate::project::check_format(&text, "map file, project or package")?;
let dir = ps(p, "path").and_then(|path| std::path::absolute(path).ok()).and_then(|p| p.parent().map(Path::to_path_buf));
super::connections::clear_active();
let maps: Vec<Map> = match serde_json::from_str::<Map>(&text) {
Expand Down Expand Up @@ -451,7 +451,7 @@ fn import_maps(s: &mut Session, p: &Value) -> CmdResult {
}
Err(error) => {
super::connections::restore_active(s);
return Err(error);
return Err(not_a(error, "map file, project or package"));
Comment thread
giswqs marked this conversation as resolved.
Resolved
}
},
};
Expand Down Expand Up @@ -487,6 +487,16 @@ fn import_maps(s: &mut Session, p: &Value) -> CmdResult {
Ok(json!({"maps": names}))
}

/// A project read's refusal for an import that also takes `kinds`: text that is none of them is
/// said to be none of them, not only a failed project. Other refusals (such as "Update RustGIS")
/// are kept as they are.
fn not_a(error: String, kinds: &str) -> String {
match error.strip_prefix("not a RustGIS project: ") {
Some(why) => format!("not a RustGIS {kinds}: {why}"),
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
None => error,
}
}

fn add_style(s: &mut Session, mut item: StyleItem) {
item.name = item.name.trim().chars().take(200).collect();
if item.name.is_empty() {
Expand Down Expand Up @@ -515,7 +525,7 @@ fn import_styles(s: &mut Session, p: &Value) -> CmdResult {
super::connections::clear_active();
let project = Project::from_import_json(&text, None);
super::connections::restore_active(s);
let mut project = project?;
let mut project = project.map_err(|e| not_a(e, "layer file, map file or project"))?;
for notice in std::mem::take(&mut project.label_read_notices) {
s.notify(notice, false);
}
Expand Down
30 changes: 23 additions & 7 deletions crates/engine/src/cmd/layer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1763,7 +1763,8 @@ fn ground_overlays(s: &mut Session, file_name: &str, bytes: &[u8]) -> Vec<(Strin
let dir = Path::new(file_name).parent().filter(|d| d.is_absolute());
for (g, picture) in found {
let href = g.href.trim();
let picture = picture.or_else(|| {
// `Err(None)`: not in the file or beside it; `Err(Some(why))`: beside it but unreadable.
let picture = picture.ok_or(None::<String>).or_else(|_| {
// Beside the document: no web addresses, absolute paths or climbing out of its folder.
let rel = Path::new(href);
let safe = !href.contains("://")
Expand All @@ -1772,14 +1773,29 @@ fn ground_overlays(s: &mut Session, file_name: &str, bytes: &[u8]) -> Vec<(Strin
// Windows too (`C:x`, `\\x`).
&& rel.components().all(|c| matches!(c, std::path::Component::Normal(_) | std::path::Component::CurDir));
if !safe {
return None;
return Err(None);
}
let p = dir?.join(rel);
crate::project::read_file(&p, (256 << 20) - 1).ok()
let p = dir.ok_or(None)?.join(rel);
// Only a missing picture isn't beside it; one that can't be looked at says why.
// (Followed as the read follows it: a link to nothing is missing too.)
match std::fs::metadata(&p) {
Ok(_) => {}
// (A file named as a folder on its way, `a.png/b.png`, isn't beside it either.)
Err(e) if matches!(e.kind(), std::io::ErrorKind::NotFound | std::io::ErrorKind::NotADirectory) => return Err(None),
Err(e) => return Err(Some(e.to_string())),
Comment thread
giswqs marked this conversation as resolved.
Resolved
}
crate::project::read_file(&p, (256 << 20) - 1).map_err(Some)
});
let Some(picture) = picture else {
notes.push(format!("{} (its picture {href} isn't in the file or beside it)", g.name));
continue;
let picture = match picture {
Ok(p) => p,
Err(None) => {
notes.push(format!("{} (its picture {href} isn't in the file or beside it)", g.name));
continue;
}
Err(Some(why)) => {
notes.push(format!("{} (its picture {href} can't be read: {why})", g.name));
continue;
}
};
match rustgis_data::raster::from_image(&g.name, &picture, g.bbox, g.rotation) {
Ok(r) => out.push((g.name.clone(), r)),
Expand Down
4 changes: 1 addition & 3 deletions crates/engine/src/cmd/layout.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3479,11 +3479,9 @@ fn series_page(s: &mut Session, p: &Value) -> CmdResult {
Ok(json!({"page": page, "pages": n, "name": name}))
}

/// Renders a layout to PNG or PDF bytes. `all` renders every map series page (PDF only).
/// What an export of a layout whose texts name installed families is told while the app is still
/// indexing the fonts (its first seconds; the app's exports don't wait for them): those texts
/// are drawn in their fallbacks. A script or the command line indexes them at
/// once.
/// are drawn in their fallbacks. A script or the command line indexes them at once.
fn named_fonts_unread(layout: &Layout) -> Option<&'static str> {
let named = layout.elements.iter().any(|e| matches!(&e.kind, ElementKind::Text { font_name: Some(_), .. }));
(named && rustgis_render::families::available().is_none()).then_some(
Expand Down
12 changes: 7 additions & 5 deletions crates/engine/src/cmd/search.rs
Original file line number Diff line number Diff line change
Expand Up @@ -398,15 +398,17 @@ fn search_cmd(s: &mut Session, p: &Value) -> CmdResult {
/// A place typed as an MGRS reference (the centre of the square it names) or a UTM position, in
/// the forms [`rustgis_geom::parse_mgrs`] and [`rustgis_geom::parse_utm`] accept; `None` for
/// anything else, an ambiguous UTM zone letter or a reference that names no place.
#[cfg(test)]
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
pub fn parse_grid_reference(text: &str) -> Option<Coord> {
read_grid_reference(text).ok()
}

/// As [`parse_grid_reference`], with why a reference was refused: [`GridError::Syntax`] for text
/// written as neither form, otherwise the reason of the form it was written in. A 100 km square
/// alone written with spaces (`5 Ave`, `4K TV`) is a number and a word as often as a reference,
/// so one that names no square is [`GridError::Syntax`] too; written as one word (`38TMJ`), it
/// says why.
/// A place typed as an MGRS reference (the centre of the square it names) or a UTM position, in
/// the forms [`rustgis_geom::parse_mgrs`] and [`rustgis_geom::parse_utm`] accept, or why it was
/// refused: [`GridError::Syntax`] for text written as neither form, otherwise the reason of the
/// form it was written in. A 100 km square alone written with spaces (`5 Ave`, `4K TV`) is a
/// number and a word as often as a reference, so one that names no square is
/// [`GridError::Syntax`] too; written as one word (`38TMJ`), it says why.
pub fn read_grid_reference(text: &str) -> Result<Coord, GridError> {
match rustgis_geom::parse_mgrs(text) {
Ok(m) => m.centre(),
Expand Down
Loading
Loading