Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 33 additions & 2 deletions crates/analysis/src/rasterops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,18 @@ pub fn derive(r: &Raster, name: &str, bands: Vec<Vec<f32>>) -> Result<Raster, St
.map_err(|e| e.to_string())
}

/// [`derive`] for a [`hillshade`]'s pixels: one band of whole numbers 1–255 stored as bytes
/// (`UInt8`), with 0 as no-data, so it is written as 8-bit samples, not 32-bit floats.
pub fn derive_hillshade(r: &Raster, name: &str, mut shade: Vec<f32>) -> Result<Raster, String> {
r.require_coordinates()?;
for v in &mut shade {
if !v.is_finite() || *v < 1.0 {
*v = 0.0;
}
}
Raster::new(name, r.width, r.height, vec![shade], r.transform, r.crs, Some(0.0), ColorHint::Gray, "UInt8").map_err(|e| e.to_string())
}

fn band(r: &Raster, b: usize) -> Result<&[f32], String> {
r.bands.get(b).map(|b| b.data.as_slice()).ok_or_else(|| format!("{} has no band {}", r.name, b + 1))
}
Expand Down Expand Up @@ -196,7 +208,8 @@ pub fn aspect(r: &Raster, b: usize) -> Result<Vec<f32>, String> {
terrain(r, d, 1.0, |gx, gy| if gx.abs() < 1e-12 && gy.abs() < 1e-12 { -1.0 } else { (-gx).atan2(-gy).to_degrees().rem_euclid(360.0) })
}

/// Hillshade (0–255) lit from `azimuth` and `altitude` degrees.
/// Hillshade (1–255) lit from `azimuth` and `altitude` degrees: a cell in full shadow is 1, so
/// that 0 is left for the no-data of the bytes [`derive_hillshade`] makes of it.
pub fn hillshade(r: &Raster, b: usize, azimuth: f64, altitude: f64, zf: f64) -> Result<Vec<f32>, String> {
let d = band(r, b)?;
let (zen, az) = ((90.0 - altitude).to_radians(), azimuth.to_radians());
Expand All @@ -205,7 +218,7 @@ pub fn hillshade(r: &Raster, b: usize, azimuth: f64, altitude: f64, zf: f64) ->
let slope = gx.hypot(gy).atan();
let aspect = (-gx).atan2(-gy);
let v = cos_zen * slope.cos() + sin_zen * slope.sin() * (az - aspect).cos();
(v.max(0.0) * 255.0).round()
(v.max(0.0) * 255.0).round().max(1.0)
})
}

Expand Down Expand Up @@ -2402,6 +2415,24 @@ mod tests {
);
}

/// A cell in full shadow is 1, not 0, so that the bytes made of a hillshade keep 0 for its
/// no-data cells.
#[test]
fn hillshade_keeps_zero_for_no_data() {
let gt = GeoTransform { origin_x: 0.0, origin_y: 1.0, pixel_width: 0.001, pixel_height: -0.001 };
let mut cells = vec![10.0; 25];
cells[12] = -9999.0;
let r = Raster::new("plane", 5, 5, vec![cells], gt, Crs::LonLat, Some(-9999.0), ColorHint::Gray, "Float32")
.unwrap_or_else(|e| panic!("{e}"));
// The sun on the horizon lights no flat ground.
let shade = hillshade(&r, 0, 315.0, 0.0, 1.0).unwrap_or_default();
assert_eq!((shade[0], shade[12]), (1.0, NODATA));
let out = derive_hillshade(&r, "shade", shade).unwrap_or_else(|e| panic!("{e}"));
assert_eq!((out.data_type.as_str(), out.nodata), ("UInt8", Some(0.0)));
assert_eq!((out.value(0, 0, 0), out.value(0, 2, 2), out.bands[0].data[12]), (Some(1.0), None, 0.0));
assert_eq!(out.bands[0].stats.count, 24);
}

#[test]
fn polygon_mask_fills_polygons_and_cuts_holes() {
let r = ramp();
Expand Down
41 changes: 37 additions & 4 deletions crates/data/src/raster/geotiff.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1640,7 +1640,7 @@ pub fn written_bands(r: &Raster, band: usize) -> Vec<usize> {

/// Writes a raster as a Deflate-compressed GeoTIFF: a colour raster's colour bands (RGB(A) bytes
/// when they are 8-bit, else 32-bit floats, alpha kept), so it reads back in colour; otherwise
/// `band` (0-based) as 32-bit floats. `band` is for rasters that aren't colour ones: a colour
/// `band` (0-based), as bytes when it is an 8-bit band of whole bytes, else as 32-bit floats. `band` is for rasters that aren't colour ones: a colour
/// raster always writes its colour bands, as its 8-bit form did before. A cancelled run stops
/// it (with [`DataError::Cancelled`]) within a strip.
pub fn write(r: &Raster, band: usize) -> Result<Vec<u8>> {
Expand Down Expand Up @@ -1855,9 +1855,26 @@ fn encode(r: &Raster, band: usize, every: bool) -> Result<Vec<u8>> {
}
} else {
let b = r.bands.get(band).ok_or_else(|| err(format!("the raster has no band {}", band + 1)))?;
let mut img = enc.new_image::<colortype::Gray32Float>(w, h).map_err(err)?;
tags!(img, &[band]);
img.write_data(&b.data).map_err(err)?;
// An 8-bit band is written as bytes when every value is a whole byte and an empty cell is
// only where no-data can mark it (as the multiband samples above); otherwise as floats.
let byte = |v: &f32| if r.is_valid(*v) { v.fract() == 0.0 && (0.0..=255.0).contains(v) } else { r.nodata.is_some() };
if r.data_type == "UInt8" && fits_bytes(r, band) && b.data.iter().all(byte) {
let nodata = r.nodata.map_or(0, |d| d as u8);
let mut data = super::room(b.data.len(), w as usize, h as usize)?;
for (i, v) in b.data.iter().enumerate() {
if i % (1 << 20) == 0 {
crate::progress::check()?;
}
data.push(if r.is_valid(*v) { *v as u8 } else { nodata });
}
let mut img = enc.new_image::<colortype::Gray8>(w, h).map_err(err)?;
tags!(img, &[band]);
img.write_data(&data).map_err(err)?;
} else {
let mut img = enc.new_image::<colortype::Gray32Float>(w, h).map_err(err)?;
tags!(img, &[band]);
img.write_data(&b.data).map_err(err)?;
}
}
Ok(out.0.into_inner())
}
Expand Down Expand Up @@ -1966,6 +1983,22 @@ mod tests {
assert_eq!(back.data_type, "Float32");
}

/// A grey 8-bit band is written as bytes, its no-data cells as the no-data value; without a
/// no-data value to mark an empty cell, or with a fraction, it stays floats.
#[test]
fn grey_bytes_round_trip_as_bytes() {
let gt = GeoTransform { origin_x: 0.0, origin_y: 3.0, pixel_width: 1.0, pixel_height: -1.0 };
let byte = |cells: Vec<f32>, nodata| {
let r = Raster::new("g", 3, 1, vec![cells], gt, Crs::LonLat, nodata, ColorHint::Gray, "UInt8").unwrap();
read(std::io::Cursor::new(write(&r, 0).unwrap()), "back").unwrap()
};
let back = byte(vec![1.0, 0.0, 255.0], Some(0.0));
assert_eq!((back.data_type.as_str(), back.nodata), ("UInt8", Some(0.0)));
assert_eq!((back.value(0, 0, 0), back.value(0, 1, 0), back.value(0, 2, 0)), (Some(1.0), None, Some(255.0)));
assert_eq!(byte(vec![1.0, f32::NAN, 2.0], None).data_type, "Float32");
assert_eq!(byte(vec![1.0, 2.5, 2.0], Some(0.0)).data_type, "Float32");
}

#[test]
fn rotated_matrix_is_turned_north_up() {
// Issue #251: GDAL's geotransform (400000, 30, 5, 5000000, 3, -30).
Expand Down
35 changes: 33 additions & 2 deletions crates/engine/src/cmd/workflows.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1019,6 +1019,7 @@ fn function_work(s: &Session, p: &Value) -> Result<Work, String> {
let src = raster_of(s, ps(p, "layer").ok_or("layer is required")?)?;
let target = resample_target(s, p)?;
let name = out_name(p, format!("{}_{}", src.name, ps(p, "function").unwrap_or("")));
let defaults = s.raster_defaults.clone();
let mut p = p.clone();
// Read here, on the command's thread, where an unreadable value is said (#1600); only a remap
// or threshold reads it.
Expand All @@ -1027,11 +1028,32 @@ fn function_work(s: &Session, p: &Value) -> Result<Work, String> {
}
Ok(Box::new(move || {
let pixels = apply_function(&src, &p, target.as_deref())?;
if is_hillshade(&p) {
return made_hillshade(rasterops::derive_hillshade(&src, &name, pixels)?, &defaults);
}
// Only a resample takes another raster's grid.
made(rasterops::derive(target.as_deref().unwrap_or(&src), &name, vec![pixels])?, None)
}))
}

/// Whether raster function step `p` is a hillshade, whose pixels are bytes
/// ([`rasterops::derive_hillshade`]).
fn is_hillshade(p: &Value) -> bool {
ps(p, "function") == Some("hillshade")
}

/// [`made`] for a hillshade, drawn as a stretch on the session's raster `defaults` however few
/// shades it holds: as whole numbers, 16 or fewer (a gentle slope) would otherwise be drawn as
/// classes (Unique Values).
fn made_hillshade(raster: Raster, defaults: &rustgis_map::RasterDefaults) -> Result<Made, String> {
let mut m = made(raster, None)?;
let style = &mut m.layer.style;
style.renderer = rustgis_map::RasterRenderer::Stretch { band: 0, ramp: defaults.ramp.clone(), invert: defaults.invert };
style.stretch = defaults.stretch;
style.resampling = defaults.resampling;
Ok(m)
}

/// Runs raster functions one after another, each on the previous one's output.
fn function_chain(s: &mut Session, p: &Value) -> CmdResult {
made_or_ran(s, p, "raster.functionChain")
Expand All @@ -1056,18 +1078,27 @@ fn function_chain_work(s: &Session, p: &Value) -> Result<Work, String> {
.map(|(i, st)| resample_target(s, st).map_err(|e| format!("step {} ({}): {e}", i + 1, func(st))))
.collect::<Result<_, _>>()?;
let named = Some(out_name(p, String::new())).filter(|n| !n.is_empty());
let defaults = s.raster_defaults.clone();
Ok(Box::new(move || {
let mut cur: Raster = (*src).clone();
let mut names = Vec::new();
for (i, (st, target)) in steps.iter().zip(&targets).enumerate() {
let f = func(st);
let pixels = apply_function(&cur, st, target.as_deref()).map_err(|e| format!("step {} ({f}): {e}", i + 1))?;
// Only a resample takes another raster's grid; otherwise the grid is the current one.
cur = rasterops::derive(target.as_deref().unwrap_or(&cur), &cur.name, vec![pixels])?;
cur = if is_hillshade(st) {
rasterops::derive_hillshade(&cur, &cur.name, pixels)?
} else {
rasterops::derive(target.as_deref().unwrap_or(&cur), &cur.name, vec![pixels])?
};
names.push(f);
}
cur.name = named.unwrap_or_else(|| format!("{}_{}", cur.name, names.join("_")));
made(cur, None)
// A chain that ends in a hillshade is drawn as one.
match steps.last() {
Some(last) if is_hillshade(last) => made_hillshade(cur, &defaults),
_ => made(cur, None),
}
}))
}

Expand Down
52 changes: 52 additions & 0 deletions crates/engine/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11450,6 +11450,58 @@ fn a_scan_warped_onto_its_points_is_placed_for_real() {
std::fs::remove_dir_all(&dir).ok();
}

/// A hillshade is made as bytes, 1–255 with 0 for no data, so it is exported as 8-bit samples (a
/// GeoTIFF and a COG); it is still drawn as a grey stretch, even of a plane with a single shade,
/// which as whole numbers would otherwise be drawn as classes. A chain ending in one is the same.
#[test]
fn hillshade_is_made_and_written_as_bytes() {
use base64::Engine;
use rustgis_data::raster::{ColorHint, Crs, GeoTransform, Raster, geotiff};
let mut s = world();
s.run("layer.addDataBytes", &json!({"fileName": "dem.tif", "data": dem_b64()})).unwrap();
let gt = GeoTransform { origin_x: 10.0, origin_y: 45.04, pixel_width: 0.001, pixel_height: -0.001 };
let plane = Raster::new("plane", 10, 10, vec![vec![50.0; 100]], gt, Crs::LonLat, None, ColorHint::Gray, "Float32").unwrap();
let plane = base64::engine::general_purpose::STANDARD.encode(geotiff::write(&plane, 0).unwrap());
s.run("layer.addDataBytes", &json!({"fileName": "plane.tif", "data": plane})).unwrap();
let shade = |s: &mut Session, cmd: &str, p: Value| {
let r = s.run(cmd, &p).unwrap_or_else(|e| panic!("{cmd} {p}: {e}"));
let l = s.map().layer(r["layer"].as_u64().unwrap() as u32).unwrap().raster.clone().unwrap();
let d = l.data.clone().unwrap();
assert_eq!((d.data_type.as_str(), d.nodata), ("UInt8", Some(0.0)), "{cmd} {p}");
assert!(matches!(l.style.renderer, rustgis_map::RasterRenderer::Stretch { .. }), "{cmd} {p}: {:?}", l.style.renderer);
(s.map().layer(r["layer"].as_u64().unwrap() as u32).unwrap().name.clone(), d)
};
let (hill, d) = shade(&mut s, "raster.function", json!({"layer": "dem", "function": "hillshade"}));
let st = &d.bands[0].stats;
assert!(st.min >= 1.0 && st.max <= 255.0 && st.count == 1600, "{st:?}");
let (_, flat) = shade(&mut s, "raster.function", json!({"layer": "plane", "function": "hillshade"}));
assert_eq!(flat.bands[0].stats.categories.as_ref().map(Vec::len), Some(1));
let steps = json!([{"function": "bandMath", "expression": "b1 * 2"}, {"function": "hillshade"}]);
shade(&mut s, "raster.functionChain", json!({"layer": "dem", "steps": steps}));
// A step after a hillshade reads its no-data, and makes floats again.
let r = s
.run(
"raster.functionChain",
&json!({"layer": "dem", "steps": [{"function": "hillshade"}, {"function": "threshold", "value": 128}]}),
)
.unwrap();
let after = s.map().layer(r["layer"].as_u64().unwrap() as u32).unwrap().raster.clone().unwrap().data.unwrap();
assert_eq!((after.data_type.as_str(), after.bands[0].stats.count), ("Float32", 1600));
let dir = std::env::temp_dir().join(format!("rustgis-hillshade-bytes-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
for cog in [false, true] {
let tif = dir.join(format!("hill-{cog}.tif"));
s.run("share.exportRaster", &json!({"layer": hill, "path": tif.display().to_string(), "cog": cog})).unwrap();
let back = rustgis_data::raster::read_path(&tif).unwrap();
assert_eq!(
(back.data_type.as_str(), back.nodata, back.bands[0].data.as_slice()),
("UInt8", Some(0.0), d.bands[0].data.as_slice()),
"cog: {cog}"
);
}
std::fs::remove_dir_all(&dir).ok();
}

#[test]
fn exporting_a_raster_as_a_cog() {
let mut s = world();
Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -1252,7 +1252,7 @@ click them on the map:

| Workflow | What it does |
| --- | --- |
| Raster Functions | Slope, aspect, hillshade, NDVI, band math (`(b4-b3)/(b4+b3)`), remapping, thresholds, resampling |
| Raster Functions | Slope, aspect, hillshade (8-bit shades from 1 to 255, with 0 for no data), NDVI, band math (`(b4-b3)/(b4+b3)`), remapping, thresholds, resampling |
| Function Editor | Chains raster functions, each on the previous output |
| Suitability Modeler | Scores rasters 1–10 and adds them with weights |
| Visibility Analysis | A viewshed from a point on an elevation raster, with the Earth's curvature and the refraction of visible light (1/7, as `gdal_viewshed`). The ground along each ray is the cell each point falls in by default; `heights: "bilinear"` (**Ground heights › Between cells (bilinear)** in the Workflows pane) reads it between the cells' centres instead. Both are estimates of the ground between cell centres; between cells is closer on ground that bulges (a hill seen from its top), where the nearest cell can hide cells just inside the horizon. With no radius, or one longer than the raster's diagonal, the viewshed covers the whole raster. The map's **Viewshed N km Here** reads the ground as the Workflows pane's **Ground heights** says. On a raster round the whole Earth the rays don't cross its east–west edge, which the analysis says when the radius reaches it and the way across is shorter: the ground just across the edge is left out, worked out the long way round where the radius reaches the far edge through the raster too (always with no radius), or partly both, near the observer's row and not farther from it. The observer's height must be from 0 to 10,000 m. Ground with no data hides nothing, and next to it bilinear heights take the nearest cell's height; the analysis says how many cells this touched, in a notice. The answer gives both counts (0 when there is nothing to say): `acrossNoData` counts the visible cells whose rays crossed ground with no data (a hidden cell stays hidden whatever that ground is, so it isn't counted, and 0 doesn't mean no ray crossed any); `readNearest` counts every cell, visible or hidden, whose ray took a nearest height next to no data in bilinear mode |
Expand Down
3 changes: 2 additions & 1 deletion docs/user-guide/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -1182,7 +1182,8 @@ exporting it (`layer.addData`, `share.exportRaster`) give the full size as `over
and the MCP server see it too. A GeoTIFF read in place that is added before its overview is in
(the desktop app, the browser) answers `"reading": true`, sized by its header when that is in, and
with `null` for its size and bands until then.
`share.exportRaster` writes an untiled GeoTIFF; with `"cog": true` it writes a cloud-optimised one:
`share.exportRaster` writes an untiled GeoTIFF, with an 8-bit grey band (a hillshade, say) as
bytes and another single band as 32-bit floats; with `"cog": true` it writes a cloud-optimised one:
512-pixel tiles compressed with Deflate after a predictor (the floating-point one for decimal
values, differences along the row for whole numbers, none for classes), overviews (each half the
size of the last, down to one tile), and every directory ahead of the tiles, so a reader fetches the header once and then only the
Expand Down
Loading