Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 122 additions & 14 deletions crates/analysis/src/rasterops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -464,7 +464,8 @@ impl Heights {

/// The ground at a point of a ray, `(col, row)` in cells (whole numbers at cell centres), and
/// whether it is the nearest cell's, read in place of cells between which a neighbour has
/// no data. `None` where the point's own cell has no data.
/// no data. `None` where the point's own cell has no data: between cells, a neighbour weighed
/// with no data falls back to that cell, so `None` there too only when it has none.
fn at(self, r: &Raster, b: usize, col: f64, row: f64) -> Option<(f64, bool)> {
let (nc, nr) = (col.round() as i64, row.round() as i64);
let nearest = |fell_back: bool| r.value(b, nc, nr).map(|v| (f64::from(v), fell_back));
Expand Down Expand Up @@ -502,10 +503,59 @@ pub struct Viewshed {
pub read_nearest: usize,
}

/// The ray steps a viewshed from `(oc, or)` takes over the window `cols` × `rows` (inclusive): for
/// each cell no farther than `radius_m` (cell centres, `cx` × `cy` metres apart, as the viewshed
/// measures them), its ray's steps, the larger of its column and row offsets. A row's cells within
/// the radius are a run of columns, whose sum has a closed form, so the count is one step a row.
/// It is the sum of the rays' full lengths, so the most steps the viewshed takes: a ray stops
/// where the ground first hides its cell, and a cell with no data walks none (finding out which
/// have none would read them all).
fn ray_steps(observer: (i64, i64), cols: (i64, i64), rows: (i64, i64), cell: (f64, f64), radius_m: f64) -> u128 {
let ((oc, or), (left, right), (first, last), (cx, cy)) = (observer, cols, rows, cell);
// Σ max(x, a) for x in 0..=n.
let up_to = |n: i128, a: i128| -> i128 {
if n < 0 {
0
} else if n <= a {
(n + 1) * a
} else {
(a + 1) * a + (n * (n + 1) - a * (a + 1)) / 2
}
};
let within = |dc: i64, dr: i64| (dc as f64 * cx).hypot(dr as f64 * cy) <= radius_m;
let mut total: i128 = 0;
for row in first..=last {
let dr = row - or;
if !within(0, dr) {
continue;
}
// The farthest column offset within the radius on this row, at most the window's, worked
// out in cells (so a radius near f64's range doesn't overflow when squared) and put right
// at the edge: the corrections only undo rounding, and stay within the window's width.
let span = (oc - left).max(right - oc);
let (rc, yc) = (radius_m / cx, dr as f64 * cy / cx);
let reach = ((rc - yc) * (rc + yc)).max(0.0).sqrt().floor();
Comment thread
pick2510 marked this conversation as resolved.
Resolved
let mut k = reach.clamp(0.0, span as f64) as i64;
while k < span && within(k + 1, dr) {
k += 1;
}
while k > 0 && !within(k, dr) {
k -= 1;
}
// The run's offsets from the observer's column, within the window: the observer is in
// it, so the run takes in its column (offset 0, counted once).
let (west, east) = (i128::from(oc - (oc - k).max(left)), i128::from((oc + k).min(right) - oc));
let a = i128::from(dr.abs());
total += up_to(west, a) + up_to(east, a) - a;
}
// (Never negative; were it, the cap refuses the run rather than letting it through.)
u128::try_from(total).unwrap_or(u128::MAX)
}

/// Cells visible from an observer (1) or not (0) within `radius_m`; outside the radius no-data.
/// The Earth curves as visible light sees it (`SIGHT_RADIUS_M`). The ground along each ray is read
/// as `heights` says; ground with no data hides nothing, and the cells whose rays crossed some are
/// counted.
/// as `heights` says; ground with no data hides nothing, and the visible cells whose rays crossed
/// some are counted ([`Viewshed::through_gaps`]).
pub fn viewshed(r: &Raster, b: usize, observer: Coord, height: f64, radius_m: f64, heights: Heights) -> Result<Viewshed, String> {
viewshed_on(None, r, b, observer, height, radius_m, heights)
}
Expand Down Expand Up @@ -534,15 +584,15 @@ fn viewshed_on(
let rad_r = (radius_m / cy).ceil() as i64;
let (first, last) = (or.saturating_sub(rad_r).max(0), or.saturating_add(rad_r).min(r.height as i64 - 1));
let (left, right) = (oc.saturating_sub(rad_c).max(0), oc.saturating_add(rad_c).min(r.width as i64 - 1));
// Each cell walks a ray back to the observer: bound cells × ray length, not just cells.
let cells = (right - left + 1).saturating_mul(last - first + 1);
let longest = (oc - left).max(right - oc).max(or - first).max(last - or);
// A ray steps a whole cell along its longer axis, so a bilinear point lies on a cell's
// centre line and reads the two cells it falls between ([`Heights::at`] skips a neighbour
// that weighs nothing; next to no data, the nearest cell too) where a nearest one reads one:
// weighed as two.
let per_step = if heights == Heights::Bilinear { 2 } else { 1 };
let ray_work = usize::try_from(cells.saturating_mul(longest / 2 + 1).saturating_mul(per_step)).unwrap_or(usize::MAX);
// Each cell within the radius walks a ray back to the observer, one step a cell along its
// longer axis: the steps counted exactly, a row at a time, not estimated.
let ray_work = usize::try_from(ray_steps((oc, or), (left, right), (first, last), (cx, cy), radius_m).saturating_mul(per_step as u128))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The count is an upper bound, not exact, when the window has no-data cells. The main loop skips a target cell with no data (let Some(zt) = r.value(b, col, row) else { continue };, about line 627), so it walks no ray for it. ray_steps counts every cell within the radius, so it overstates the steps of a DEM with holes.

Failure scenario: a 1,200 × 1,200 DEM from the middle with about half its cells no-data. ray_steps gives about 576M steps, over the 400M one-thread cap, so the run is refused as "takes 576000000 ray steps". The walk would take about 288M steps and pass.

The refusal is conservative, not unsafe. But the PR text and the user guide ("counted exactly before it starts") are wrong for such rasters. Either say "an upper bound, exact where no cell in the window is no-data" in the docs, or skip no-data cells in the count. A test with a holed DEM would pin whichever is chosen.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Fixed in c7f43ac by saying so, not by reading the cells: finding out which cells have no data would read every one before the run. The count is now documented as the most the run takes, exact where every cell has data (the doc comment, the user guide and the PR text), and the refusal says "takes up to N ray steps". On a holed DEM it refuses on the safe side.

.unwrap_or(usize::MAX);
// Each cell walks its own ray and writes only itself, so the rows within the radius are made
// on several threads when the work is large (#1252), and may take as many times the work.
let rows = usize::try_from(last - first + 1).unwrap_or(0);
Expand All @@ -558,7 +608,7 @@ fn viewshed_on(
("the radius", "use a smaller radius")
};
return Err(format!(
"{what} takes {ray_work} ray steps{}, more than the {cap} this computer takes on {threads} thread(s); {fix}{}",
"{what} takes up to {ray_work} ray steps{}, more than the {cap} this computer takes on {threads} thread(s); {fix}{}",
if per_step > 1 { " (a step between cells counted as two)" } else { "" },
crate::parallel::UNCAPPED_HINT
));
Expand Down Expand Up @@ -1712,9 +1762,9 @@ mod tests {

/// A bilinear ray step is weighed as two in the work cap (it reads the two cells it falls
/// between), and a refusal says so: 1,000 × 1,000 cells from the middle on one thread take
/// 251 million nearest steps, under the cap, and 502 million weighed bilinear ones, over it.
/// A radius reaching every cell is refused as the whole raster; one short of the corners,
/// with the same window and steps, as a radius.
/// 333,333,500 nearest steps, under the cap, and twice that weighed bilinear, over it. A
/// radius reaching every cell is refused as the whole raster; one short of the corners, as a
/// radius, with the steps of the cells within it.
#[test]
fn a_bilinear_viewshed_step_is_weighed_as_two_in_the_work_cap() {
let w = 1_000u32;
Expand All @@ -1725,16 +1775,74 @@ mod tests {
let e = viewshed_on(Some(1), &dem, 0, middle, 2.0, 1e9, Heights::Bilinear).unwrap_err();
assert!(
e.starts_with(&format!(
"the whole raster takes 502000000 ray steps (a step between cells counted as two), more than the {MAX_VIEWSHED_WORK} this computer takes on 1 thread(s); give a radius to cover only part of it"
"the whole raster takes up to 666667000 ray steps (a step between cells counted as two), more than the {MAX_VIEWSHED_WORK} this computer takes on 1 thread(s); give a radius to cover only part of it"
)),
"{e}"
);
// 600 of the longer cell side: past every edge (500 cells away), short of the corners
// (about 707 cells of the side's mean away; the cells are longer north–south at 45°).
let (cx, cy) = dem.cell_size_m();
assert!(600.0 * cx.max(cy) < (500.0 * cx).hypot(500.0 * cy));
let e = viewshed_on(Some(1), &dem, 0, middle, 2.0, 600.0 * cx.max(cy), Heights::Bilinear).unwrap_err();
assert!(e.starts_with("the radius takes 502000000 ray steps") && e.contains("; use a smaller radius"), "{e}");
let radius = 600.0 * cx.max(cy);
let e = viewshed_on(Some(1), &dem, 0, middle, 2.0, radius, Heights::Bilinear).unwrap_err();
let steps = 2 * brute_ray_steps((500, 500), (0, 999), (0, 999), (cx, cy), radius);
assert!(
steps < 666_667_000
&& e.starts_with(&format!("the radius takes up to {steps} ray steps"))
&& e.contains("; use a smaller radius"),
"{e}"
);
}

/// Each cell's ray steps within the radius, one at a time.
fn brute_ray_steps(observer: (i64, i64), cols: (i64, i64), rows: (i64, i64), cell: (f64, f64), radius_m: f64) -> u128 {
let mut n = 0u128;
for row in rows.0..=rows.1 {
for col in cols.0..=cols.1 {
let (dc, dr) = (col - observer.0, row - observer.1);
if (dc as f64 * cell.0).hypot(dr as f64 * cell.1) <= radius_m {
n += u128::from(dc.unsigned_abs().max(dr.unsigned_abs()));
}
}
}
n
}

/// The ray steps counted a row at a time are each cell's, counted one by one: an observer in
/// the middle, at a corner, on an edge and outside the window's middle row; square and
/// oblong cells; a radius inside the window, cutting its corners, past it, and on a cell's
/// centre exactly. Cells and radii so large that the squared radius is past f64's range are
/// counted as the rest (they once walked a column at a time towards the radius).
#[test]
fn viewshed_ray_steps_are_counted_exactly() {
let window = ((0, 39), (0, 24));
for observer in [(20, 12), (0, 0), (39, 7), (5, 24)] {
for cell in [(10.0, 10.0), (7.0, 13.0), (30.0, 2.5)] {
for radius in [0.5, 10.0, 35.0, 120.0, 260.0, 1e9, 1e300] {
assert_eq!(
ray_steps(observer, window.0, window.1, cell, radius),
brute_ray_steps(observer, window.0, window.1, cell, radius),
"{observer:?} {cell:?} {radius}"
);
}
}
}
assert_eq!(
ray_steps((0, 0), (0, 49), (0, 0), (1e150, 1e150), 1e155),
brute_ray_steps((0, 0), (0, 49), (0, 0), (1e150, 1e150), 1e155)
);
assert_eq!(
ray_steps((0, 0), (0, 49), (0, 9), (1e182, 1e182), 1e200),
brute_ray_steps((0, 0), (0, 49), (0, 9), (1e182, 1e182), 1e200)
);
for radius in [1.5e151, 1.5e154] {
let (observer, cell) = ((25, 5), (1e150, 1e150));
assert_eq!(
ray_steps(observer, (0, 49), (0, 9), cell, radius),
brute_ray_steps(observer, (0, 49), (0, 9), cell, radius),
"{radius}"
);
}
}

/// Viewshed rows made on several threads see what one thread sees (#1252), with the radius
Expand Down
17 changes: 11 additions & 6 deletions docs/user-guide/analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,17 @@ can take up to about twice as long as a one-thread run at its own (measured on o
Silver 4114; it can be longer on others); in the web app, on one thread, the limits stay as they
are. The error names the limit on this computer and the threads it counted. The limits on memory
(the cells a grid may have, and those on the following pages) hold everywhere. Viewshed has no limit
on its window (a radius past the raster's diagonal is taken as the diagonal), only on its ray steps:
with no radius it covers the whole raster, so on a large one it can be refused where nobody can stop
the run (the web app, a script, the assistant). A smaller radius, or a run from its pane on the
desktop or with `rustgis-cli run`, gets it through. A
tool's own default cell size still follows the one-thread limit, so it makes the same grid on any
computer and from any caller; a finer cell size given is what a larger limit, or none, lets through.
on its window (a radius past the raster's diagonal is taken as the diagonal), only on its ray steps,
counted before it starts as the most it can take: each cell within the radius a step per cell
between it and the observer (two with bilinear heights), though a ray stops where the ground first
hides its cell and a cell with no data walks none. With no radius it covers the whole raster, so
on a large one it can be refused where a run can't be stopped (listed above); a smaller radius, or
a run from its pane on the desktop or with `rustgis-cli run` typed at a terminal, gets it through.
Nothing caps it there, so a whole-raster viewshed of a large DEM can take long: its steps are at
most its cells times the raster's longer side (counted twice with bilinear heights). Its progress
shows, and **Cancel** stops it. A tool's own default cell size still follows the one-thread limit,
so it makes the same grid on any computer and from any caller; a finer cell size given is what a
larger limit, or none, lets through.

From the command line or a script (`gp.run`), a built-in tool's parameter with a list of choices, such as Buffer's
units, takes one of them in any letter case (`"meters"` for `Meters`); any other value is an error
Expand Down
Loading