Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions apps/rustgis-web/src/web.rs
Original file line number Diff line number Diff line change
Expand Up @@ -545,7 +545,8 @@ fn read_bitmap(bitmap: &web_sys::ImageBitmap, w: u32, h: u32) -> Result<Vec<u8>,
/// How long a tile or document fetch may go without anything coming in: the answer's headers,
/// or the next part of its body. A slow but steady answer is never cut off; one that stalls is
/// given up, so it doesn't stay under way forever, holding its server's place in the queue.
/// Downloads the user asked for have none.
/// So is a picture the pop-up fetches. Downloads the user asked for have none
/// (`cache::gives_up_when_stalled`).
const STALL_MS: i32 = 30_000;

/// What a step of a fetch waits for, or `None` once it has gone [`STALL_MS`] with nothing (the
Expand Down Expand Up @@ -619,6 +620,7 @@ async fn fetch_reached(
&still,
|| async {
let window = web_sys::window().ok_or("no window")?;
let gives_up = cache::gives_up_when_stalled(url, ceiling);
// A data download carries its caller's budget (at most MAX_DOWNLOAD); other fetches use 64 MB.
let (url, limit) = cache::split_download_limit(url);
let max = limit.map(|n| n as u64).unwrap_or(64 << 20);
Expand Down Expand Up @@ -660,8 +662,8 @@ async fn fetch_reached(
(None, None) => None,
};
let init = web_sys::RequestInit::new();
// A tile or document that stalls is given up (see `STALL_MS`).
let abort = if limit.is_none() { web_sys::AbortController::new().ok() } else { None };
// A tile, document or picture that stalls is given up (see `STALL_MS`).
let abort = if gives_up { web_sys::AbortController::new().ok() } else { None };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs / PR description (confidence: high). Further down this request (line 687, outside this diff) sets RequestRedirect::Manual for a request without a login, or Error with one. check_redirect (around line 988) then turns the opaque redirect into an error. So in the web app a picture download never follows a redirect. That matches docs/privacy.md and docs/user-guide/editing.md, which say the web app refuses one.

The PR body still says "In the web app the browser follows redirects and resolves names", and the earlier reply on the redirect thread says "In the web app the browser's fetch follows it". Both are out of date. Please correct the PR description before merge, since it is the privacy description. No code change is needed; refusing is the safer behaviour.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, corrected the PR description: in the web app a picture download refuses a redirect, and the browser resolves names, so only the address as written is checked there. Also added a correction to the earlier reply on the redirect thread.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude autofix (unclear): The code matches this comment: the web fetch sets RequestRedirect::Manual or Error (apps/rustgis-web/src/web.rs:687), and check_redirect (web.rs:988, called at web.rs:726 and 958) refuses the opaque redirect. The PR description still needs correcting, which is for the author to edit; this change does not touch it.

if let Some(a) = &abort {
init.set_signal(Some(&a.signal()));
}
Expand Down
150 changes: 147 additions & 3 deletions crates/tiles/src/cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ fn lock() -> std::sync::MutexGuard<'static, Cache> {
}

static GENERATION: AtomicU64 = AtomicU64::new(0);
static CLEARS: AtomicU64 = AtomicU64::new(0);
static IN_FLIGHT: AtomicUsize = AtomicUsize::new(0);

fn fetcher() -> &'static Mutex<Option<Arc<Fetcher>>> {
Expand Down Expand Up @@ -379,6 +380,13 @@ pub fn generation() -> u64 {
GENERATION.load(Ordering::Relaxed)
}

/// How many times [`clear`] has run. What was read through the downloads before it and is kept
/// elsewhere (the identify pop-up's pictures) goes too: it may have been read with a login that
/// has since ended.
pub fn clears() -> u64 {
CLEARS.load(Ordering::Relaxed)
}

/// Consumes every elapsed deadline together and returns the next one to rearm. Keeping the
/// clock as an input lets the ordering contract be tested without scheduler deadlines.
fn take_due(alarms: &mut std::collections::BTreeSet<u64>, now: u64) -> (bool, Option<u64>) {
Expand Down Expand Up @@ -681,7 +689,10 @@ pub fn clear() {
crate::cog::forget();
crate::warp::forget();
rustgis_data::source::remote::forget();
GENERATION.fetch_add(1, Ordering::Relaxed);
CLEARS.fetch_add(1, Ordering::Relaxed);
// The map draws again (and the pop-up's pictures are forgotten): the waker wakes an idle app,
// as when a fetch ends, so a clear from a worker thread isn't left until the next input.
notify();
}

/// The resource a cache key reads: its address without a download budget, a byte range or a
Expand Down Expand Up @@ -1066,6 +1077,14 @@ pub fn split_download_limit(url: &str) -> (&str, Option<usize>) {
}
}

/// Whether the browser gives up a fetch of `url` that stalls, as an installed fetcher is handed it
/// with `ceiling`. Tiles and documents are given up. So is a download kept to internet addresses
/// ([`download_public_limited`]): its address comes from data, like a picture for the pop-up.
/// A download the user named is not given up.
pub fn gives_up_when_stalled(url: &str, ceiling: crate::address::Reach) -> bool {
split_download_limit(url).1.is_none() || ceiling == crate::address::Reach::Public
}

fn download_address(url: &str, max: usize) -> String {
if max >= MAX_DOWNLOAD { format!("{DOWNLOAD}{url}") } else { format!("{LIMITED_DOWNLOAD}{max}:{url}") }
}
Expand Down Expand Up @@ -1238,9 +1257,32 @@ pub fn download_detailed(url: &str, max: usize, done: DoneDetailed) {
download_detailed_within(url, max, DOWNLOAD_TIMEOUT, &[], done);
}

/// Like [`download_limited`], connecting natively only to internet addresses (as
/// [`download_public_within`], off the frame): a name, or a redirect to one, that resolves to this
/// computer or the local network is refused when it is resolved. For an address that comes from
/// data rather than the user (a picture named in a feature's values), which mustn't be able to
/// reach local services. An installed fetcher (the browser's) is handed the internet-only ceiling.
pub fn download_public_limited(url: &str, max: usize, done: Done) {
download_reaching(url, max, DOWNLOAD_TIMEOUT, &[], Reaching::Public, Box::new(move |r| done(r.map_err(|f| f.message))));
}

/// Which addresses a native download may connect to.
#[derive(Clone, Copy)]
enum Reaching {
/// Wherever the address's name resolves (the user named it).
Any,
/// Only internet addresses, judged where names resolve and redirects lead.
Public,
}

/// Like [`download_detailed`], giving up after `timeout` natively; a status in `omit` isn't
/// recorded as a failure.
fn download_detailed_within(url: &str, max: usize, timeout: std::time::Duration, omit: &'static [u16], done: DoneDetailed) {
download_reaching(url, max, timeout, omit, Reaching::Any, done);
}

/// [`download_detailed_within`], natively connecting only where `reaching` allows.
fn download_reaching(url: &str, max: usize, timeout: std::time::Duration, omit: &'static [u16], reaching: Reaching, done: DoneDetailed) {
if let Err(e) = check_current() {
done(Err(Failure::new(url, e)));
return;
Expand All @@ -1264,11 +1306,17 @@ fn download_detailed_within(url: &str, max: usize, timeout: std::time::Duration,
match f {
Some(f) => f(
marked.clone(),
crate::address::Reach::Local,
match reaching {
Reaching::Any => crate::address::Reach::Local,
Reaching::Public => crate::address::Reach::Public,
},
Box::new(move |r| done(r.map(|fetched| fetched.bytes).map_err(|e| Failure::new(&marked, e)))),
omit,
),
None => native::download_within(marked, timeout, omit, done),
None => match reaching {
Reaching::Any => native::download_within(marked, timeout, omit, done),
Reaching::Public => native::download_public_within(marked, timeout, done),
},
}
}

Expand Down Expand Up @@ -2393,6 +2441,20 @@ mod native {
});
}

/// Like [`download_within`], connecting only to internet addresses ([`public_agent`]).
pub(super) fn download_public_within(url: String, timeout: std::time::Duration, done: DoneDetailed) {
let capture = Capture::current();
let still = super::current_still();
std::thread::spawn(move || {
super::with_optional_still(still, || {
let _scope = capture.enter();
let r =
super::check_current().map_err(Failure::from).and_then(|_| get_detailed_omitting(&public_agent(timeout), &url, &[]));
done(r);
})
});
}

/// Dispatches each queued read through the pool matching its inherited IP ceiling.
fn work() {
let agent = shared_agent();
Expand Down Expand Up @@ -2764,6 +2826,10 @@ mod native {
done(Err(crate::failure::Failure::new(&url, message)));
}

pub(super) fn download_public_within(url: String, timeout: std::time::Duration, done: DoneDetailed) {
download_within(url, timeout, &[], done);
}

pub(super) fn post(url: String, _body: String, done: Done) {
done(Err(format!("{url}: this build has no way to send requests")));
}
Expand Down Expand Up @@ -3335,6 +3401,84 @@ mod tests {
assert!(err.contains("resolves to this computer or the local network"), "{err}");
}

/// The same off the frame ([`download_public_limited`] without a fetcher installed, called
/// directly: other tests here install theirs): one that resolves locally is refused there.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn public_downloads_off_the_frame_refuse_names_that_resolve_locally() {
let _turn = turn();
let (tx, rx) = std::sync::mpsc::channel();
let url = download_address("http://localhost:9/picture.png", 1024);
native::download_public_within(url, std::time::Duration::from_secs(5), Box::new(move |r| drop(tx.send(r))));
let err = rx.recv_timeout(std::time::Duration::from_secs(30)).unwrap().unwrap_err().message;
assert!(err.contains("resolves to this computer or the local network"), "{err}");
}

/// A public download asked for by a request that has since ended sends nothing, as any
/// download off the frame doesn't.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn public_downloads_off_the_frame_send_nothing_once_their_request_ended() {
let (tx, rx) = std::sync::mpsc::channel();
let url = download_address("https://example.org/picture.png", 1024);
with_still(Arc::new(|| false), || {
native::download_public_within(url, std::time::Duration::from_secs(5), Box::new(move |r| drop(tx.send(r))));
});
let err = rx.recv_timeout(std::time::Duration::from_secs(30)).unwrap().unwrap_err().message;
assert!(err.contains("cancelled before it was sent"), "{err}");
}

/// An installed fetcher (the browser's) is handed the internet-only ceiling for a public
/// download, and an answer that came from the local network is refused.
#[test]
fn public_downloads_through_a_fetcher_keep_the_public_ceiling() {
use crate::address::{Origin, Reach};
let _turn = turn();
let ceilings = Arc::new(Mutex::new(Vec::new()));
let sent = ceilings.clone();
set_fetcher_with_origin(move |_, ceiling, done, _| {
sent.lock().unwrap().push(ceiling);
done(Ok(Fetched { bytes: vec![1], origin: Origin { url: "https://pictures.example/a.png".into(), reach: Reach::Local } }));
});
let (tx, rx) = std::sync::mpsc::channel();
let public = tx.clone();
download_public_limited("https://pictures.example/a.png", 1024, Box::new(move |r| drop(public.send(r))));
let err = rx.recv_timeout(std::time::Duration::from_secs(30)).unwrap().unwrap_err();
assert!(err.contains("exceeds its request's network reach"), "{err}");
download_limited("https://pictures.example/a.png", 1024, Box::new(move |r| drop(tx.send(r))));
assert_eq!(rx.recv_timeout(std::time::Duration::from_secs(30)).unwrap().unwrap(), vec![1]);
assert_eq!(*ceilings.lock().unwrap(), [Reach::Public, Reach::Local]);
}

/// A picture for the pop-up is given up in the browser when it stalls, as a tile is; a
/// download the user named, with or without a smaller budget, is not.
#[test]
fn only_downloads_the_user_named_wait_out_a_stall() {
let _turn = turn();
let handed = Arc::new(Mutex::new(Vec::new()));
let sent = handed.clone();
set_fetcher_with_origin(move |url, ceiling, done, _| {
sent.lock().unwrap().push(gives_up_when_stalled(&url, ceiling));
done(Err("not fetched".into()));
});
let (tx, rx) = std::sync::mpsc::channel::<()>();
let answered = || -> Done {
let tx = tx.clone();
Box::new(move |_| {
let _ = tx.send(());
})
};
let url = "https://pictures.example/a.png";
download_public_limited(url, 1024, answered());
download_limited(url, 1024, answered());
download(url, answered());
for _ in 0..3 {
rx.recv_timeout(std::time::Duration::from_secs(30)).unwrap();
}
assert_eq!(*handed.lock().unwrap(), [true, false, false]);
assert!(gives_up_when_stalled("https://tiles.example/1/2/3.png", crate::address::Reach::Local));
}

#[test]
fn announced_tiff_header_lengths() {
assert_eq!(tiff_header_length(" 16384 "), Some(16384));
Expand Down
2 changes: 1 addition & 1 deletion crates/ui-egui/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ unicode-normalization = "0.1"
sys-locale = { version = "0.3", features = ["js"] }

# Pictures kept with features (photo thumbnails), shown in the identify popup.
image = { workspace = true, features = ["jpeg"] }
image = { workspace = true, features = ["jpeg", "webp"] }

[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
rfd = { workspace = true }
Expand Down
1 change: 1 addition & 0 deletions crates/ui-egui/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ mod panes;
mod pluginpanel;
mod pluginstart;
mod pluginsui;
mod popupmedia;
mod portalsui;
#[cfg(not(target_arch = "wasm32"))]
mod readjob;
Expand Down
Loading
Loading