Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions crates/tiles/src/cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1722,6 +1722,31 @@ pub(crate) fn fetch_worker(url: String, done: Done) {
native::fetch(url, done, &[]);
}

/// A read the map waits for that runs outside the fetch queue (an MBTiles file's metadata, read
/// on its own thread): counted in [`in_flight`] from now until it is dropped, which wakes the map
/// as a fetch's end does. A complete render takes nothing under way for nothing to come, so a
/// read it doesn't count is one it stops waiting for at once (#1801). Native only: in the browser
/// the metadata is read in place.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) struct UnderWay(());

#[cfg(not(target_arch = "wasm32"))]
impl UnderWay {
/// Counts one more read under way, until the returned value is dropped.
pub(crate) fn begin() -> Self {
IN_FLIGHT.fetch_add(1, Ordering::Relaxed);
Self(())
}
}

#[cfg(not(target_arch = "wasm32"))]
impl Drop for UnderWay {
/// The read is over: no longer counted, and the map draws again, as at a fetch's end.
fn drop(&mut self) {
finished();
}
}

/// Reads a range of a local file (`<path>#rustgis-range=a-b`) on the fetch workers, counted like
/// [`fetch_counted`]; never through an installed fetcher, which only knows web addresses.
pub(crate) fn fetch_local(url: String, done: Done) {
Expand Down
28 changes: 27 additions & 1 deletion crates/tiles/src/mbtiles.rs
Original file line number Diff line number Diff line change
Expand Up @@ -322,14 +322,19 @@ pub fn info_ready(archive: &str, timeout: Duration) -> Result<Option<Arc<Info>>,
let address = archive.to_string();
let worker_key = key.clone();
let still = crate::cache::current_still();
// Under way from before the thread starts until what it read is kept (or the thread
// fails to start), so a render waiting for the map's reads waits for this one too.
let under_way = crate::cache::UnderWay::begin();
let spawned = std::thread::Builder::new().name("rustgis-mbtiles-info".into()).spawn(move || {
crate::cache::with_optional_still(still, || {
let result = crate::cache::check_still(&crate::cache::current_still()).and_then(|_| read_info(&address));
remember_info(&address, &result, epoch);
});
// Over once what it read is kept, before its waiters wake: whoever sees it over
// sees the metadata, and a caller that waited sees it no longer counted.
drop(under_way);
pending.lock().unwrap_or_else(|e| e.into_inner()).remove(&worker_key);
ready.notify_all();
Comment on lines +333 to 337

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible lost wake-up (confidence: moderate, ~40%). drop(under_way) runs finished(), which bumps the generation, decrements IN_FLIGHT and wakes the map, all before pending.remove and ready.notify_all(). The old trailing crate::cache::notify() (which came after the removal) is also gone, so the last wake now comes before the key leaves the pending set.

When remember_info doesn't store the result (the epoch moved, or check_still refused, as the test at mbtiles.rs:475 arranges), a render woken by that bump can run in the gap and see the key still pending. info_ready then returns Ok(None) and nothing wakes the map after the removal. The export loop recovers because its next render re-spawns the read. The interactive map has no such recovery in this path.

Dropping the count last fixes the ordering: anyone who sees in_flight() == 0 also sees the read off the pending set, and the final wake follows the removal.

Suggested change
// Over once what it read is kept, before its waiters wake: whoever sees it over
// sees the metadata, and a caller that waited sees it no longer counted.
drop(under_way);
pending.lock().unwrap_or_else(|e| e.into_inner()).remove(&worker_key);
ready.notify_all();
// Over last: once the read is off the pending set and its waiters are woken, so the
// wake that ends the count follows the removal, and whoever sees it over sees both.
pending.lock().unwrap_or_else(|e| e.into_inner()).remove(&worker_key);
ready.notify_all();
drop(under_way);

crate::cache::notify();
});
if let Err(error) = spawned {
all.remove(&key);
Expand Down Expand Up @@ -478,6 +483,8 @@ mod tests {
let _ = crate::cache::with_still(still, || info_ready("/reentrant-pending-read.mbtiles", Duration::ZERO));
assert!(checks.load(Ordering::SeqCst) >= 2, "the post-wait check must execute");
assert!(available.load(Ordering::SeqCst), "Still ran with pending metadata reads locked");
// The read it started is counted until it ends: over before the next test's turn.
crate::cache::wait(Duration::from_secs(10));
}

/// The metadata worker inherits its initiating transport context instead of reading at root.
Expand Down Expand Up @@ -580,6 +587,25 @@ mod tests {
assert!(!metadata_stalled(&reads, &next), "capacity can recover when the OS returns");
}

/// A metadata read counts as under way while its worker runs, so an export's complete render
/// waits for it instead of finding nothing in flight and giving up at once (#1801).
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn a_metadata_read_counts_as_under_way_while_it_runs() {
let _turn = crate::cache::turn();
let during = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let seen = during.clone();
let still: crate::cache::Still = Arc::new(move || {
if std::thread::current().name() == Some("rustgis-mbtiles-info") {
seen.fetch_max(crate::cache::in_flight(), Ordering::SeqCst);
}
true
});
let address = "/counted-metadata-read.mbtiles";
let _ = crate::cache::with_still(still, || info_ready(address, Duration::from_secs(10)));
assert!(during.load(Ordering::SeqCst) >= 1, "the worker's own read wasn't counted as under way");
}

#[test]
fn addresses() {
assert_eq!(parse_url("mbtiles:3/1/2@/data/a.mbtiles"), Some((3, 1, 2, "/data/a.mbtiles")));
Expand Down
Loading