feat: Add uniflight crate for duplicate request coalescing #118

schgoo · 2025-12-10T15:40:08Z

Adds uniflight, a crate for coalescing duplicate async tasks into a single execution. When multiple tasks request the same work (identified by a key), only the leader performs the work while followers wait and receive a clone of the result.

Handles cancellation and panic gracefully (followers become the new leader)
Includes comprehensive module documentation
Includes example demonstrating cache thundering herd prevention

schgoo · 2025-12-10T15:41:29Z

@microsoft-github-policy-service agree company="Microsoft"

codecov · 2025-12-10T15:45:50Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (241a287) to head (cb49236).

Additional details and impacted files

@@           Coverage Diff            @@
##             main     #118    +/-   ##
========================================
  Coverage   100.0%   100.0%            
========================================
  Files         105      106     +1     
  Lines        6767     6902   +135     
========================================
+ Hits         6767     6902   +135

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…niflight

geeknoid · 2025-12-10T18:02:23Z

I like the name :-)

geeknoid

The logo.png file isn't rendering for me. Is this a valid png file? I tried even going to your repo & branch, and GitHub won't show the image to me.

schgoo · 2025-12-11T14:49:01Z

@geeknoid I hadn't uploaded the logo yet. I just added it

crates/uniflight/examples/cache_population.rs

crates/uniflight/Cargo.toml

crates/uniflight/src/lib.rs

Co-authored-by: Martin Taillefer <geeknoid@users.noreply.github.com>

crates/uniflight/src/lib.rs

crates/uniflight/examples/cache_population.rs

crates/uniflight/src/lib.rs

crates/uniflight/benches/comparison.rs

crates/uniflight/src/lib.rs

…xports

…flight

…niflight

geeknoid · 2026-01-23T21:36:46Z

crates/uniflight/Cargo.toml

+
+[package.metadata.cargo_check_external_types]
+allowed_external_types = [
+    "thread_aware::cell::builtin::PerProcess",


Why do we need to reexport these?

geeknoid · 2026-01-23T21:38:55Z

crates/uniflight/src/lib.rs

+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+//! Coalesces duplicate async tasks into a single execution.


I suggest using this line as the description in Cargo.toml and in the repo-level README file.

geeknoid · 2026-01-23T21:43:33Z

crates/uniflight/src/lib.rs

+//! `Merger` handles task cancellation and panics gracefully:
+//!
+//! - If the leader task is cancelled or dropped, a follower becomes the new leader
+//! - If the leader task panics, a follower becomes the new leader and executes its work


This feels wrong. Shouldn't the panic propagate to all followers?

geeknoid · 2026-01-23T21:45:09Z

crates/uniflight/src/lib.rs

+//!
+//! - If the leader task is cancelled or dropped, a follower becomes the new leader
+//! - If the leader task panics, a follower becomes the new leader and executes its work
+//! - Followers that join before the leader completes receive the cached result


"receive the cached result" confuses me. Can we just say "receive the value the leader returns"?

geeknoid · 2026-01-23T21:53:08Z

crates/uniflight/src/lib.rs

+//!
+//! # Memory Management
+//!
+//! Completed entries are automatically removed from the internal map when the last caller


Hmmm. Seems like having a TTL here might be desirable? Keep an entry around for N msec in case someone asks for the result? This can turn into a simple short-lived cache.

The cargo-rank app I'm working on issues sometimes thousands of requests over the course of a few minutes as it runs. I am planning to adopt this crate once it's available. But if I want to truly be efficient, I'm going to need to implement a cache in front of uniflight to avoid issuing the same request multiple times over the course of those minutes. if I could tell uniflight to hold on to the keys & values for a bit, then it could totally avoid me doing the work N times.

But this leads to Hmmm #2. Is uniflight just a cache with an oddball TTL? Instead of holding on to items for a specific time period, it throws out cached data is nobody is actively waiting for it. Why should I use uniflight instead of a cache of tasks? I can achieve the same semantics by setting a TTL of 0, and get more flexible semantics if I increase the TTL.

geeknoid · 2026-01-23T21:58:41Z

crates/uniflight/src/lib.rs

+{
+    fn default() -> Self {
+        Self {
+            inner: TaArc::new(DashMap::new),


You probably should override the hasher, or allow the caller to override the hasher. The default hasher used by DashMap is the default system hasher which is really slow. Consider using ahash instead, or one of the other many fast hashers.

geeknoid · 2026-01-23T22:23:53Z

crates/uniflight/src/lib.rs

+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,
+    S: Strategy + Send + Sync,


Suggested change

S: Strategy + Send + Sync,

S: Strategy,

Probably applies in other places too.

geeknoid · 2026-01-23T22:29:54Z

crates/uniflight/src/lib.rs

+impl<K, T, S> Default for Merger<K, T, S>
+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,


Suggested change

T: Clone + Send + Sync + 'static,

T: Send + Sync + 'static,

geeknoid · 2026-01-23T22:31:14Z

crates/uniflight/src/lib.rs

+impl<K, T, S> Merger<K, T, S>
+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,


Suggested change

T: Clone + Send + Sync + 'static,

T: Send + Sync + 'static,

geeknoid · 2026-01-23T22:31:28Z

crates/uniflight/src/lib.rs

+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,
+    S: Strategy + Send + Sync,


Suggested change

S: Strategy + Send + Sync,

S: Send + Sync,

geeknoid · 2026-01-23T22:31:55Z

crates/uniflight/src/lib.rs

+impl<K, T> Merger<K, T, PerProcess>
+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,


Suggested change

T: Clone + Send + Sync + 'static,

T: Send + Sync + 'static,

geeknoid · 2026-01-23T22:32:21Z

crates/uniflight/src/lib.rs

+impl<K, T> Merger<K, T, PerNuma>
+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,


Suggested change

T: Clone + Send + Sync + 'static,

T: Send + Sync + 'static,

geeknoid · 2026-01-23T22:32:37Z

crates/uniflight/src/lib.rs

+impl<K, T> Merger<K, T, PerCore>
+where
+    K: Hash + Eq + Send + Sync + 'static,
+    T: Clone + Send + Sync + 'static,


Suggested change

T: Clone + Send + Sync + 'static,

T: Send + Sync + 'static,

geeknoid · 2026-01-23T22:34:33Z

crates/uniflight/src/lib.rs

+
+impl<K, T, S: Strategy> Merger<K, T, S>
+where
+    K: Hash + Eq,


Remove this trait bound, it's not needed.

geeknoid · 2026-01-23T22:40:16Z

crates/uniflight/src/lib.rs

+impl<K, T, S> Merger<K, T, S>
+where
+    K: Hash + Eq + Send + Sync,
+    T: Clone + Send + Sync,


Remove Clone from here, and put it as a bound on the execute method instead.

schgoo added 2 commits December 10, 2025 10:22

Initial commit of uniflight

e5fbae3

Update docs

9eb29c0

Merge branch 'main' into uniflight

dc9c254

schgoo changed the title ~~Add uniflight (name pending) crate for duplicate request coalescing~~ build: Add uniflight (name pending) crate for duplicate request coalescing Dec 10, 2025

schgoo added 2 commits December 10, 2025 10:49

Fix clippy

e29c3f4

Merge branch 'uniflight' of https://github.com/schgoo/oxidizer into u…

eac8971

…niflight

geeknoid reviewed Dec 10, 2025

View reviewed changes

Add logo

3f508a5

schgoo added 2 commits December 11, 2025 09:49

Merge with main

b692e0e

Fix formatting

ff628d0

martintmk reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/examples/cache_population.rs Outdated Show resolved Hide resolved

Update favicon and code coverage

8aa3ffb

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/Cargo.toml Outdated Show resolved Hide resolved

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/Cargo.toml Outdated Show resolved Hide resolved

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

geeknoid reviewed Dec 11, 2025

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

schgoo and others added 2 commits December 11, 2025 16:27

Update crates/uniflight/Cargo.toml

6ce6e11

Co-authored-by: Martin Taillefer <geeknoid@users.noreply.github.com>

Update crates/uniflight/Cargo.toml

76e0ede

Co-authored-by: Martin Taillefer <geeknoid@users.noreply.github.com>

ralfbiedert reviewed Dec 12, 2025

View reviewed changes

Move tests to separate file

e6e7458

schgoo changed the title ~~build: Add uniflight (name pending) crate for duplicate request coalescing~~ feat: Add uniflight (name pending) crate for duplicate request coalescing Dec 16, 2025

schgoo added 2 commits December 16, 2025 16:54

Merge with main

1ca1d3e

Use tick

bf25a1a

Undo ToC changes to README.md

5345752

ralfbiedert reviewed Jan 21, 2026

View reviewed changes

crates/uniflight/benches/comparison.rs Outdated Show resolved Hide resolved

ralfbiedert reviewed Jan 21, 2026

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

ralfbiedert reviewed Jan 21, 2026

View reviewed changes

crates/uniflight/src/lib.rs Outdated Show resolved Hide resolved

schgoo and others added 8 commits January 22, 2026 10:19

Improve benchmarks for incremental development. Remove thread_aware e…

ab92d59

…xports

Update readme

f19b674

Add thread aware types to external type check

168acbb

Clippy, fmt

3792000

Merge branch 'main' into uniflight

80d0766

Clippy, readme

ae9ce04

Merge branch 'main' of https://github.com/microsoft/oxidizer into uni…

d6dd361

…flight

Merge branch 'uniflight' of https://github.com/schgoo/oxidizer into u…

cb49236

…niflight

schgoo requested review from geeknoid and ralfbiedert January 23, 2026 16:36

schgoo enabled auto-merge (squash) January 23, 2026 16:36

geeknoid reviewed Jan 23, 2026

View reviewed changes

crates/uniflight/src/lib.rs

impl<K, T, S: Strategy> Merger<K, T, S>

where

K: Hash + Eq,

Copy link

Member

geeknoid Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this trait bound, it's not needed.

geeknoid reviewed Jan 23, 2026

View reviewed changes

feat: Add uniflight crate for duplicate request coalescing #118

Are you sure you want to change the base?

feat: Add uniflight crate for duplicate request coalescing #118

Uh oh!

Conversation

schgoo commented Dec 10, 2025

Uh oh!

schgoo commented Dec 10, 2025

Uh oh!

codecov bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

geeknoid commented Dec 10, 2025

Uh oh!

geeknoid left a comment

Choose a reason for hiding this comment

Uh oh!

schgoo commented Dec 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Dec 10, 2025 •

edited

Loading