The v1 opt-in dedupe in @stitchapi/download (#451) has two documented limitations:
- Keys off
item.url / explicit id, not the resolved request URL. Two { path: '/x' } items sharing a baseUrl are not deduped (their keys differ) — only items given a full url (or the same explicit id) collapse.
- Coarse cancel-under-dedupe. A follower reuses the leader's in-flight promise; cancelling a follower only detaches it (the shared fetch keeps running), and cancelling the leader fails the followers.
Proposal
- Key dedupe off the resolved request target (baseUrl + path + query), so logically-identical items collapse regardless of spelling.
- Ref-count the sharers of an in-flight fetch: cancelling a sharer decrements; the shared request aborts only when the last sharer cancels.
Also: dedupe collapses only items in-flight together today — decide whether a just-finished result is reusable within a short window (probably not, since blobs are uncacheable) and document it.
Introduced alongside #451; P18 in the rig spec.
The v1 opt-in
dedupein@stitchapi/download(#451) has two documented limitations:item.url/ explicitid, not the resolved request URL. Two{ path: '/x' }items sharing abaseUrlare not deduped (their keys differ) — only items given a fullurl(or the same explicitid) collapse.Proposal
Also: dedupe collapses only items in-flight together today — decide whether a just-finished result is reusable within a short window (probably not, since blobs are uncacheable) and document it.
Introduced alongside #451; P18 in the rig spec.