Split out of #353 (item 3), which asked whether pack:npm's dist/source/** input should be a structured input with mode: "jit". Items 1 and 2 of that issue are fixed; this one is separated because the fix carries a CI trade-off worth deciding on its own.
The premise holds
Measured against this repo with turbo 2.10.9. Two --dry=json runs of pack:npm for one package, with nothing changed between them except the contents of an already-compiled file on disk:
|
compile:ts hash |
pack:npm hash |
| before |
04f614c1a83581f7 |
6037007f25a5bc6c |
after appending a line to dist/source/src/index.js |
04f614c1a83581f7 |
583cdfdfffc9e48f |
pack:npm's key moved while its dependency's key held. A plain glob is therefore hashed from the filesystem as it stands before the run, so pack:npm is keyed to the previous run's compiled output rather than the one it is about to pack. That matches turbo's own framing of jit — it "delays file hash computation until immediately before task execution, after dependencies complete" — and the fact that jit and dependencyOutputs tasks report hash: null under --dry=json precisely because a plain input's hash is knowable upfront.
Why it is worse for consumers than for this repo
Tooling is shielded by accident. Any workspace source edit moves hashOfInternalDependencies (see "Turbo cache miss on workspace edits" in AGENTS.md), which moves pack:npm's key for an unrelated reason and forces the repack.
A consumer resolves these packages externally and gets granular caching, so nothing covers for it there:
- Edit a source file.
compile:ts misses and emits new output; pack:npm's dist/source glob was already hashed against the old output, matching the previous run's key, so it hits and restores the previous tarball.
- On a fresh CI checkout the hole is wider:
dist/source does not exist at hash time, so that input contributes nothing and pack:npm's key is identical across every source state.
dependencyOutputs looks like the better fix
mode: "dependencyOutputs" keys off the intersection of the producing tasks' declared outputs with the globs, rather than off whatever happens to be on disk when the task is dispatched. Now that compile:ts, compile:skills, and pack:npm each declare only what they write, that intersection is exactly the compiled tree plus skills — so the negation globs are no longer needed:
"inputs": [
"$TURBO_ROOT$/LICENSE",
"$TURBO_ROOT$/package.json",
"LICENSE",
"README.md",
{ "globs": ["dist/source/**"], "mode": "dependencyOutputs" },
"package.json"
]
from defaults to the task's direct dependencies, which is already compile:ts (+ compile:skills), so it can be omitted.
Tried against this repo: a no-op re-run still reports FULL TURBO, and a source edit correctly misses. Caching is intact.
The cost, and why this is its own issue
Both jit and dependencyOutputs make the cache key unknowable until runtime. Under --dry=json, pack:npm then reports an empty hash and cache.status: MISS:
taskId hash cacheStatus
compile:ts 04f614c1a83581f7 HIT
pack:npm MISS
The turbo-run composite action decides whether to skip pnpm install by requiring every task with outputs to report HIT. So adopting either mode means pnpm install runs on every CI job whose graph contains pack:npm — build:ci, build, and test:e2e. There is no way to recover it: the hash genuinely is not knowable at dry-run time, and treating an empty hash as a hit would be a guess.
AGENTS.md already notes the install-skip mostly pays off "on root-only edits and re-runs"; this would remove that remaining payoff for those jobs.
Split out of #353 (item 3), which asked whether
pack:npm'sdist/source/**input should be a structured input withmode: "jit". Items 1 and 2 of that issue are fixed; this one is separated because the fix carries a CI trade-off worth deciding on its own.The premise holds
Measured against this repo with turbo 2.10.9. Two
--dry=jsonruns ofpack:npmfor one package, with nothing changed between them except the contents of an already-compiled file on disk:compile:tshashpack:npmhash04f614c1a83581f76037007f25a5bc6cdist/source/src/index.js04f614c1a83581f7583cdfdfffc9e48fpack:npm's key moved while its dependency's key held. A plain glob is therefore hashed from the filesystem as it stands before the run, sopack:npmis keyed to the previous run's compiled output rather than the one it is about to pack. That matches turbo's own framing ofjit— it "delays file hash computation until immediately before task execution, after dependencies complete" — and the fact thatjitanddependencyOutputstasks reporthash: nullunder--dry=jsonprecisely because a plain input's hash is knowable upfront.Why it is worse for consumers than for this repo
Tooling is shielded by accident. Any workspace source edit moves
hashOfInternalDependencies(see "Turbo cache miss on workspace edits" inAGENTS.md), which movespack:npm's key for an unrelated reason and forces the repack.A consumer resolves these packages externally and gets granular caching, so nothing covers for it there:
compile:tsmisses and emits new output;pack:npm'sdist/sourceglob was already hashed against the old output, matching the previous run's key, so it hits and restores the previous tarball.dist/sourcedoes not exist at hash time, so that input contributes nothing andpack:npm's key is identical across every source state.dependencyOutputslooks like the better fixmode: "dependencyOutputs"keys off the intersection of the producing tasks' declaredoutputswith the globs, rather than off whatever happens to be on disk when the task is dispatched. Now thatcompile:ts,compile:skills, andpack:npmeach declare only what they write, that intersection is exactly the compiled tree plus skills — so the negation globs are no longer needed:fromdefaults to the task's direct dependencies, which is alreadycompile:ts(+compile:skills), so it can be omitted.Tried against this repo: a no-op re-run still reports
FULL TURBO, and a source edit correctly misses. Caching is intact.The cost, and why this is its own issue
Both
jitanddependencyOutputsmake the cache key unknowable until runtime. Under--dry=json,pack:npmthen reports an empty hash andcache.status: MISS:The
turbo-runcomposite action decides whether to skippnpm installby requiring every task with outputs to reportHIT. So adopting either mode meanspnpm installruns on every CI job whose graph containspack:npm—build:ci,build, andtest:e2e. There is no way to recover it: the hash genuinely is not knowable at dry-run time, and treating an empty hash as a hit would be a guess.AGENTS.mdalready notes the install-skip mostly pays off "on root-only edits and re-runs"; this would remove that remaining payoff for those jobs.