A failed command is forgotten: the next build reports "Nothing to do" and exits 0

[typeless]

Summary

A command that exits nonzero is not remembered. If its declared outputs exist on disk when the next build starts, that build reports "Nothing to do (up to date)" and exits 0 — so the failure disappears and the tree is left holding whatever the failed run produced, or whatever a previous successful run left behind.

Reproduced

mkdir p && cd p && touch Tupfile.ini
echo ORIGINAL > src.txt
printf ': src.txt |> cp %%f %%o |> out.txt\n' > Tupfile
putup configure -B . && putup -B .            # out.txt = ORIGINAL

echo "NEW CONTENT" > src.txt
printf ': src.txt |> cp %%f %%o && false |> out.txt\n' > Tupfile   # command now fails

build 2:  [.] Build completed: 0 commands (1 failed)     exit 1     out.txt = NEW CONTENT
build 3:  [.] Nothing to do (up to date).                exit 0     out.txt = NEW CONTENT

Build 3 reports success for a command whose only execution exited 1.

Why it matters

putup -k in CI, followed by any later putup, reports green. I hit this while iterating on a compile error: a build with failures was followed by "Nothing to do", and the link had silently reused stale .o files from the previous successful build. It took a deliberate probe to notice, because nothing in the second build's output suggests anything is wrong.

The control cases both behave correctly, which localizes it:

• A failing command that produces no output keeps failing on every subsequent run (0 commands (1 failed), exit 1) — retry scheduling itself is fine.
• Without -k, a single failing command also keeps failing, because the build aborts before recording anything.

So the trigger is: the run completed (with -k, or with the failure late in the schedule) and the failed command's outputs exist.

Suspected mechanism

Not verified — the observed behaviour is consistent with the index recording the command's input state after a run in which that command failed, so the next build's change detection sees nothing to do. Exit status does not appear to be part of the recorded state.

Suggested direction

A command's recorded state should not advance unless it succeeded. Either skip index update for failed commands, or persist a failed marker that forces a re-run regardless of output presence.

Provenance

Found incidentally while iterating on #185's compile errors. Reproduced from scratch with the recipe above before filing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

[typeless]

Root cause: #189 — the index has no single definition of a command's recorded state, so each category needs its record/compare/route legs wired by hand. This issue is one of the categories where a leg is missing.

[typeless]

Bisect result: first bad commit is 6ae36d1c2

6ae36d1c2  2025-12-29  Fix -k keep-going flag not saving successful outputs
parent     26acb43a4   GOOD 5/5
6ae36d1c2              BAD  5/5

Bisected over c9caa5bad..fc14f89de (634 commits). This is a genuine regression — the
behaviour was correct before this commit and has been wrong ever since. It shipped in
v1.0.0 and every release since.

The change

The whole diff to src/cli/cmd_build.cpp is three lines:

-    if (stats.failed_jobs == 0 && !opts.dry_run) {
+    if (!opts.dry_run) {
+        // Save index even after partial failures - successful outputs are recorded
+        // so they won't be rebuilt. Failed outputs don't exist, so stat will fail
+        // and they'll be detected as changed on next build.

The commit message states the premise directly:

Failed outputs don't exist on disk, so stat fails and they're detected as changed.

That holds only when the failed command produced nothing. cp %f %o && false writes its
output and then fails, so the stat succeeds, the hash matches what the index just
recorded, and the next build sees nothing to do. This is exactly the control-case split in
the issue: "a failing command that produces no output keeps failing on every subsequent
run" — that is the case the premise was true for.

Confirms the suspected mechanism in the issue ("the index recording the command's input
state after a run in which that command failed"), and localises it to the one commit that
made the index write unconditional.

Reproducer used

Exits 0 if the behaviour is correct, 1 if the bug is present, 125 if the binary is too old
to express the scenario.

#!/bin/bash
P="$1"
D=$(mktemp -d) || exit 125
trap 'rm -rf "$D"' EXIT
cd "$D" || exit 125

touch Tupfile.ini
echo ORIGINAL > src.txt
printf ': src.txt |> cp %%f %%o |> out.txt\n' > Tupfile

"$P" configure -B . >/dev/null 2>&1 || exit 125
"$P" -B . >/dev/null 2>&1 || exit 125
[ "$(cat out.txt 2>/dev/null)" = ORIGINAL ] || exit 125

echo "NEW CONTENT" > src.txt
printf ': src.txt |> cp %%f %%o && false |> out.txt\n' > Tupfile

"$P" -B . -k >/dev/null 2>&1 && exit 125   # build 2 must fail, else premise broken
[ -s out.txt ] || exit 125                 # the failed command must have left its output

if "$P" -B . -k >/dev/null 2>&1; then exit 1; fi   # build 3 exits 0 => bug present
exit 0

At 26acb43a4 build 3 re-runs the command and reports FAILED: cp src.txt out.txt && false.

Method

Historical commits were built from the committed bootstrap-linux.sh (driver objects only,
-flto stripped, ccache + parallel xargs). Only optimisation and link flags differ from
a normal build; nothing behaviour-relevant. Every boundary verdict was re-measured 5x — a
single-shot verdict from an interrupted run is not trustworthy.