The index has no single definition of a command's recorded state

[typeless]

Summary

The index is a hand-maintained mirror of what the previous build knew. There is no single definition of "a command's recorded state" — instead each category of state has its own bespoke record / compare / route path, written separately in src/cli/cmd_build.cpp:

:178   collect_implicit_dep_files        :1177  expand_implicit_deps
:223   collect_inactive_output_paths     :1346  detect_new_commands
:255   find_changed_files_with_implicit  :1406  reconcile_input_set
:835   preserve_old_implicit_edges       :1517  remove_stale_outputs
:986   merge_out_of_scope_commands

Adding a category of state — implicit deps, order-only edges, exported/imported env vars, group membership, command exit status — requires someone to remember all three legs: record it, compare it, route its consequences. Each walk also open-codes its own subset of LinkType's five members (include/pup/core/types.hpp:129-135). A missing leg is a silent wrong build, and the same failure keeps recurring in a new category.

The instance history is the argument

Closed: #125 (a scoped build evicted out-of-scope commands from the saved index), #126/#128 (keep-going deleted outputs when a Tupfile failed to parse or evaluate), #138 (in-tree incremental never rescheduled cross-directory consumers), #152 (putup-initiated deletions were not invalidated), #166 (a command's input set was not tracked across builds).

Open: #169, #172, #187 — each a category whose legs are incomplete.

#187 is the clearest: exit status was never a recorded category at all. A command that exits nonzero is not remembered, so if its declared outputs exist the next build reports "Nothing to do" and exits 0. Reproduced from scratch:

build 2:  Build completed: 0 commands (1 failed)    exit 1
build 3:  Nothing to do (up to date).               exit 0

Direction

One record per command, holding every category of recorded state — inputs, implicit deps, order-only edges, group membership, env values, and exit status — with one comparison and one routing path. A new category then cannot be half-wired, because there is one place it must be added.

Two constraints found while reviewing this, both of which a naive implementation gets wrong:

• Do not seed the dirty set with config commands. find_config_commands (src/cli/config_commands.cpp) returns every graph command whose output ends in tup.config. These are real, guard-satisfied commands from the two-stage configure pass; treating them as dirty permanently kills the up-to-date fast path (cmd_build.cpp:1811) for every project with a config rule.
• Route a failed command through changed_outputs, not forced_cmds. forced_cmds are OR'd into the affected set at cmd_build.cpp:1936, after collect_affected_commands has already run — so they do not propagate to downstream consumers. Re-running a failed command via forced_cmds would re-run that command and nothing that depends on it.

Corrections to earlier characterisations in this campaign

• remove_stale_outputs runs after all detection, so its own deletions are never routed to consumers #172's "self-heals on the next build" is true only when the consumer declares the dependency. With an undeclared dependency (e.g. a rule that lists a directory rather than naming the file) the healing does not happen. The original triage did not state that condition.
• v16 and v17 in the index version history are not instances of "the index did not record X" — both are "the identity function's input changed", which belongs to the other root cause.

Scope

This issue is the structure. #169, #172 and #187 stay open as the concrete instances; they are what the structure would make impossible rather than things to keep patching one at a time.

Not covered here: identity being a function of the parse traversal — separate root cause, filed separately. The two meet at find_by_identity (cmd_build.cpp:929), so whichever is implemented second must not assume the first left that function alone.

Provenance

Citations verified against merged main. #187 reproduced directly. #169 remains without a reproducer — it should not be treated as demonstrated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA