feat: add deterministic dbt fault injection check

CONTRIBUTING.md

@@ -87,6 +87,24 @@ bun run local         # build + run
 The script handles platform detection (including Rosetta 2), `NODE_PATH` setup
 for native modules like `@altimateai/altimate-core`, and binary resolution.
 
+#### Running fault injection against a locally built altimate-core
+
+`altimate-code fault-injection` and the `dbt_fault_injection` tool need `FaultInjectionSession`
+from `@altimateai/altimate-core`. To develop against a core build that is not published yet, point
+`ALTIMATE_CORE_DEV_PATH` at the built Node binding (the `crates/altimate-core-node` directory of an
+altimate-core checkout, after `npm run build` there). This is for development only: it loads a
+native addon from an arbitrary path, so published releases (`latest`, `beta`) ignore it, and the
+report says when it was used.
+
+```bash
+ALTIMATE_CORE_DEV_PATH=~/code/altimate-core/crates/altimate-core-node \
+ALTIMATE_DBT_PATH=~/venvs/dbt-duckdb/bin/dbt \
+  bun dev fault-injection path/to/dbt-project
+```
+
+The same two variables enable `test/altimate/fault-injection-e2e.test.ts`, which is skipped when
+dbt-duckdb or the engine class is unavailable.
+
 To compile all 12 platform targets (CI/release):
 
 ```bash

docs/docs/data-engineering/tools/dbt-tools.md

@@ -140,6 +140,162 @@ unit_tests:
 
 ---
 
+## dbt_fault_injection
+
+Find the upstream data faults a dbt project's own tests would miss.
+
+It corrupts one upstream relation at a time in a private copy of the database, rebuilds every model
+downstream of it, runs the project's tests, and reports which faults no test noticed. It is
+deterministic and uses no model, so it is also available as a plain command that needs no API key:
+
+```bash
+altimate-code fault-injection                      # the dbt project in the current directory
+altimate-code fault-injection path/to/project --budget 50
+altimate-code fault-injection --model stg_orders   # corrupt only this model, seed or source
+altimate-code fault-injection --format json --fail-under 60   # for CI
+```
+
+**Parameters** (tool) / **flags** (command):
+- `project_dir` / `[project]` (optional): dbt project root. Defaults to the working directory
+- `budget` / `--budget` (optional, default 20): maximum number of faults to inject
+- `model` / `--model` (optional): corrupt only this model, seed, snapshot or source
+- `target` / `--target`, `profiles_dir` / `--profiles-dir` (optional): as for dbt
+- `--seed`, `--work-dir`, `--format text|json`, `--fail-under <percent>`: command only
+
+The seven faults are duplicated rows, dropped rows, a column set to NULL, a number multiplied by 100,
+a category value never seen before, a date moved one day, and a foreign key pointing nowhere. Each
+touches 5% of the eligible rows of one relation (at least one row).
+
+### Reading the output
+
+A real run on the 5-model jaffle_shop project, trimmed (`...` marks removed lines):
+
+```
+Fault injection: jaffle_shop_snowflake (duckdb)
+
+Catch rate: 68.0% (34 of 50 faults that mattered were caught)
+  50 faults injected: 34 caught, 16 slipped through, 0 harmless, 0 invalid
+  ...
+
+Slipped through (16): the project's tests did not notice these
+
+1. seed raw_customers: `first_name` set to NULL in 5 of 100 rows
+   Fault id: jaffle_shop_snowflake|seed.jaffle_shop_snowflake.raw_customers|null_out|first_name
+   5 tests ran and none failed because of the fault. Changed downstream:
+     - model customers: 5 rows changed (first_name: 5) of 100 (matched on customer_id)
+     - model sample: content changed, row count unchanged at 100; no unique key is declared for this relation, so no row-level detail
+     ...
+   Proposed test: not_null
+   Verified on the data: passes on the clean data and fails on the corrupted copy (5 failing).
+      seeds:
+        - name: raw_customers
+          columns:
+            - name: first_name
+              data_tests:
+                - not_null
+   Note: `first_name` has no NULL in the baseline, so a `not_null` test passes today and fails on the first NULL.
+...
+3. seed raw_orders: `order_date` moved one day later in 5 of 99 rows
+   ...
+   No test is proposed for this fault: `order_date` is a date or timestamp, and a one-day shift stays inside the span the data legitimately covers. ...
+...
+8. seed raw_payments: `amount` multiplied by 100 in 6 of 113 rows
+   ...
+   Proposed test: range (singular test)
+   Verified on the data: passes on the clean data and fails on the corrupted copy (1 failing).
+   The standard test for this needs a package this project has not installed, so this is a singular test (plain SQL, no package needed).
+   Save as tests/fault_injection/fault_injection__raw_payments__range__amount.sql (or in another folder listed under test-paths in dbt_project.yml):
+      SELECT * FROM {{ ref('raw_payments') }} WHERE "amount" < 0 OR "amount" > 100000
+...
+These are gaps in the project's tests. They are not evidence that the data in the warehouse today is wrong.
+Took 195s: setup and baseline build 6.1s; 50 faults 139s (2.8s each on average); profiling and no-fault controls 51s.
+```
+
+- **Caught**: a test failed, or a downstream model failed to build
+- **Slipped through**: no test failed and downstream data changed. These are the findings
+- **Harmless**: nothing downstream changed, so the fault is left out of the catch rate
+- **Invalid**: the fault touched no row, or its sandbox failed twice
+- **Catch rate**: caught / (caught + slipped through), rounded down
+
+Each slipped fault lists the downstream models that changed, row by row where the model has a
+declared unique key and by row count and checksum otherwise, and either a test that would catch it
+or the reason there is none.
+
+A proposed test is offered only if it passes on the clean data and fails on the corrupted copy (both
+are checked by running it) and is unlikely to be a snapshot of today's data. So:
+
+- A range is not the column's current minimum and maximum. It leaves an order of magnitude of room,
+  and a fault that stays inside that room gets no range test.
+- A column computed from the current date or time (found from the model's SQL, through its parents,
+  and from relations that change on every rebuild with no fault injected) gets no range, no list of
+  values and no non-null-share floor, because they would fail as the clock moves.
+- A list of accepted values is proposed only for a column that is plausibly a category: at most 20
+  distinct values, at least 30 non-null rows and at least 10 rows per value. Such a column is also the
+  only kind the "new category value" fault is injected into.
+- A date moved by one day gets no test: any range narrow enough to see it fails when the next row arrives.
+- A test for a model or source that an installed dbt package defines, or one whose standard test
+  needs a package the project has not installed, is a singular test (a SQL file to save under
+  `tests/`) because dbt accepts only one schema entry per resource.
+- A `relationships` test names its parent from what the project already declares (relationship tests,
+  foreign-key constraints), from joins in the models that read the column, and from declared keys, and
+  is offered only if it verifies on both copies. With no parent that verifies, the report says so.
+
+`--format json` prints the same result as JSON, including every executed fault (not only the ones
+that slipped through), the no-fault controls and per-fault timings.
+
+!!! note
+    Findings are gaps in the project's tests. They are not evidence that the data in the warehouse
+    today is wrong: the corruption only ever exists in the copy.
+
+### Cost
+
+One `dbt build --full-refresh` of the models, seeds and snapshots on the copy, then for every
+corrupted relation three no-fault control runs, and for every fault one `dbt run` of the downstream
+models plus one `dbt test`. All of it is single-threaded. The example above took 195 seconds on a
+laptop for 50 faults. The controls are a fixed cost per relation, so a small budget
+spread over many relations is dominated by them: on a larger project, 20 faults spread over 17
+relations took 186 seconds, 129 of them in profiling and controls. Use `--model` to concentrate the
+budget.
+
+### Safety
+
+dbt runs on a copy of the database and a copy of the project, both in a temporary directory. `--work-dir` names the parent
+of that directory, and the command creates an `altimate-fault-injection-*` directory inside it,
+with its own profile, target path and log path. Only that directory is removed, on success,
+failure and interrupt, and the command reports where it was. The project's database is opened only
+to copy it, and the command checks afterwards that its size and modification time are unchanged.
+
+It refuses to run when it can see that dbt would reach beyond that copy: an in-memory or MotherDuck
+database, a profile with `attach` or `plugins`, a database with a pending write-ahead log, a model
+with the `external` materialization, a relation in another database, or a hook that runs `ATTACH`,
+`COPY` or `EXPORT DATABASE`.
+
+It does not inspect what macros and Python models do. Code there that writes to an absolute path, or
+attaches another database by absolute path, would act on the real thing. A `kill -9` leaves the
+temporary directory behind.
+
+### Limits
+
+- **DuckDB only.** Any other warehouse is refused before anything runs
+- The models, seeds and snapshots must build on the copy; a build error stops the run. Tests that
+  already fail do not: they are reported and cannot catch a fault
+- Needs dbt-core (not dbt Fusion) with the project's adapter. Set `ALTIMATE_DBT_PATH` if dbt is not
+  found
+- Needs an `@altimateai/altimate-core` that includes the fault-injection engine; the command says so
+  when the installed one does not
+- Row-level detail (which columns changed in how many rows) needs a declared unique key. Up to 16,384
+  rows both versions of a changed model are compared in full. Above that, up to 2,000,000 rows, the
+  command compares a hash of every row and then reads only the changed rows, which is exact for the
+  counts of added, removed and changed rows; when more than 20,000 rows changed the per-column
+  counts come from the first 20,000 of them, and the report says so. A 580,000-row model took about
+  7 seconds per fault in a test on a loaded laptop. Larger models keep the row count and checksum.
+- A proposed test assumes the data keeps its shape: an accepted-values list fails when a new
+  legitimate value first appears, and a row-count floor when the model legitimately shrinks.
+- Each fault copies the database once. On filesystems without copy-on-write clones that is a full
+  copy per fault
+
+---
+
 ## altimate-dbt CLI
 
 `altimate-dbt` is a standalone CLI for dbt workflows. It auto-detects your dbt project directory, Python environment, and adapter type (Snowflake, BigQuery, Databricks, Redshift, etc.).

docs/docs/data-engineering/tools/index.md

@@ -13,7 +13,7 @@ altimate has 100+ specialized tools organized by function.
 | [Schema Tools](schema-tools.md) | 7 tools | Inspection, search, PII detection, tagging, diffing |
 | [FinOps Tools](finops-tools.md) | 8 tools | Cost analysis, warehouse sizing, unused resources, RBAC |
 | [Lineage Tools](lineage-tools.md) | 1 tool | Column-level lineage tracing with confidence scoring |
-| [dbt Tools](dbt-tools.md) | 3 tools + 6 skills | Run, manifest parsing, unit test generation, scaffolding, `altimate-dbt` CLI |
+| [dbt Tools](dbt-tools.md) | 4 tools + 6 skills | Run, manifest parsing, unit test generation, fault injection, scaffolding, `altimate-dbt` CLI |
 | [Warehouse Tools](warehouse-tools.md) | 6 tools | Environment scanning, connection management, discovery, testing |
 | [Altimate Memory](memory-tools.md) | 3 tools | Persistent cross-session memory for warehouse config, conventions, and preferences |
 | [Training](../training/index.md) | 3 tools + 3 skills | Correct the agent once, it remembers forever, your team inherits it |

docs/docs/usage/cli.md

@@ -28,6 +28,7 @@ altimate --agent analyst
 | ----------- | ------------------------------ |
 | `run`       | Run a prompt non-interactively |
 | `check`     | Run deterministic SQL checks (no LLM required) -- see [SQL Check](check.md) |
+| `fault-injection` | Find the upstream data faults a dbt project's tests miss (no LLM required) -- see [dbt Tools](../data-engineering/tools/dbt-tools.md#dbt_fault_injection) |
 | `serve`     | Start the HTTP API server      |
 | `web`       | Start the web UI               |
 | `agent`     | Agent management               |