fix(query): size registration to the host, private spill dir, no root, no extension installs at query time

AGENTS.md

@@ -141,6 +141,17 @@ For an LLM agent driving `atif-sql`, the discovery loop is three commands:
    prefer `atif-sql search 'query'` — it embeds the text first, then runs
    the same `semantic_search` kNN.
 
+   What the query sandbox will and won't do: `SELECT`, `EXPLAIN`, `SET
+   TimeZone`, and in-memory DDL/DML run; `COPY`, `EXPORT`, `ATTACH`,
+   `DETACH`, `INSTALL`, `LOAD`, `PREPARE` and `EXECUTE` exit 70 with kind
+   `sandbox_refused` before anything executes, so emit results on stdout
+   rather than writing files. Running as uid 0 exits 77 (`root_refused`)
+   unless `ATIF_SQL_ALLOW_ROOT=1`. The connection is sized to the host
+   before registration (`ATIF_SQL_QUERY_MEMORY_LIMIT` / `ATIF_SQL_QUERY_THREADS`
+   override), and no extension is ever installed at query time: `atif-sql
+   status` reports `vector_search` as `ready`, `no_store`, or
+   `extension_missing` (fix the last with `atif-sql embed --install-extension`).
+
 Adding a view/macro? The drift tests force: a `DESCRIPTIONS` entry in
 `atif_duck/domain/catalog.py`, an `ARG_EXEMPLARS` entry for any new
 parameter name, `TABLE_MACRO_NAMES` membership if the DDL is `AS TABLE`, and

README.md

@@ -129,6 +129,15 @@ not in a public issue. Supported versions, the disclosure expectations, and what
 vulnerability in a tool that reads local transcripts are in
 **[SECURITY.md](SECURITY.md)**.
 
+`atif-sql query` runs agent-composed SQL, so it runs it in a box: sized to the host before the
+corpus is registered (override with `ATIF_SQL_QUERY_MEMORY_LIMIT` and `ATIF_SQL_QUERY_THREADS`),
+a private spill directory outside the corpus that's removed on exit, no extension installs at
+query time (`atif-sql embed --install-extension` is where the lance extension comes from), file
+facing statements (`COPY`, `EXPORT`, `ATTACH`, `INSTALL`, `LOAD`, `PREPARE`, `EXECUTE`) refused
+before they run, and a refusal to run as root unless `ATIF_SQL_ALLOW_ROOT=1` says so. The details
+and the one accepted disclosure (`duckdb_settings()` lists the granted paths) are in
+[docs/reference/cli.md](docs/reference/cli.md#query).
+
 ## License
 
 [Apache License 2.0](LICENSE). Each of the seven module directories carries the same license

SECURITY.md

@@ -55,4 +55,12 @@ If your transcripts may not be sent to a third-party model provider, do not run
 DuckDB can read and write local files. The tool assumes the caller already owns
 the shell and the data; do not treat it as a sandbox, and do not wire it behind
 an interface that lets an untrusted party choose the SQL, the corpus root, or
-the environment.
+the environment. What the hardened connection does do is keep an injected
+statement from writing the corpus or reaching the network: file-facing
+statement kinds (`COPY`, `EXPORT`, `ATTACH`, `INSTALL`, `LOAD`, `PREPARE`,
+`EXECUTE`) are refused before execution, the only granted directory is a
+private per-process spill dir, no extension is ever installed at query time,
+and `query`, `search` and `analyze` refuse to run as root (file modes don't
+bind uid 0) unless `ATIF_SQL_ALLOW_ROOT=1` is set. Caller SQL can still read
+`duckdb_settings()`, which lists the granted paths and so every session id;
+that's the local user's own corpus listing, not a leak across a boundary.

docs/CONTRACT.md

@@ -159,6 +159,15 @@ atif-sql materialize [--force] [--quiesce-seconds N] [--agent ...] [--workers N]
                                        # report adds rejected / rejected_session_ids
 atif-sql status [--agent ...]          # corpus freshness, counts, watermark age
 atif-sql query 'SQL' [--format auto|json|csv]
+                                       # sized to the host before registration
+                                       # (ATIF_SQL_QUERY_MEMORY_LIMIT / _THREADS
+                                       # override); private mkdtemp spill dir;
+                                       # refuses uid 0 unless ATIF_SQL_ALLOW_ROOT=1;
+                                       # COPY/EXPORT/ATTACH/INSTALL/LOAD/PREPARE/EXECUTE
+                                       # exit 70 sandbox_refused before execution;
+                                       # never installs an extension
+atif-sql embed --install-extension     # the ONE place the lance DuckDB extension
+                                       # is downloaded (also done by a real embed run)
 atif-sql schema                        # static, <50ms, no duckdb bind
 
 ## Parity oracle (satisfied and retired)
@@ -181,6 +190,9 @@ repo neither declares nor provides.
 source_root (default CLAUDE_CONFIG_DIR~/.claude /projects), corpus_root,
 quiesce_seconds=300, agent=claude-code (every --agent command reads it),
 materialize_workers=min(8, cpu_count) (materialize's pool size; 1 = single process).
+ATIF_SQL_QUERY_MEMORY_LIMIT (DuckDB size literal, e.g. 6GB) and ATIF_SQL_QUERY_THREADS
+override query's host-derived cap and thread count; ATIF_SQL_ALLOW_ROOT=1 lets
+query/search/analyze run as uid 0 (a warning is logged).
 _default_*() factories read env at call time. With agent=codex the two roots re-derive to $CODEX_HOME (default
 ~/.codex)/sessions and ~/.atif-sql/corpus/codex; an explicitly set
 ATIF_SQL_SOURCE_ROOT or ATIF_SQL_CORPUS_ROOT always wins over that

docs/architecture/data-flow.md

@@ -103,12 +103,19 @@ sequenceDiagram
    written by a different provider or width raises instead of binding, because vectors from
    different models live in incompatible spaces and would return numerically valid but meaningless
    cosine scores — `packages/atif-duck/src/atif_duck/domain/embedding_guard.py:46`.
-7. The fully-registered connection is then sandboxed: spill directory, memory cap, a directory
-   allowlist holding only the spill area, a path allowlist holding the individual analytics
-   parquets, `enable_external_access=false`, and `lock_configuration` last so caller SQL cannot
-   widen any of it — `packages/atif-cli/src/atif_cli/app.py:138`.
-8. The caller's statement executes against the locked connection (`:606`) and the cursor drains in
-   batches to stdout — a JSON array of row objects on a pipe, a width-aligned table on a TTY — `packages/atif-cli/src/atif_cli/output.py:154`.
+7. Before any of that, the connection was sized to the host (`_configure_query_resources`: a memory
+   cap from available RAM, a thread count from that cap, a private `mkdtemp` spill directory, and
+   extension auto-install and auto-load off), because registration is what needs the cap. The
+   fully-registered connection is then sandboxed: a directory allowlist holding only the private
+   spill area, a path allowlist holding the individual parquets the views read lazily,
+   `enable_external_access=false`, and `lock_configuration` last so caller SQL cannot widen any of
+   it (`packages/atif-cli/src/atif_cli/app.py`, `_harden_query_connection`).
+8. The statement's kinds are checked with DuckDB's own parser on the locked connection; `COPY`,
+   `EXPORT`, `ATTACH`, `DETACH`, `INSTALL`, `LOAD`, `PREPARE` and `EXECUTE` exit 70
+   (`sandbox_refused`) before anything runs. Then the caller's statement executes and the cursor
+   drains in batches to stdout: a JSON array of row objects on a pipe, a width-aligned table on a
+   TTY (`packages/atif-cli/src/atif_cli/output.py:154`). The spill directory is removed when the
+   process exits.
 
 ```mermaid
 sequenceDiagram
@@ -119,6 +126,7 @@ sequenceDiagram
     participant Lance as Lance store
 
     CLI->>DB: duckdb.connect()
+    CLI->>DB: threads, memory_limit, private temp_directory, autoinstall off
     CLI->>Duck: register(con, corpus_root, expected model + dim)
     Duck->>DB: CREATE TEMP TABLE raw readers over corpus globs
     DB->>Disk: read_json sessions/*/meta.json then the rest
@@ -130,6 +138,7 @@ sequenceDiagram
     Duck->>Duck: ensure_store_matches, then bind the view
     Duck-->>CLI: views and macros registered
     CLI->>DB: allowlists, external access off, lock_configuration
+    CLI->>DB: extract_statements(caller SQL), refuse file-facing kinds
     CLI->>DB: execute(caller SQL)
     DB-->>CLI: cursor
     CLI->>CLI: drain in batches to stdout

docs/behavior/processes.md

@@ -270,14 +270,18 @@ Entry point: `packages/atif-cli/src/atif_cli/app.py:529`
 5. Create the macros, then the analytics views and analytics macros over the
    analytics parquets, which bind against both the parquets and the base views
    — `:1005`, `packages/atif-duck/src/atif_duck/infrastructure/analytics.py:124`.
-6. Harden the connection in a fixed order: temp directory, memory cap, a
-   directory allowlist holding only the spill area, a file allowlist of the
-   analytics parquets, the config exemption list, then
-   `enable_external_access=false` and `lock_configuration=true` last —
-   `packages/atif-cli/src/atif_cli/app.py:138`.
-7. Execute the caller's statement and stream the cursor: a plain table on a
-   TTY, a JSON array of row objects on a pipe — `:606`,
-   `packages/atif-cli/src/atif_cli/output.py:154`.
+6. Harden the connection in a fixed order (the memory cap, thread count and
+   private spill directory were set before step 3, since registration is what
+   needs them): a directory allowlist holding only the spill area, a file
+   allowlist of the parquets the views read lazily, the config exemption
+   list, then `enable_external_access=false` and `lock_configuration=true`
+   last (`packages/atif-cli/src/atif_cli/app.py`, `_harden_query_connection`).
+7. Check the statement's kinds with DuckDB's parser and refuse `COPY`,
+   `EXPORT`, `ATTACH`, `DETACH`, `INSTALL`, `LOAD`, `PREPARE` and `EXECUTE`
+   before anything runs (exit 70, `sandbox_refused`); then execute the
+   caller's statement and stream the cursor: a plain table on a TTY, a JSON
+   array of row objects on a pipe (`packages/atif-cli/src/atif_cli/output.py:154`).
+   The spill directory is removed on exit.
 8. Classify any DuckDB failure into parse, catalog, or runtime — or an
    embedding-provider mismatch — and exit 64, 65, or 70 with a JSON error
    envelope — `packages/atif-cli/src/atif_cli/duck_errors.py:34`, `:66`.

docs/reference/cli.md

@@ -100,13 +100,17 @@ Flags:
 - `--corpus-root` — override the materialized corpus root. `:504`
 - `--format` — `table` on a TTY, a JSON array of row objects on a pipe. `:505`
 
-The statement runs against a hardened connection: reads reach the registered views and nothing else, and the only writable path is the query engine's own spill directory `<corpus_root>/.duckdb_tmp`. `:601`
+The statement runs against a hardened connection: reads reach the registered views and nothing else, and nothing under the corpus root is writable. Before registration the connection is sized to the host: a memory cap derived from available RAM (half of physical RAM or 8 GiB, whichever is larger, never above 80% of what's available) and a thread count of one per 2 GiB of that cap, capped at the CPUs the process may use. `ATIF_SQL_QUERY_MEMORY_LIMIT` (a DuckDB size such as `6GB`) and `ATIF_SQL_QUERY_THREADS` override both; a malformed value exits 64. The spill directory is a private `mkdtemp` (mode 0700) under the system temp dir, the only directory the sandbox grants, and it's removed when the process exits. Extension auto-install and auto-load are off, and the lance extension is loaded only when it's already installed, so registration never reaches the network (`packages/atif-cli/src/atif_cli/app.py`, `_configure_query_resources`).
+
+Two layers keep caller SQL from writing the corpus. DuckDB's file grants are read-write and it has no read-only grant, so `COPY ... TO <granted parquet> (USE_TMP_FILE false)` would overwrite one; the CLI therefore refuses every statement kind that names a file before executing anything, using DuckDB's own parser: `COPY`, `EXPORT`, `ATTACH`, `DETACH`, `INSTALL`, `LOAD`, `PREPARE` and `EXECUTE` exit 70 with kind `sandbox_refused`, for any uid, and a batch containing one of them runs nothing. And because a `0444` file mode doesn't bind root, `query` refuses to run as uid 0 (exit 77, kind `root_refused`) unless `ATIF_SQL_ALLOW_ROOT=1` is set, which logs a warning. `search` and `analyze` refuse root the same way.
+
+What caller SQL can still see: `duckdb_settings()` and `current_setting(...)` return the sandbox's own configuration, including the corpus root, the spill directory, the memory cap and every granted parquet path, which names every session id. DuckDB can't hide a setting from SQL and the grants have to be per file, so this is accepted: the caller is the local user, who can list the corpus and `SELECT session_id FROM sessions` anyway, and `query` isn't a privilege boundary (see SECURITY.md).
 
 The views themselves carry no corpus path as statement text. The registry hands its globs and file lists to `read_json(?)` as bound parameters and builds the parquet readers through DuckDB's relation API, so a corpus root such as `o'brien ?; --$1` and transcript content carrying SQL text both register as data (`packages/atif-duck/src/atif_duck/infrastructure/registry.py`). A session directory whose name fails the session id boundary (`packages/atif-duck/src/atif_duck/domain/session_id.py`) registers nothing and is logged once.
 
 Sessions that carry current columnar artifacts are served from their parquet files, so no JSON is parsed for them at query time; the rest are read from `trajectory.json`, and the views union the two. The per-session parquet files the registry bound are granted to the sandbox the same way the analytics parquets are (as individual `allowed_paths` entries, `packages/atif-cli/src/atif_cli/app.py:221`), and they're written read-only (`0444`), so a `COPY ... TO` at one of them fails at the filesystem even though DuckDB's grant is read-write. `atif-sql status` says which path a corpus takes.
 
-Exit codes: `64` parse error, `65` catalog error, `65` embedding mismatch, `70` runtime error. `:550`
+Exit codes: `64` parse error or a malformed `ATIF_SQL_QUERY_*` override, `65` catalog error, `65` embedding mismatch, `70` runtime error, `70` `sandbox_refused` (a statement kind the sandbox never runs), `77` `root_refused`. `:550`
 
 ## analyze