fix(fleet): bind each node to the credential that heartbeats it

[eldad-caura-ai]

Audit finding

Addresses M-85: fleet node identity was the caller-supplied node_name. Any agent or install credential in a tenant could heartbeat as another node, receive and ack its queued commands (deploy payloads included, which an agent credential may not queue itself), report results for any command by UUID, and list every command's payload and result.

Change

Each fleet node is now bound to the credential that heartbeats it.

• Principal. A gateway-verified agent key binds as agent:<id> and an install credential as install:<uuid>. Every other credential is tenant-wide (tenant): tenant, user and admin keys, the shared CAURA_API_KEY, standalone mode, and an X-Agent-ID the caller asserted itself. OSS standalone and shared-key deployments therefore see no change. An install credential that arrives without its install UUID is refused (403 INSTALL_UUID_MISSING) rather than bound to a shared placeholder.
• Heartbeat. fleet_upsert_node decides in its ON CONFLICT ... WHERE, against the locked row. A new or unbound node takes the caller's principal. A node bound to another narrow credential refuses a narrow caller with 403 FLEET_NODE_BOUND_TO_OTHER_CREDENTIAL, and the refused heartbeat changes nothing and delivers nothing. A tenant-wide credential is always admitted and takes the node back.
• Command result and listing. A narrow credential reports on, and lists, only commands of nodes bound to it. Another node's command is the same 404 as a missing one.
• Release. POST /api/v1/fleet/nodes/{node_id}/release (tenant credentials only) moves a node to a new credential, for example after rotating its key. With bind_agent_id or bind_install_uuid the node is bound to that credential in the same statement, so no other credential can claim it in between. Bare, it clears the binding and the next heartbeat binds the node.
• Migration 055 adds nullable fleet_nodes.owner_principal, with no backfill. Existing nodes bind on their first heartbeat after deploy.

Trade-offs (approved policy)

• Existing nodes bind on first heartbeat, and that heartbeat also receives the node's pending commands. Between deploy and each existing node's first heartbeat, a narrow credential in the same tenant that heartbeats first can claim the node and take what is queued. The tenant's own next heartbeat takes the node back; a node whose own credential is narrow is rebound with a release that names it. Nothing recorded which credential an existing node uses, and binding every existing node to the tenant instead would lock out nodes that heartbeat with agent keys or install credentials until an operator rebinds each one.
• Moving a node to a different narrow credential takes a release that names it. A bare release reopens the first-heartbeat window for that node.
• Deploy core-storage-api (migration 055) before core-api, as the pipeline already does. A new core-api sends owner_principal, which an older storage cannot write.

Not in this PR

• Interview fleet-node windows (the rest of M-86) follow in a separate PR, now that caura PR fix(interview): accept adapter streams and bind them to their agent #1788 has merged: a window for a fleet node must cite a delivered interview_request, used once. It relies on this PR's listing filter, which stops an agent credential reading another node's pending command ids.
• The enterprise dashboard has no button for the release endpoint yet.

Verification

• Storage tests cover bind, refuse-without-change, refresh, reclaim, legacy unbound rows, caller-omitted binding, release (including tenant scoping and binding straight to a named credential), and the result and listing filters. Route tests cover the drain attack, over-refusal guards for own nodes and asserted headers, install credentials (including one with no install UUID), release authorization, release that binds a named agent key or install credential with no window for a squatter, at most one bind target, and 404 for an unknown node.
• Tests first: the tests-only push (run 37074834988) failed all 10 attack tests in tests/test_fleet_node_binding.py against unfixed main, with 8459 passing. The drain test's response carried the victim node's queued command. The two over-refusal guards passed, as they must without the fix. That step stopped the job before the storage suite ran. The fix is now amended into the same single commit.
• No repository scripts, dependency installs, tests or builds were run on Eldad's Mac under its safety hold. CI runs the suites.

One signed-off commit. Do not merge or enable auto-merge; Eldad merges after review.

🤖 Generated with Claude Code

[github-actions]

Claude Code Review — skipped: draft PR

[github-actions]

🤖 Review by Claude Code

Claude Code Review ✅ No issues found.

---

Reviewed by claude-sonnet-5-5 · cost $0.125535

[eldad-caura-ai]

@erni-a pelrq ppaove

[eldad-caura-ai]

@claude

[github-actions]

🤖 Review by Claude Code

Summary

This PR binds each fleet node to the credential that heartbeats it. It adds an owner_principal column and enforces the binding inside the upsert's ON CONFLICT … WHERE, so the check and the bind are atomic. It also filters command results and listings to the caller's bound nodes, and adds a tenant-only release route. The design is sound and I found no correctness or security regressions in the storage logic. The one problem is a defensive fallback in the principal derivation that fails open.

Medium/Low Issues

Install credentials without a UUID share one binding principal

Severity: Low
File: core-api/src/core_api/routes/fleet.py:163-178
Problem: _node_principal falls back to the literal install:unknown when auth.install_uuid is empty, so any two such install credentials are treated as the same owner and can act as each other's nodes.

🤖 Claude Code Prompt

In core-api/src/core_api/routes/fleet.py, function `_node_principal` (around lines 163-178), the install-credential branch builds the principal as "install:" followed by `auth.install_uuid or 'unknown'`.

If an install credential ever arrives with an empty or None `install_uuid`, every such credential shares the principal "install:unknown". That defeats the per-credential isolation M-85 adds.

Change the branch so it never collapses distinct credentials into one shared principal. The safest option is to refuse the request when `is_install_credential` is true but `install_uuid` is missing. Do this by raising an HTTPException 403 with a coded detail, or by having `_node_principal` raise.

The alternative is to guarantee that `install_uuid` is always populated for install credentials. If you choose that, add a defensive assertion or log here instead of the 'unknown' fallback.

Keep the existing behaviour for agent and tenant credentials.

Add a test in tests/test_fleet_node_binding.py covering an install credential with no uuid.

---

Reviewed by claude-sonnet-5-5 · cost $0.4199339999999999

[eldad-caura-ai]

Addressed the Low finding in 7afc45e, still a single signed-off commit. _node_principal no longer falls back to install:unknown. An install credential without an install UUID now gets a 403 with the new code INSTALL_UUID_MISSING on heartbeat, command result and command listing, so no two such credentials can share a binding. The release route already refused install credentials. New test: test_an_install_credential_without_a_uuid_is_refused. Without the fix the same heartbeat returns 200 and binds the shared principal.

🤖 Addressed by Claude Code

[eldad-caura-ai]

@claude

[github-actions]

🤖 Review by Claude Code

Summary

This PR binds each fleet node to the credential that heartbeats it. It adds an owner_principal column (migration 055), makes the binding check atomic in the upsert's conflict arm, and filters command results and listings to the caller's own nodes. The check is applied correctly on the storage side. I found one design gap that leaves part of the original attack open. I did not review the changes under tests/ and core-storage-api/tests/, per the review scope.

Medium/Low Issues

Unbound nodes can be claimed by any narrow credential, so the queue-drain attack still works after migration 055 or a release

Severity: Medium
File: core-storage-api/src/core_storage_api/services/postgres_service.py:12010-12030 (also core-api/src/core_api/routes/fleet.py:~893-935, core-storage-api/.../versions/055_fleet_node_owner_principal.py)
Problem: A node with NULL owner_principal is bound by whichever narrow credential heartbeats it first, and that heartbeat also returns and acks the node's pending commands, deploy payloads included. This covers every pre-existing node right after 055 (no backfill) and every node after /release. During that window an agent or install credential in the same tenant can claim the node, drain its queue, and lock out the real owner with a 403 until a tenant credential releases it. This is the M-85 attack, just narrowed to the unbound window.

🤖 Claude Code Prompt

Context: M-85 fleet node binding. In core-storage-api/src/core_storage_api/services/postgres_service.py, fleet_upsert_node (around lines 11993-12030) lets a narrow principal (agent:<id> / install:<uuid>) bind any node whose owner_principal is NULL. The conflict-arm WHERE is `owner_principal IS NULL OR owner_principal = <caller>`. In core-api/src/core_api/routes/fleet.py, heartbeat() then calls get_pending_commands and ack_commands for that node. As a result, right after migration 055 (no backfill), or right after POST /fleet/nodes/{node_id}/release, any narrow credential in the tenant can claim an unbound node by name, receive and ack its queued commands (including deploy payloads), and lock out the real owner.

Please reduce this window without breaking the documented flows. Options, pick one and document it:
(a) Do not let a narrow credential claim an existing row with NULL owner_principal and a non-NULL created_at/last_heartbeat. Only a tenant-wide credential may adopt unbound pre-existing nodes, while a narrow credential may still create a new node or bind one it just inserted.
(b) Or have release_node (core-api/src/core_api/routes/fleet.py ~1241 and the storage fleet_release_node) accept an optional target principal to bind the node to directly, instead of clearing it to NULL, so rotation has no open window.
(c) At minimum, in heartbeat(), do not deliver or ack pending commands on the heartbeat that first binds a narrow credential to a previously unbound existing node.

Update the docstring of fleet_upsert_node, the 055 migration docstring, and docs/api-reference.md to describe whichever behaviour you choose. Add tests in tests/test_fleet_node_binding.py and core-storage-api/tests/test_fleet_node_owner_binding.py covering a narrow credential heartbeating an unbound existing node that has queued commands.

---

Reviewed by claude-sonnet-5-5 · cost $0.5732772500000001

[github-actions]

🤖 Review by Claude Code

Claude Code Review ✅ No issues found.

---

Reviewed by claude-sonnet-5-5 · cost $0.45248449999999996

[eldad-caura-ai]

@erni-a ready for human review. CI, CodeQL, DCO and a fresh Claude review (no issues found) all pass on the single commit b42b574. Since the last review:

• the shared install:unknown fallback is gone: an install credential with no install UUID now gets 403 INSTALL_UUID_MISSING;
• release can bind a node straight to a named credential (bind_agent_id / bind_install_uuid), so a key rotation leaves no window for another credential;
• the remaining window is documented: existing nodes bind on their first heartbeat after migration 055.

Both additions were tests-first: the tests-only run 37121821097 failed only the 3 new release tests. Deploy core-storage-api (migration 055) before core-api. Not auto-merging; Eldad merges.

🤖 Posted by Claude Code