Conversation
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
SummaryThis PR adds an "agent scope probe" that explains why an agent-filtered Medium/Low IssuesProbe adds an unconditional extra round-trip to every filtered search/recall, contradicting its own docstringSeverity: Medium 🤖 Claude Code PromptReviewed by |
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
…lt is empty
A tenant-scoped caller that passes a wrong `filter_agent_id` gets
`HTTP 200 · items [] · warnings null` — byte-identical to a correct id
that simply has nothing relevant to say. Measured, one memory written by
`agent-real`:
filter_agent_id = "agent-real" -> 200 items=1
filter_agent_id = "agnet-real" -> 200 items=0 warnings=None
The filter is a SQL `WHERE memories.agent_id = ?`, so a typo matches no
rows and the query returns nothing exactly as an empty query would. It
fails in the safe-looking direction — an empty list reads as "no data
yet", the most ordinary thing a memory product can say, so nobody
investigates. It cost a 589-query benchmark run against an empty store.
## Scope: a tenant-scoped-caller bug only
An agent-scoped credential carries a verified `X-Agent-ID`, and
`enforce_self_agent` **403s** it for naming any agent but itself —
measured, for a typo and for a peer id. So the silent path needs a
credential with no agent identity, asserting the id in the body.
Not a corner: it is the dashboard's hand-typed `filter_agent_id` text box
(tenant-scoped by its own comment, "no X-Agent-ID"), the documented
`caura_recall` parameter on the public for-agents page, every benchmark
harness we own, and the multi-user backend shape where one key serves
many end users.
## When it runs, and what it costs
Only after the search, and the storage probe only when the search came
back EMPTY. **A successful agent-filtered search pays nothing** — no
extra round trip, no extra query. Pinned by
`test_a_successful_search_makes_no_probe_call`, which counts calls rather
than trusting the reading.
This is the second attempt, after review of #1434. The first probed
BEFORE the search to skip the embedding call on a misconfigured request —
but that saving lands on requests already broken, which are rare, while
the cost lands on every healthy one, which is the norm. Wrong way round
on expected cost. The review also caught that the docstring claimed the
probe "runs only when a search is about to return nothing", which was
false of that ordering; it is true of this one.
The empty path is now one query, not two: `include_agent_registered`
lets the probe skip the `agents` lookup, because the route already made
it (see below).
## Which fact decides what, and why not the obvious one
The natural implementation — look the id up in `agents`, and if absent
report "no such agent" — is wrong in the direction that misleads.
`DELETE /agents/{id}` says so itself: *"Delete an agent. Memories written
by this agent are NOT deleted."* The rows stay live and searchable with
no agent row, and rows predating agent tracking were never registered at
all. A missing agent row means "deregistered or never registered", never
"nothing to find".
So `has_memories` establishes there was nothing to find, and
`agent_preexisted` only chooses the wording:
returned rows + agent gone -> filter_agent_deregistered (free)
empty + has_memories + gone -> filter_agent_deregistered
empty + no memories + known -> filter_agent_empty
empty + no memories + unknown -> filter_agent_unknown
anything else -> silence
`agent_preexisted` comes from the route's own `get_or_create_agent`, via
a new `registration_ctx` out-dict in the same shape as `diagnostic_ctx` /
`warnings_ctx` / `recall_ctx`. That call already does the `agents`
lookup, so it is free — and it *has* to come from there, because by the
time the search has run the row exists whether or not it did beforehand,
and asking afterwards would report every typo as a registered agent. An
out-dict rather than a changed return type so the other seven callers
stay untouched.
Known boundary: an admin credential skips `get_or_create_agent`
(`if auth.tenant_id:`), so it has no free pre-existence signal and is not
told an agent is deregistered on a NON-empty result. Buying that would
put a query back on the success path. Documented on the `tenant_scoped`
fixture.
## Ratchets
`test_c27_strict_fleet_scoping` pins two things, and the probe is a fifth
fleet-scoped read:
* it must go through `_fleet_scope_clause`, never a hand-rolled
`fleet_id.in_(fleet_ids)` — "exactly how A54 leaked". It does.
* the helper's call-site count moves 5 -> 6, with the reasoning recorded
at the constant.
`strict=False` on purpose: non-strict is a SUPERSET (it also admits
tenant-shared null-fleet rows and `scope_org`), so the probe stays at
least as permissive as the search it explains and can never report "no
memories" for rows the search could see.
An earlier draft short-circuited ahead of `enforce_fleet_read_many` and
the usage metering — a caller naming a fleet it has no rights to got a
200 with a warning instead of a 403, and the warning disclosed whether an
agent id exists. `test_c27_strict_fleet_scoping` and
`test_h05_multi_fleet_read_gate` caught it. That whole class is gone with
the short-circuit: the search now always runs.
## `/recall` too
It parses the same `SearchRequest` and had the identical silent empty.
Leaving it out is how the two routes came to disagree about what these
fields mean — the reason `_resolve_read_identity` is shared at all. Its
response gains `warnings`, seeded from the probe and then extended by the
pipeline, so the field means what `/search`'s does rather than carrying
only this one code.
## Known limitation, pre-existing
The read path registers the asserted id via `get_or_create_agent`, so a
repeated typo is registered by the first call and the second reports
`filter_agent_empty` rather than `filter_agent_unknown`. Both still say
the filter matched nothing, and the wording is advisory. Not introduced
here and not fixed here: the clean answers are to stop registering on
reads, or agent soft-delete (a `deleted_at` plus making
`uq_agents_tenant_agent` partial, or re-registration breaks) — both
larger than a warning warrants.
## Verification
- `tests/test_caura723_agent_scope_warning.py` — 10 cases: the four
states, the fleet-narrowed probe, `/recall`, the agent-scoped 403 left
unchanged, no-filter and good-filter silence, a failing probe leaving
search working, and the no-cost-on-success contract.
- Targeted sweep post-rebase: 110 passed (`test_c27_strict_fleet_scoping`,
`test_h05_multi_fleet_read_gate`, `test_route_authz_gaps`,
`test_search_recall_tracked_flag`, `test_c4_recall_items_alias`,
`test_mcp_recall`, `test_agent_admin_authz`).
- Full `tests/`: 39 failures, byte-identical to `main`'s set on this
machine (pre-existing FTS / relation-weight / request-observation env
failures). `core-storage-api/tests/`: 369 passed, same 2 pre-existing
CORS failures.
- ruff check and format clean; mypy identical to `main` on every changed
file. `do_not_touch_sentinel` all 35 survive, `tenant_scope_gate`
exit 0, `legacy_name_ratchet` **"No new lines."** — the test reaches the
dual-read Settings field through `tests/_legacy_contracts`'
`LEGACY_API_KEY_FIELD` rather than minting the literal, in keeping with
#1436-#1438.
No schema changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Eyal Blyachman <eyal.b@caura.ai>
223d246 to
072f6b4
Compare
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
1 similar comment
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
SummaryThis PR adds a diagnostic "why did my agent-filtered search return nothing" warning to Medium/Low IssuesAgent-scope warning is keyed on the resolved identity, not the actual
|
…lt is empty
A tenant-scoped caller that passes a wrong `filter_agent_id` gets
`HTTP 200 · items [] · warnings null` — byte-identical to a correct id
that simply has nothing relevant to say. Measured, one memory written by
`agent-real`:
filter_agent_id = "agent-real" -> 200 items=1
filter_agent_id = "agnet-real" -> 200 items=0 warnings=None
The filter is a SQL `WHERE memories.agent_id = ?`, so a typo matches no
rows and the query returns nothing exactly as an empty query would. It
fails in the safe-looking direction — an empty list reads as "no data
yet", the most ordinary thing a memory product can say, so nobody
investigates. It cost a 589-query benchmark run against an empty store.
## Scope: a tenant-scoped-caller bug only
An agent-scoped credential carries a verified `X-Agent-ID`, and
`enforce_self_agent` **403s** it for naming any agent but itself —
measured, for a typo and for a peer id. So the silent path needs a
credential with no agent identity, asserting the id in the body.
Not a corner: it is the dashboard's hand-typed `filter_agent_id` text box
(tenant-scoped by its own comment, "no X-Agent-ID"), the documented
`caura_recall` parameter on the public for-agents page, every benchmark
harness we own, and the multi-user backend shape where one key serves
many end users.
## When it runs, and what it costs
Only after the search, and the storage probe only when the search came
back EMPTY. **A successful agent-filtered search pays nothing** — no
extra round trip, no extra query. Pinned by
`test_a_successful_search_makes_no_probe_call`, which counts calls rather
than trusting the reading.
This is the second attempt, after review of #1434. The first probed
BEFORE the search to skip the embedding call on a misconfigured request —
but that saving lands on requests already broken, which are rare, while
the cost lands on every healthy one, which is the norm. Wrong way round
on expected cost. The review also caught that the docstring claimed the
probe "runs only when a search is about to return nothing", which was
false of that ordering; it is true of this one.
The empty path is now one query, not two: `include_agent_registered`
lets the probe skip the `agents` lookup, because the route already made
it (see below).
## Which fact decides what, and why not the obvious one
The natural implementation — look the id up in `agents`, and if absent
report "no such agent" — is wrong in the direction that misleads.
`DELETE /agents/{id}` says so itself: *"Delete an agent. Memories written
by this agent are NOT deleted."* The rows stay live and searchable with
no agent row, and rows predating agent tracking were never registered at
all. A missing agent row means "deregistered or never registered", never
"nothing to find".
So `has_memories` establishes there was nothing to find, and
`agent_preexisted` only chooses the wording:
returned rows + agent gone -> filter_agent_deregistered (free)
empty + has_memories + gone -> filter_agent_deregistered
empty + no memories + known -> filter_agent_empty
empty + no memories + unknown -> filter_agent_unknown
anything else -> silence
`agent_preexisted` comes from the route's own `get_or_create_agent`, via
a new `registration_ctx` out-dict in the same shape as `diagnostic_ctx` /
`warnings_ctx` / `recall_ctx`. That call already does the `agents`
lookup, so it is free — and it *has* to come from there, because by the
time the search has run the row exists whether or not it did beforehand,
and asking afterwards would report every typo as a registered agent. An
out-dict rather than a changed return type so the other seven callers
stay untouched.
Known boundary: an admin credential skips `get_or_create_agent`
(`if auth.tenant_id:`), so it has no free pre-existence signal and is not
told an agent is deregistered on a NON-empty result. Buying that would
put a query back on the success path. Documented on the `tenant_scoped`
fixture.
## Ratchets
`test_c27_strict_fleet_scoping` pins two things, and the probe is a fifth
fleet-scoped read:
* it must go through `_fleet_scope_clause`, never a hand-rolled
`fleet_id.in_(fleet_ids)` — "exactly how A54 leaked". It does.
* the helper's call-site count moves 5 -> 6, with the reasoning recorded
at the constant.
`strict=False` on purpose: non-strict is a SUPERSET (it also admits
tenant-shared null-fleet rows and `scope_org`), so the probe stays at
least as permissive as the search it explains and can never report "no
memories" for rows the search could see.
An earlier draft short-circuited ahead of `enforce_fleet_read_many` and
the usage metering — a caller naming a fleet it has no rights to got a
200 with a warning instead of a 403, and the warning disclosed whether an
agent id exists. `test_c27_strict_fleet_scoping` and
`test_h05_multi_fleet_read_gate` caught it. That whole class is gone with
the short-circuit: the search now always runs.
## Which field is explained (review round 2)
Keyed on `filter_agent_id` whenever it is set, not on the resolved
identity. `_resolve_read_identity` resolves `caller_agent_id or
filter_agent_id`, but only `filter_agent_id` becomes
`WHERE memories.agent_id = ?` — so keying on the identity explained the
wrong id when a caller sent a valid `caller_agent_id` beside a typo'd
filter, and the typo went unreported. Measured:
filter=TYPO only -> ['filter_agent_unknown']
caller=REAL + filter=TYPO -> [] <-- the gap
caller=REAL + filter=TYPO -> ['filter_agent_empty'] naming 'agnet-real' (fixed)
Two knock-on corrections the first cut of this fix needed:
* `agent_preexisted` describes the RESOLVED identity, which is not
always the id being explained. Applying one agent's registration
state to another reported a typo'd filter as "registered but empty",
so `preexistence_of` now names which id the flag is about and the
free signal is used only when it matches. Otherwise the probe is
asked, which is one query on a path that already returned nothing.
* `details` hardcoded `filter_agent_id`, which was wrong whenever the
id came from `caller_agent_id` — a client acting on it would have
corrected the wrong knob. It now carries `field` plus that field's
own key.
`caller_agent_id` alone still reports, but as visibility rather than
filtering: it restricts no rows, it only decides which `scope_agent` rows
are visible, so its message must not claim to have matched nothing.
## `/recall` too
It parses the same `SearchRequest` and had the identical silent empty.
Leaving it out is how the two routes came to disagree about what these
fields mean — the reason `_resolve_read_identity` is shared at all. Its
response gains `warnings`, seeded from the probe and then extended by the
pipeline, so the field means what `/search`'s does rather than carrying
only this one code.
## Known limitation, pre-existing
The read path registers the asserted id via `get_or_create_agent`, so a
repeated typo is registered by the first call and the second reports
`filter_agent_empty` rather than `filter_agent_unknown`. Both still say
the filter matched nothing, and the wording is advisory. Not introduced
here and not fixed here: the clean answers are to stop registering on
reads, or agent soft-delete (a `deleted_at` plus making
`uq_agents_tenant_agent` partial, or re-registration breaks) — both
larger than a warning warrants.
## Verification
- `tests/test_caura723_agent_scope_warning.py` — 12 cases: the four
states, the fleet-narrowed probe, `/recall`, the agent-scoped 403 left
unchanged, no-filter and good-filter silence, a failing probe leaving
search working, the no-cost-on-success contract, and both field-keying
cases (a typo'd filter beside a valid caller id, and a caller id alone
reported as visibility rather than filtering).
- Targeted sweep post-rebase: 110 passed (`test_c27_strict_fleet_scoping`,
`test_h05_multi_fleet_read_gate`, `test_route_authz_gaps`,
`test_search_recall_tracked_flag`, `test_c4_recall_items_alias`,
`test_mcp_recall`, `test_agent_admin_authz`).
- Full `tests/`: 39 failures, byte-identical to `main`'s set on this
machine (pre-existing FTS / relation-weight / request-observation env
failures). `core-storage-api/tests/`: 369 passed, same 2 pre-existing
CORS failures.
- ruff check and format clean; mypy identical to `main` on every changed
file. `do_not_touch_sentinel` all 35 survive, `tenant_scope_gate`
exit 0, `legacy_name_ratchet` **"No new lines."** — the test reaches the
dual-read Settings field through `tests/_legacy_contracts`'
`LEGACY_API_KEY_FIELD` rather than minting the literal, in keeping with
#1436-#1438.
No schema changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Eyal Blyachman <eyal.b@caura.ai>
f8e0abf to
e7e0798
Compare
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
SummaryThis PR adds a diagnostic warning ( High Issues
|
SummaryThis PR adds a well-documented "agent scope" diagnostic (CAURA-723) that explains empty search/recall results caused by an unknown or deregistered Medium/Low IssuesCross-tenant reads: peer-tenant agents get mislabeled as "deregistered"Severity: Medium 🤖 Claude Code PromptReviewed by |
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
…lt is empty
A tenant-scoped caller that passes a wrong `filter_agent_id` gets
`HTTP 200 · items [] · warnings null` — byte-identical to a correct id
that simply has nothing relevant to say. Measured, one memory written by
`agent-real`:
filter_agent_id = "agent-real" -> 200 items=1
filter_agent_id = "agnet-real" -> 200 items=0 warnings=None
The filter is a SQL `WHERE memories.agent_id = ?`, so a typo matches no
rows and the query returns nothing exactly as an empty query would. It
fails in the safe-looking direction — an empty list reads as "no data
yet", the most ordinary thing a memory product can say, so nobody
investigates. It cost a 589-query benchmark run against an empty store.
## Scope: a tenant-scoped-caller bug only
An agent-scoped credential carries a verified `X-Agent-ID`, and
`enforce_self_agent` **403s** it for naming any agent but itself —
measured, for a typo and for a peer id. So the silent path needs a
credential with no agent identity, asserting the id in the body.
Not a corner: it is the dashboard's hand-typed `filter_agent_id` text box
(tenant-scoped by its own comment, "no X-Agent-ID"), the documented
`caura_recall` parameter on the public for-agents page, every benchmark
harness we own, and the multi-user backend shape where one key serves
many end users.
## When it runs, and what it costs
Only after the search, and the storage probe only when the search came
back EMPTY. **A successful agent-filtered search pays nothing** — no
extra round trip, no extra query. Pinned by
`test_a_successful_search_makes_no_probe_call`, which counts calls rather
than trusting the reading.
This is the second attempt, after review of #1434. The first probed
BEFORE the search to skip the embedding call on a misconfigured request —
but that saving lands on requests already broken, which are rare, while
the cost lands on every healthy one, which is the norm. Wrong way round
on expected cost. The review also caught that the docstring claimed the
probe "runs only when a search is about to return nothing", which was
false of that ordering; it is true of this one.
The empty path is now one query, not two: `include_agent_registered`
lets the probe skip the `agents` lookup, because the route already made
it (see below).
## Which fact decides what, and why not the obvious one
The natural implementation — look the id up in `agents`, and if absent
report "no such agent" — is wrong in the direction that misleads.
`DELETE /agents/{id}` says so itself: *"Delete an agent. Memories written
by this agent are NOT deleted."* The rows stay live and searchable with
no agent row, and rows predating agent tracking were never registered at
all. A missing agent row means "deregistered or never registered", never
"nothing to find".
So `has_memories` establishes there was nothing to find, and
`agent_preexisted` only chooses the wording:
returned rows + agent gone -> filter_agent_deregistered (free)
empty + has_memories + gone -> filter_agent_deregistered
empty + no memories + known -> filter_agent_empty
empty + no memories + unknown -> filter_agent_unknown
anything else -> silence
`agent_preexisted` comes from the route's own `get_or_create_agent`, via
a new `registration_ctx` out-dict in the same shape as `diagnostic_ctx` /
`warnings_ctx` / `recall_ctx`. That call already does the `agents`
lookup, so it is free — and it *has* to come from there, because by the
time the search has run the row exists whether or not it did beforehand,
and asking afterwards would report every typo as a registered agent. An
out-dict rather than a changed return type so the other seven callers
stay untouched.
Known boundary: an admin credential skips `get_or_create_agent`
(`if auth.tenant_id:`), so it has no free pre-existence signal and is not
told an agent is deregistered on a NON-empty result. Buying that would
put a query back on the success path. Documented on the `tenant_scoped`
fixture.
## Ratchets
`test_c27_strict_fleet_scoping` pins two things, and the probe is a fifth
fleet-scoped read:
* it must go through `_fleet_scope_clause`, never a hand-rolled
`fleet_id.in_(fleet_ids)` — "exactly how A54 leaked". It does.
* the helper's call-site count moves 5 -> 6, with the reasoning recorded
at the constant.
`strict=False` on purpose: non-strict is a SUPERSET (it also admits
tenant-shared null-fleet rows and `scope_org`), so the probe stays at
least as permissive as the search it explains and can never report "no
memories" for rows the search could see.
An earlier draft short-circuited ahead of `enforce_fleet_read_many` and
the usage metering — a caller naming a fleet it has no rights to got a
200 with a warning instead of a 403, and the warning disclosed whether an
agent id exists. `test_c27_strict_fleet_scoping` and
`test_h05_multi_fleet_read_gate` caught it. That whole class is gone with
the short-circuit: the search now always runs.
## Which field is explained (review round 2)
Keyed on `filter_agent_id` whenever it is set, not on the resolved
identity. `_resolve_read_identity` resolves `caller_agent_id or
filter_agent_id`, but only `filter_agent_id` becomes
`WHERE memories.agent_id = ?` — so keying on the identity explained the
wrong id when a caller sent a valid `caller_agent_id` beside a typo'd
filter, and the typo went unreported. Measured:
filter=TYPO only -> ['filter_agent_unknown']
caller=REAL + filter=TYPO -> [] <-- the gap
caller=REAL + filter=TYPO -> ['filter_agent_empty'] naming 'agnet-real' (fixed)
Two knock-on corrections the first cut of this fix needed:
* `agent_preexisted` describes the RESOLVED identity, which is not
always the id being explained. Applying one agent's registration
state to another reported a typo'd filter as "registered but empty",
so `preexistence_of` now names which id the flag is about and the
free signal is used only when it matches. Otherwise the probe is
asked, which is one query on a path that already returned nothing.
* `details` hardcoded `filter_agent_id`, which was wrong whenever the
id came from `caller_agent_id` — a client acting on it would have
corrected the wrong knob. It now carries `field` plus that field's
own key.
`caller_agent_id` alone still reports, but as visibility rather than
filtering: it restricts no rows, it only decides which `scope_agent` rows
are visible, so its message must not claim to have matched nothing.
## `/recall` too
It parses the same `SearchRequest` and had the identical silent empty.
Leaving it out is how the two routes came to disagree about what these
fields mean — the reason `_resolve_read_identity` is shared at all. Its
response gains `warnings`, seeded from the probe and then extended by the
pipeline, so the field means what `/search`'s does rather than carrying
only this one code.
## Known limitation, pre-existing — and why the wording carries it
`get_or_create_agent` runs on the READ path and creates a row for
whatever id was asserted, so a search mints agent rows from free-text
input. The first request carrying a typo reports `filter_agent_unknown`
correctly and, in doing so, creates the row that makes the next one
report `filter_agent_empty`:
1st search, filter_agent_id="agnet-real" -> filter_agent_unknown
2nd search, same typo -> filter_agent_empty
A harness reusing one wrong id gets the sharp signal once and the soft
one thereafter — the exact shape of the incident behind this work.
An earlier revision noted this and called the wording "advisory", which
under-rated it: "agent is registered but has no memories" reads as a
benign new-agent state, so a caller takes "registered" for "the id is
right" and stops looking. That is the outcome the warning exists to
prevent, so the message now says outright that registration is weak
evidence and that a repeated typo lands there. Both codes are kept
because the first-occurrence signal is accurate and worth having.
Pinned by `test_a_repeated_typo_still_warns_and_never_reads_as_benign`,
which asserts the second request still warns, still names the offending
id, and does not read as benign — a test rather than a commit note, so a
future change to registration makes it visible either way.
Not fixable from here. The root fix is to stop registering on reads, and
it is security-adjacent: the route needs the row for trust-level fleet
forcing and `enforce_fleet_read_many`, so "don't create it" first
requires deciding what trust and fleet apply to an unknown agent.
Tracked as CAURA-724, which also covers the two consequences beyond this
warning — a read with a write side effect, and a tenant key being able
to mint unlimited agent rows one search at a time.
## Verification
- `tests/test_caura723_agent_scope_warning.py` — 13 cases: the four
states, the fleet-narrowed probe, `/recall`, the agent-scoped 403 left
unchanged, no-filter and good-filter silence, a failing probe leaving
search working, the no-cost-on-success contract, and both field-keying
cases (a typo'd filter beside a valid caller id, and a caller id alone
reported as visibility rather than filtering), and the repeated-typo
downgrade.
- Targeted sweep post-rebase: 110 passed (`test_c27_strict_fleet_scoping`,
`test_h05_multi_fleet_read_gate`, `test_route_authz_gaps`,
`test_search_recall_tracked_flag`, `test_c4_recall_items_alias`,
`test_mcp_recall`, `test_agent_admin_authz`).
- Full `tests/`: 39 failures, byte-identical to `main`'s set on this
machine (pre-existing FTS / relation-weight / request-observation env
failures). `core-storage-api/tests/`: 369 passed, same 2 pre-existing
CORS failures.
- ruff check and format clean; mypy identical to `main` on every changed
file. `do_not_touch_sentinel` all 35 survive, `tenant_scope_gate`
exit 0, `legacy_name_ratchet` **"No new lines."** — the test reaches the
dual-read Settings field through `tests/_legacy_contracts`'
`LEGACY_API_KEY_FIELD` rather than minting the literal, in keeping with
#1436-#1438.
No schema changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Eyal Blyachman <eyal.b@caura.ai>
6c04592 to
dbf54c0
Compare
|
Claude Code Review — skipped: PR author 'eyal-bl' is not a public member of the 'caura-ai' org |
SummaryThis PR adds a well-documented, thoughtfully-tested feature (CAURA-723) that explains why an agent-filtered search/recall returned an empty result. The core logic (probe-only-on-empty, Medium/Low Issues
|
…lt is empty
A tenant-scoped caller that passes a wrong
filter_agent_idgetsHTTP 200 · items [] · warnings null— byte-identical to a correct id that simply has nothing relevant to say. Measured, one memory written byagent-real:The filter is a SQL
WHERE memories.agent_id = ?, so a typo matches no rows and the query returns nothing exactly as an empty query would. Nothing in the response separates them, and it fails in the safe-looking direction — an empty list reads as "no data yet", the most ordinary thing a memory product can say, so nobody investigates. It cost a 589-query benchmark run against an empty store.Scope: this is a tenant-scoped-caller bug only
An agent-scoped credential carries a verified
X-Agent-ID, andenforce_self_agent403s it for naming any agent but itself — measured, for a typo and for a peer id. So the silent path needs a credential with no agent identity, asserting the id in the body.That is not a corner: it is the dashboard's hand-typed
filter_agent_idtext box (tenant-scoped by its own comment, "no X-Agent-ID"), the documentedcaura_recallparameter on the public for-agents page, every benchmark harness we own, and the multi-user backend shape where one key serves many end users. Gating on the asserted identity keeps the probe off the path where the condition is unreachable.Which table decides what, and why not the obvious one
The natural implementation — look the id up in
agents, and if absent report "no such agent" and skip — is wrong in the direction that loses data.DELETE /agents/{id}says so itself: "Delete an agent. Memories written by this agent are NOT deleted." The rows stay live and searchable with no agent row, and rows predating agent tracking were never registered at all. So:has_memories(thememoriestable) decides whether to skip. It is the only fact that can prove the search would return nothing.agent_registered(theagentstable) decides only the wording, where being wrong costs a slightly-off message.Four states, three codes:
Row three is the regression guard: the memories outlive the agent row, so they must still be returned.
Ordering, and a hole the ordering first created
The probe is computed immediately after identity resolution and BEFORE the route's
get_or_create_agent, which registers the asserted id — a typo included. Probing after it would find anagentsrow the read itself had just created, and every typo would report as registered.The first draft also RETURNED there, which skipped
enforce_fleet_read_manyand the usage metering: a caller naming a fleet it has no rights to got a 200 with a warning instead of a 403, and the warning disclosed whether an agent id exists.test_c27_strict_fleet_scopingandtest_h05_multi_fleet_read_gatecaught it. The short-circuit now sits after every gate; the saving it exists for — the embedding call and the scored search (plus the recall model call on/recall) — is still entirely ahead of that point.Cost
One round trip answering both halves, only on the asserted-identity path with a filter set. Two
LIMIT 1index probes:ix_memories_tenant_agentanduq_agents_tenant_agent. A search that returns results pays that and nothing else; a misconfigured one is strictly cheaper than today, trading an embedding API call and a vector scan for two index hits.read=Truedeliberately: the search it explains reads the replica, so answering from the same replica keeps the probe's story consistent with the result rather than reporting rows the search could not see.The probe never raises. A failure degrades to today's behaviour — an unexplained result — because a diagnostic hint must not be able to fail a working search.
Ratchets moved, both deliberately
test_c27_strict_fleet_scopingpins two things, and the probe is a fifth fleet-scoped read:_fleet_scope_clause, never a hand-rolledfleet_id.in_(fleet_ids)— "exactly how A54 leaked". It does.The probe calls the helper with
strict=Falseon purpose. Non-strict is a SUPERSET (it also admits tenant-shared null-fleet rows andscope_org), so the probe stays at least as permissive as the search it explains and can never skip a search that would have returned rows. Threading the tenant's realstrict_fleet_scopingwould tighten it and buy exactly that risk./recalltooIt parses the same
SearchRequestand had the identical silent empty. Leaving it out is how the two routes came to disagree about what these fields mean — the reason_resolve_read_identityis shared at all. Its response gainswarnings, seeded with any scope warning and then extended by the pipeline, so the field means what/search's does rather than carrying only this one code.Known limitation, pre-existing
The read path registers the asserted id via
get_or_create_agent, so a repeated typo is registered by the first call and the second reportsfilter_agent_emptyrather thanfilter_agent_unknown. Both still say the filter matched nothing, and the wording is advisory. Not introduced here and not fixed here: the clean answers are to stop registering on reads, or agent soft-delete (adeleted_atplus makinguq_agents_tenant_agentpartial, or re-registration breaks) — both larger changes than a warning warrants.Verification
tests/test_caura723_agent_scope_warning.py— 9 cases: the four states, the fleet-narrowed probe,/recall, the agent-scoped 403 left unchanged, no-filter and good-filter silence, and a failing probe leaving search working. 5 fail without the source change; the other 4 are no-regression guards that must hold in both directions.test_c27_strict_fleet_scoping,test_h05_multi_fleet_read_gate,test_search_recall_tracked_flag,test_search_caller_identity,test_c4_recall_items_alias,test_mcp_recall,test_route_authz_gaps).tests/: 39 failures, byte-identical tomain's set on this machine (pre-existing FTS / relation-weight / request-observation env failures).core-storage-api/tests/: 369 passed, same 2 pre-existing CORS failures.mainon every changed file.do_not_touch_sentinelall 35 survive,tenant_scope_gateexit 0,legacy_name_ratchetreports one exemptlegacy-name-okline: the test patchessettings.memclaw_api_keybecause that is the dual-read field auth Path 2 reads — patchingcaura_api_keyis inert, since the two collapse at validation time.No schema changes.
Summary
Related Issue
Type of Change
How Has This Been Tested?
Checklist
ruff checkandruff format --checkpassmypypassespytestpasses locallyCHANGELOG.mdunder theUnreleasedsection (if user-facing)Additional Notes