Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions crates/fractal-api-core/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9612,6 +9612,12 @@ fn assistant_grant_subject_maintenance_checks(
SELECT pending.grant_subject, 'retracted authority effective-state recomputation'
FROM assistant_grant_recompute work
JOIN assistant_grant_pending pending ON pending.claim_id=work.claim_id
UNION
SELECT grant_subject, 'authenticated grant attachment in progress'
FROM assistant_grant_attachments
UNION
SELECT grant_subject, 'attachment conflict-witness cleanup'
FROM assistant_grant_attachment_witnesses
ORDER BY 1,2",
)?;
let rows = statement.query_map([], |row| {
Expand Down
642 changes: 642 additions & 0 deletions crates/smallclaims/src/store/assistant_grant_attachment_tests.rs

Large diffs are not rendered by default.

509 changes: 337 additions & 172 deletions crates/smallclaims/src/store/assistant_grant_authority.rs

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,8 @@ fn measure_backfill(config: Config) {
DELETE FROM assistant_grant_authority;
DELETE FROM assistant_grant_effective;
DELETE FROM assistant_grant_pending;
DELETE FROM assistant_grant_attachments;
DELETE FROM assistant_grant_attachment_witnesses;
DELETE FROM assistant_grant_dirty;
DELETE FROM assistant_grant_auth_cursor;
DELETE FROM assistant_grant_additions;
Expand Down
201 changes: 170 additions & 31 deletions crates/smallclaims/src/store/assistant_grant_authority_tests.rs

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ fn receipt_projection_work_state(connection: &Connection) -> Vec<Vec<Vec<SqlValu
"SELECT claim_id FROM assistant_grant_dirty ORDER BY claim_id",
"SELECT id,writer,member_key,lower_sequence,upper_sequence,sequence,envelope_hash,position,claim_id,active_claim FROM assistant_grant_additions ORDER BY id",
"SELECT writer,claim_id FROM assistant_grant_auth_work ORDER BY writer,claim_id",
"SELECT claim_id,candidate_kind,grant_subject,origin,actor,after_claim_id,upper_claim_id FROM assistant_grant_attachments ORDER BY claim_id,candidate_kind",
"SELECT base_claim_id,grant_subject,principal,owner,profile,expires_at,revoked,grant_record FROM assistant_grant_attachment_witnesses ORDER BY base_claim_id",
"SELECT queue,owner,subowner,sequence FROM assistant_grant_queue_owners ORDER BY queue,sequence",
"SELECT queue,sequence FROM assistant_grant_queue_sequences ORDER BY queue",
"SELECT key,value FROM meta WHERE key LIKE 'assistant_grant_owner_cursor/%' OR key LIKE 'assistant_grant_owner_upper/%' OR key='assistant_grant_maintenance_turn' ORDER BY key",
].into_iter().map(|sql| {
let mut statement = connection.prepare(sql).unwrap();
let columns = statement.column_count();
Expand Down
61 changes: 55 additions & 6 deletions docs/st3/external-assistants.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,12 +253,25 @@ work index, maintained atomically with unauthenticated rows and cursor/recheck
obligations. Each task then seeks that claim's own receipts in the added range.
One claim selection, receipt attempt or claim completion consumes a budget unit;
unrelated history is never enumerated. Total work is proportional to needing
grant rows and their own matching receipts, not all writer receipts. Ordinary
dirty authentication receives the first half of each transaction with a durable
round-robin cursor, so an addition backlog cannot keep dirty subjects fenced.
Late dirty work sorting before the saved cursor wraps within the same slice,
without waiting for a later maintenance wakeup. Repeated cursor or recheck writes
retain one work-set entry even under an outer SQL upsert's conflict policy.
grant rows and their own matching receipts, not all writer receipts.
After demotion, maintenance uses one durable round-robin queue turn, in order:
dirty authentication, proof rechecks, proof invalidation, membership retraction,
coverage addition, positive attachment, conflict-witness cleanup, initial history
backfill. Each nonempty turn performs one charged unit and advances the persisted
turn before a slice can end, including `Duration::ZERO`. Empty queues cost a
bounded indexed probe each; dispatch visits at most eight queues per unit.
Thus, absent demotion work, each runnable queue receives a unit within eight
one-unit slices even under continuous arrivals to other queues. There are no
per-queue slice reservations. Every owner-based queue (the first seven turns)
also uses a locally assigned monotone integer admission sequence, an indexed
sequence cursor, and a fixed per-round upper frontier. Re-enqueuing an owner
already present does not change its sequence. Each visit advances to the next
owner even if that owner's authentication or pair cursor is unfinished. New
arrivals, regardless of their content-hash ordering, wait behind the current
round's owners. An owner present at round start receives a visit within that
round's owner count; its own page cursor resumes on later rounds.
Repeated cursor or recheck writes retain one work-set entry even under an outer
SQL upsert's conflict policy.
A claim with a durable proof recheck has exactly one authentication-cursor
owner: the recheck verifier. Ordinary dirty dispatch invokes that same verifier
step and discharges the obligation on completion, so even a one-unit or
Expand All @@ -271,6 +284,42 @@ the budget. A key learned below a cursor can restore a demoted row through its
independent addition task. On upgrade, committed legacy writer-retry obligations
become durable verifier rechecks before their old queue is dropped. Old key-only
cursors are requeued rather than interpreted in the new start/key ordering.
Positive attachment uses a separate durable task for each authenticated claim
and opposite claim kind. Its exact subject/origin/actor/kind index fixes a
high-water claim ID when the task is created; a monotone keyset cursor visits one
candidate per unit, including incompatible candidates. Task completion and
conflict-witness cleanup also consume the shared 64-unit transaction budget
(a zero-duration slice performs at most one unit). Authentication and task
creation commit atomically; no append or receive transaction enumerates the
opposite history. A first local base with no opposite history remains immediately
usable. Late opposite claims create their own tasks, including pairs beyond an
older task's high-water mark.

The subject remains fenced until its attachment tasks and witness cleanup finish.
Cleanup seeks exactly one witness by its queue admission sequence before
probing that subject's attachment index. It advances and charges the visit whether
the witness is deleted or skipped because its own subject still has tasks.
Unrelated attachment backlogs do not prevent eligible cleanup; a long ineligible
prefix consumes separate bounded visits instead of an uncharged eligibility scan.
The fixed round frontier prevents new arrivals from repeatedly overtaking
waiting witnesses.
Before changing an effective base, maintenance retains its previous effective
record as a conflict witness, so partially attached restrictive transitions cannot
hide an established cross-subject conflict. Demotion cancels the demoted claim's
tasks; attachment candidates must still have authenticated endpoints. Private
doctor diagnostics report `authenticated grant attachment in progress` and
`attachment conflict-witness cleanup`. Total attachment work and eventual pair
storage can still be \(O(\text{bases}\times\text{transitions})\), but each
transaction is bounded and its cursor survives restart.

This adds task/witness tables, a queue-owner admission registry and indexes without
changing projection version 1 or rescanning raw history. Installation indexes only
the already-queued owner set once; persistent queue insert/delete triggers keep it
current atomically. Sequence counters survive queue retirement so deleted owners'
positions cannot be reused. Earlier projection builds attached authenticated
claims atomically, so their committed pairs need no replay; unfinished
authentication/backfill continues through the new bounded tasks.

Subject readiness starts with indexed subject claims and probes the queued
obligations by their keys; unrelated queue growth does not cause a queue scan.
It is not constant cost: a subject with many claims or matching proof/range
Expand Down
Loading