From f9eb12247ffe8afb9056573d27404409c8c363b3 Mon Sep 17 00:00:00 2001 From: fedster99 Date: Fri, 24 Jul 2026 13:06:41 +0200 Subject: [PATCH] fix(progress): keep incomplete bodies below 100 percent --- SESSION_HANDOFF.md | 32 +++-- apps/api/src/__tests__/schema.test.ts | 25 +++- .../__tests__/sync-engine.integration.test.ts | 91 +++++++++++++ apps/api/src/repository.ts | 7 +- ...0023_exact_body_completion_percentages.sql | 126 ++++++++++++++++++ .../supabase/migrations/public/manifest.json | 6 +- docs/agent/reliability-invariants.md | 2 +- ...table-body-policy-row-accurate-coverage.md | 8 +- .../reliability-and-three-lanes.md | 6 +- docs/schema.md | 2 +- docs/spec-conformance.md | 4 +- 11 files changed, 283 insertions(+), 26 deletions(-) create mode 100644 apps/api/supabase/migrations/public/0023_exact_body_completion_percentages.sql diff --git a/SESSION_HANDOFF.md b/SESSION_HANDOFF.md index bb1d2bc..0631b1e 100644 --- a/SESSION_HANDOFF.md +++ b/SESSION_HANDOFF.md @@ -1,5 +1,16 @@ # Session Handoff +## 2026-07-24 — Exact body completion percentages + +- Branch `fedster99/fix-body-progress-rounding` makes 100% an exact completion + claim for live and priority account progress and per-folder current body + progress. Incomplete ratios are floored, so 199 fetched bodies out of 200 + reports 99%, not 100%. +- Public migration `0023_exact_body_completion_percentages` replaces + `imap_account_progress`; the repository applies the same rule to folder rows. + A real-Postgres Scenario R regression covers priority, live, and folder + progress at the rounding boundary. + ## 2026-07-23 — Content extract and body-store seam - Branch `fedster99/content-body-store-seam` adds public migration @@ -165,20 +176,16 @@ Commit/merge, immutable-image publication, and downstream production re-pin are the next actions. -Last updated: 2026-07-23 +Last updated: 2026-07-24 This is the tracked restart point for future agents. Keep it concise, factual, and safe to publish. Put private local notes, credentials, customer/provider probes, and one-off scratch work in `.context/` instead. ## Current Branch -- Active branch `fedster99/content-body-store-seam` implements ADR 0028's - evidence-first content seam. The cloud image re-pin is deliberately deferred - to a separate private-repo task. -- Active branch `fedster99/live-body-coverage-policy` makes - `body_fetch_policy` mutable for existing accounts and replaces counter-derived - live/priority body coverage with current-row evidence. ADR 0027 records the - contract. This change adds one view-replacement migration and one partial - live-body progress index. +- Active branch `fedster99/fix-body-progress-rounding` makes 100% an exact + completion claim for live and priority account progress and per-folder + current body progress. Migration `0023` floors incomplete ratios while + preserving the integer API fields. - Active branch `fedster99/fix-reconcile-health-after-repair` separates observed reconcile gaps from unresolved reconcile state. A pass that fully tombstones provider-missing rows or backfills missing-in-DB UIDs now finishes clean and can return the account to `HEALTHY`; the run still records its bounded gap count. Missing-in-DB overflow is detected with a 5,001st sentinel row, remains degraded, and retries on the next full-sync cadence. ADR 0026 records the contract; no migration or public API change is required. - Local branch `fedster99/smtp-account-lock-v2` is rebased onto current `origin/main` at `88c025d` and is intentionally uncommitted/unpushed. It closes @@ -349,10 +356,9 @@ This is the tracked restart point for future agents. Keep it concise, factual, a ## Next Best Actions -- Review and land the feature-branch PR for - `fedster99/live-body-coverage-policy`. Publish a new immutable public-core - image only after human merge, then update downstream Signal through its - public-core prebuild and re-pin flow. +- Review and land `fedster99/fix-body-progress-rounding`. Publish a new + immutable public-core image only after human merge, then update downstream + Signal through its public-core prebuild and re-pin flow. - Review and land `fedster99/fix-reconcile-health-after-repair`, publish its immutable image, then re-pin downstream consumers. Confirm ordinary provider delete/move drift records a nonzero gap count without leaving a fully repaired account stuck `DEGRADED`. - Review and land `fedster99/fix-imap-abort-race`, publish its immutable image, then re-pin downstream consumers and canary the Sent/full-sweep deadline boundary. Confirm there are no further `Already logged out`, `process.uncaughtException`, or Render restart events. - Keep this file updated at the end of substantial sessions. diff --git a/apps/api/src/__tests__/schema.test.ts b/apps/api/src/__tests__/schema.test.ts index bcfe016..d3d7aed 100644 --- a/apps/api/src/__tests__/schema.test.ts +++ b/apps/api/src/__tests__/schema.test.ts @@ -48,6 +48,10 @@ const contentExtractBodyStoreMigrationPath = resolve( process.cwd(), "supabase/migrations/public/0022_content_extract_body_store.sql" ); +const exactBodyCompletionPercentagesMigrationPath = resolve( + process.cwd(), + "supabase/migrations/public/0023_exact_body_completion_percentages.sql" +); describe("initial schema", () => { it("adds a bounded search extract without moving full body payloads into metadata", async () => { @@ -145,9 +149,9 @@ describe("initial schema", () => { const version = await getRequiredPublicSchemaVersion(); const sql = await readPublicMigrations(); - expect(version).toBe("0022_content_extract_body_store"); + expect(version).toBe("0023_exact_body_completion_percentages"); expect(manifest).toEqual({ - schemaVersion: "0022_content_extract_body_store", + schemaVersion: "0023_exact_body_completion_percentages", migrations: [ { id: "0001_imap_mirror", file: "0001_imap_mirror.sql" }, { id: "0002_stuck_degraded_escalation", file: "0002_stuck_degraded_escalation.sql" }, @@ -170,7 +174,11 @@ describe("initial schema", () => { { id: "0019_authored_delivery_evidence", file: "0019_authored_delivery_evidence.sql" }, { id: "0020_threading_fingerprint_closure", file: "0020_threading_fingerprint_closure.sql" }, { id: "0021_row_accurate_body_progress", file: "0021_row_accurate_body_progress.sql" }, - { id: "0022_content_extract_body_store", file: "0022_content_extract_body_store.sql" } + { id: "0022_content_extract_body_store", file: "0022_content_extract_body_store.sql" }, + { + id: "0023_exact_body_completion_percentages", + file: "0023_exact_body_completion_percentages.sql" + } ] }); expect(sql).toContain("CREATE TABLE IF NOT EXISTS public.imap_accounts"); @@ -202,6 +210,17 @@ describe("initial schema", () => { expect(sql).toContain("FUNCTION public.imap_search_extract_fts"); }); + it("keeps incomplete body percentages below 100", async () => { + const sql = await readFile(exactBodyCompletionPercentagesMigrationPath, "utf8"); + + expect(sql).toContain("CREATE OR REPLACE VIEW public.imap_account_progress"); + expect(sql).toContain("b.priority_bodies_fetched_count >= b.priority_bodies_target_count"); + expect(sql).toContain("b.live_bodies_fetched_count >= b.live_bodies_target_count"); + expect(sql.match(/ELSE floor\(/g)).toHaveLength(2); + expect(sql).not.toContain("stripe"); + expect(sql).not.toContain("tenant"); + }); + it("indexes delivery fingerprints for bounded threading closure", async () => { const sql = await readFile(threadingFingerprintClosureMigrationPath, "utf8"); diff --git a/apps/api/src/__tests__/sync-engine.integration.test.ts b/apps/api/src/__tests__/sync-engine.integration.test.ts index f7691dd..596c319 100644 --- a/apps/api/src/__tests__/sync-engine.integration.test.ts +++ b/apps/api/src/__tests__/sync-engine.integration.test.ts @@ -2987,6 +2987,97 @@ integration("sync-engine integration (real Postgres + fixture IMAP)", () => { bodies_pct: 0 }) ])); + + await h.pool.query( + ` + INSERT INTO public.imap_message_bodies ( + message_id, raw_mime, raw_bytes, raw_truncated, body_text + ) + SELECT id, NULL, 20, false, 'marker repaired' + FROM public.imap_messages + WHERE account_id = $1 + AND folder_path = 'INBOX' + AND uidvalidity = 72001 + AND uid = 4 + `, + [h.account.id] + ); + await h.pool.query( + ` + INSERT INTO public.imap_messages ( + account_id, folder_path, uidvalidity, uid, internal_date, + subject, body_fetched_at, deleted_in_provider, window_status + ) + SELECT + $1, + 'INBOX', + 72001, + uid, + now(), + 'rounding boundary ' || uid, + now(), + false, + 'IN_WINDOW' + FROM generate_series(7, 202) AS uid + `, + [h.account.id] + ); + await h.pool.query( + ` + INSERT INTO public.imap_message_bodies ( + message_id, raw_mime, raw_bytes, raw_truncated, body_text + ) + SELECT id, NULL, 20, false, 'rounding boundary complete' + FROM public.imap_messages + WHERE account_id = $1 + AND folder_path = 'INBOX' + AND uidvalidity = 72001 + AND uid BETWEEN 7 AND 202 + `, + [h.account.id] + ); + + const roundingBoundaryProgress = ( + await h.pool.query<{ + priority_bodies_fetched_count: number; + priority_bodies_target_count: number; + priority_bodies_complete_pct: number; + live_bodies_fetched_count: number; + live_bodies_target_count: number; + live_bodies_complete_pct: number; + }>( + ` + SELECT + priority_bodies_fetched_count, + priority_bodies_target_count, + priority_bodies_complete_pct, + live_bodies_fetched_count, + live_bodies_target_count, + live_bodies_complete_pct + FROM public.imap_account_progress + WHERE account_id = $1 + `, + [h.account.id] + ) + ).rows[0]; + expect(roundingBoundaryProgress).toEqual({ + priority_bodies_fetched_count: 199, + priority_bodies_target_count: 200, + priority_bodies_complete_pct: 99, + live_bodies_fetched_count: 199, + live_bodies_target_count: 200, + live_bodies_complete_pct: 99 + }); + + const roundingBoundaryDetails = await h.repository.getAccountDetails(h.account.id); + expect(roundingBoundaryDetails?.folders).toEqual(expect.arrayContaining([ + expect.objectContaining({ + path: "INBOX", + live_bodies_fetched_count: 199, + live_bodies_target_count: 200, + bodies_pct: 99 + }) + ])); }); it("Scenario S — settings PATCH makes an existing non-priority live body eligible on the next sync", async () => { diff --git a/apps/api/src/repository.ts b/apps/api/src/repository.ts index fa13777..63cb71a 100644 --- a/apps/api/src/repository.ts +++ b/apps/api/src/repository.ts @@ -696,7 +696,12 @@ export class MirrorRepository { END AS headers_pct, CASE WHEN COALESCE(b.live_bodies_target_count, 0) > 0 - THEN round((b.live_bodies_fetched_count::numeric / b.live_bodies_target_count::numeric) * 100)::int + THEN CASE + WHEN b.live_bodies_fetched_count >= b.live_bodies_target_count THEN 100 + ELSE floor( + (b.live_bodies_fetched_count::numeric / b.live_bodies_target_count::numeric) * 100 + )::int + END WHEN f.live_window_target_count IS NOT NULL THEN 100 ELSE 0 END AS bodies_pct, diff --git a/apps/api/supabase/migrations/public/0023_exact_body_completion_percentages.sql b/apps/api/supabase/migrations/public/0023_exact_body_completion_percentages.sql new file mode 100644 index 0000000..b2f2a32 --- /dev/null +++ b/apps/api/supabase/migrations/public/0023_exact_body_completion_percentages.sql @@ -0,0 +1,126 @@ +-- A current live body percentage is complete only when its fetched count +-- reaches its target. Floor incomplete ratios so 199/200 cannot round to 100. +CREATE OR REPLACE VIEW public.imap_account_progress +WITH (security_invoker = true) +AS +WITH folder_progress AS ( + SELECT + f.account_id, + count(*) FILTER (WHERE f.live_window_target_count IS NOT NULL)::int AS live_window_known_folder_count, + count(*) FILTER (WHERE f.historical_target_count IS NOT NULL)::int AS historical_known_folder_count, + COALESCE(sum(LEAST(f.headers_synced_count, COALESCE(f.live_window_target_count, 0))), 0)::int + AS live_headers_synced_count, + COALESCE(sum(COALESCE(f.live_window_target_count, 0)), 0)::int AS live_headers_target_count, + COALESCE(sum(GREATEST(f.headers_synced_count - COALESCE(f.live_window_target_count, 0), 0)), 0)::int + AS historical_headers_synced_count, + COALESCE(sum(COALESCE(f.historical_target_count, 0)), 0)::int AS historical_headers_target_count, + COALESCE(sum(GREATEST(f.bodies_fetched_count - COALESCE(f.live_window_target_count, 0), 0)), 0)::int + AS historical_bodies_fetched_count, + COALESCE(sum(COALESCE(f.historical_target_count, 0)), 0)::int AS historical_bodies_target_count + FROM public.imap_folders f + WHERE f.tracked = true + AND f.status != 'MISSING' + GROUP BY f.account_id +), +active_body_folder_progress AS ( + SELECT + f.account_id, + count(*) FILTER (WHERE f.live_window_target_count IS NOT NULL)::int AS live_window_known_folder_count, + count(*) FILTER ( + WHERE f.sync_priority <= 10 + AND f.live_window_target_count IS NOT NULL + )::int AS priority_live_window_known_folder_count + FROM public.imap_folders f + WHERE f.tracked = true + AND f.missing_since IS NULL + AND f.status NOT IN ('MISSING', 'PENDING_VERIFICATION') + GROUP BY f.account_id +), +current_live_body_progress AS ( + SELECT + m.account_id, + count(*)::int AS live_bodies_target_count, + count(*) FILTER ( + WHERE m.body_fetched_at IS NOT NULL + AND b.message_id IS NOT NULL + AND NOT b.raw_truncated + )::int AS live_bodies_fetched_count, + count(*) FILTER ( + WHERE f.sync_priority <= 10 + )::int AS priority_bodies_target_count, + count(*) FILTER ( + WHERE f.sync_priority <= 10 + AND m.body_fetched_at IS NOT NULL + AND b.message_id IS NOT NULL + AND NOT b.raw_truncated + )::int AS priority_bodies_fetched_count + FROM public.imap_messages m + JOIN public.imap_folders f + ON f.account_id = m.account_id + AND f.path = m.folder_path + LEFT JOIN public.imap_message_bodies b + ON b.message_id = m.id + WHERE f.tracked = true + AND f.missing_since IS NULL + AND f.status NOT IN ('MISSING', 'PENDING_VERIFICATION') + AND m.deleted_in_provider = false + AND m.window_status = 'IN_WINDOW' + GROUP BY m.account_id +) +SELECT + a.id AS account_id, + COALESCE(p.live_headers_synced_count, 0) AS live_headers_synced_count, + COALESCE(p.live_headers_target_count, 0) AS live_headers_target_count, + CASE + WHEN COALESCE(p.live_headers_target_count, 0) > 0 + THEN LEAST(100, round((p.live_headers_synced_count::numeric / p.live_headers_target_count::numeric) * 100)::int) + WHEN COALESCE(p.live_window_known_folder_count, 0) > 0 THEN 100 + ELSE 0 + END AS live_headers_complete_pct, + COALESCE(b.priority_bodies_fetched_count, 0) AS priority_bodies_fetched_count, + COALESCE(b.priority_bodies_target_count, 0) AS priority_bodies_target_count, + CASE + WHEN COALESCE(b.priority_bodies_target_count, 0) > 0 + THEN CASE + WHEN b.priority_bodies_fetched_count >= b.priority_bodies_target_count THEN 100 + ELSE floor( + (b.priority_bodies_fetched_count::numeric / b.priority_bodies_target_count::numeric) * 100 + )::int + END + WHEN COALESCE(ab.priority_live_window_known_folder_count, 0) > 0 THEN 100 + ELSE 0 + END AS priority_bodies_complete_pct, + COALESCE(b.live_bodies_fetched_count, 0) AS live_bodies_fetched_count, + COALESCE(b.live_bodies_target_count, 0) AS live_bodies_target_count, + CASE + WHEN COALESCE(b.live_bodies_target_count, 0) > 0 + THEN CASE + WHEN b.live_bodies_fetched_count >= b.live_bodies_target_count THEN 100 + ELSE floor( + (b.live_bodies_fetched_count::numeric / b.live_bodies_target_count::numeric) * 100 + )::int + END + WHEN COALESCE(ab.live_window_known_folder_count, 0) > 0 THEN 100 + ELSE 0 + END AS live_bodies_complete_pct, + COALESCE(p.historical_headers_synced_count, 0) AS historical_headers_synced_count, + COALESCE(p.historical_headers_target_count, 0) AS historical_headers_target_count, + CASE + WHEN COALESCE(p.historical_headers_target_count, 0) > 0 + THEN LEAST(100, round((p.historical_headers_synced_count::numeric / p.historical_headers_target_count::numeric) * 100)::int) + WHEN COALESCE(p.historical_known_folder_count, 0) > 0 THEN 100 + ELSE 0 + END AS historical_headers_complete_pct, + COALESCE(p.historical_bodies_fetched_count, 0) AS historical_bodies_fetched_count, + COALESCE(p.historical_bodies_target_count, 0) AS historical_bodies_target_count, + CASE + WHEN COALESCE(p.historical_bodies_target_count, 0) > 0 + THEN LEAST(100, round((p.historical_bodies_fetched_count::numeric / p.historical_bodies_target_count::numeric) * 100)::int) + WHEN COALESCE(p.historical_known_folder_count, 0) > 0 THEN 100 + ELSE 0 + END AS historical_bodies_complete_pct, + NULL::timestamptz AS estimated_full_sync_at +FROM public.imap_accounts a +LEFT JOIN folder_progress p ON p.account_id = a.id +LEFT JOIN active_body_folder_progress ab ON ab.account_id = a.id +LEFT JOIN current_live_body_progress b ON b.account_id = a.id; diff --git a/apps/api/supabase/migrations/public/manifest.json b/apps/api/supabase/migrations/public/manifest.json index 8c21718..b0dd9f5 100644 --- a/apps/api/supabase/migrations/public/manifest.json +++ b/apps/api/supabase/migrations/public/manifest.json @@ -1,5 +1,5 @@ { - "schemaVersion": "0022_content_extract_body_store", + "schemaVersion": "0023_exact_body_completion_percentages", "migrations": [ { "id": "0001_imap_mirror", @@ -88,6 +88,10 @@ { "id": "0022_content_extract_body_store", "file": "0022_content_extract_body_store.sql" + }, + { + "id": "0023_exact_body_completion_percentages", + "file": "0023_exact_body_completion_percentages.sql" } ] } diff --git a/docs/agent/reliability-invariants.md b/docs/agent/reliability-invariants.md index 1f3ec75..b41a943 100644 --- a/docs/agent/reliability-invariants.md +++ b/docs/agent/reliability-invariants.md @@ -116,7 +116,7 @@ This is the agent-readable reliability contract distilled from `docs/spec-confor - `live_window_days` is immutable after account creation in v0.1; changing it requires a future window-status migration story. - `PATCH /accounts/:id/settings` may change `bodyFetchPolicy` only to `immediate`, `lazy`, or `priority_then_backfill`. The endpoint stays strict and rejects invalid, empty, or unknown input. - `immediate` includes every active live-window message in the automatic body lane. `lazy` disables automatic backlog fetch. `priority_then_backfill` includes only priority folders and does not promise later live-body coverage for current non-priority folders. -- `imap_account_progress` is a read model. Live and priority body fields come from current message/body rows; header and historical fields still use cumulative folder counters. Per-folder `live_bodies_fetched_count`, `live_bodies_target_count`, and `bodies_pct` use current nondeleted `IN_WINDOW` rows and store-completed, non-truncated body evidence for every returned folder. Because the folder list includes inactive folders, its targets need not sum to the active account target. Migration `0021_row_accurate_body_progress` adds `imap_messages_live_body_progress_idx` on `(account_id, folder_path, id)` for active `IN_WINDOW` rows; migration `0022_content_extract_body_store` additionally requires `body_fetched_at` so a pre-store evidence row cannot count as complete. Large existing mirrors must prebuild that exact index concurrently before applying the transactional migration. +- `imap_account_progress` is a read model. Live and priority body fields come from current message/body rows; header and historical fields still use cumulative folder counters. Per-folder `live_bodies_fetched_count`, `live_bodies_target_count`, and `bodies_pct` use current nondeleted `IN_WINDOW` rows and store-completed, non-truncated body evidence for every returned folder. Live, priority, and per-folder current body percentages can equal 100 only when the fetched count reaches the target; incomplete ratios are floored. Because the folder list includes inactive folders, its targets need not sum to the active account target. Migration `0021_row_accurate_body_progress` adds `imap_messages_live_body_progress_idx` on `(account_id, folder_path, id)` for active `IN_WINDOW` rows; migration `0022_content_extract_body_store` additionally requires `body_fetched_at` so a pre-store evidence row cannot count as complete. Large existing mirrors must prebuild that exact index concurrently before applying the transactional migration. - Account sync runs as three ordered lanes under one advisory lock: hot metadata/reconcile, capped live body backlog, then history. - History lane work must never run before hot sync or the live body lane, and it must stop when the cooperative lock budget is exhausted. - Historical backfill uses the folder `backfill_*` state and `last_archive_refresh_at`; it snapshots older-than-window UIDs and walks them newest-first in resumable batches. diff --git a/docs/architecture/decisions/0027-mutable-body-policy-row-accurate-coverage.md b/docs/architecture/decisions/0027-mutable-body-policy-row-accurate-coverage.md index 49dc628..7317596 100644 --- a/docs/architecture/decisions/0027-mutable-body-policy-row-accurate-coverage.md +++ b/docs/architecture/decisions/0027-mutable-body-policy-row-accurate-coverage.md @@ -45,9 +45,11 @@ messages are still incomplete. - `GET /accounts/:id` exposes row-current `live_bodies_fetched_count` and `live_bodies_target_count` for each folder. Per-folder `bodies_pct` uses those fields instead of the cumulative body - counter. Folder rows include untracked, missing, and pending folders, while - the account roll-up excludes them. Their targets therefore need not sum to - the account target. + counter. Live and priority account percentages and per-folder current body + percentages equal 100 only when the fetched count reaches the target; + incomplete ratios are floored. Folder rows include untracked, missing, and + pending folders, while the account roll-up excludes them. Their targets + therefore need not sum to the account target. - Incremental folder counters remain cumulative telemetry and continue to support header and historical progress. They are not the source of truth for current live or priority body completeness. diff --git a/docs/architecture/reliability-and-three-lanes.md b/docs/architecture/reliability-and-three-lanes.md index d1ada43..fbc54e5 100644 --- a/docs/architecture/reliability-and-three-lanes.md +++ b/docs/architecture/reliability-and-three-lanes.md @@ -85,8 +85,10 @@ view requires no refresh management. Both per-folder and per-account values are exposed via `GET /accounts/:id`. Each folder adds row-current `live_bodies_fetched_count` and `live_bodies_target_count`; `bodies_pct` uses those fields instead of the -cumulative body counter. Folder rows also cover inactive folders, so their -targets do not always sum to the active account target. +cumulative body counter. Live and priority account percentages and per-folder +current body percentages equal 100 only when the fetched count reaches the +target; incomplete ratios are floored. Folder rows also cover inactive folders, +so their targets do not always sum to the active account target. API surface: diff --git a/docs/schema.md b/docs/schema.md index f4ba888..555c67a 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -72,7 +72,7 @@ Historical backfill uses the folder-level `backfill_in_progress`, `backfill_targ `0021_row_accurate_body_progress` replaces `imap_account_progress`. Live and priority body targets now come from active `IN_WINDOW` `imap_messages` joined to tracked `imap_folders` whose `missing_since` is NULL and whose status is neither `MISSING` nor `PENDING_VERIFICATION`; provider-deleted rows are excluded. Migration `0022` tightens completion for the two-stage store: a fetched body counts only when `body_fetched_at` is set, an `imap_message_bodies` row exists, and `raw_truncated = false`. Priority coverage applies the existing `sync_priority <= 10` cutoff. Live-header and historical percentages continue to use the cumulative folder counters. -`GET /accounts/:id` exposes the account percentages plus a nullable `estimated_full_sync_at`; the estimate remains null until a durable rate model exists. Each folder row also exposes row-current `live_bodies_fetched_count` and `live_bodies_target_count`. Its `bodies_pct` uses those fields instead of the cumulative `bodies_fetched_count`. Folder rows cover every returned folder, including untracked, missing, and pending folders. The account roll-up includes only active eligible folders, so folder targets do not always sum to the account target. +`GET /accounts/:id` exposes the account percentages plus a nullable `estimated_full_sync_at`; the estimate remains null until a durable rate model exists. Each folder row also exposes row-current `live_bodies_fetched_count` and `live_bodies_target_count`. Its `bodies_pct` uses those fields instead of the cumulative `bodies_fetched_count`. Migration `0023` floors incomplete live, priority, and per-folder current body ratios, so 100 means the fetched count reached the target. Folder rows cover every returned folder, including untracked, missing, and pending folders. The account roll-up includes only active eligible folders, so folder targets do not always sum to the account target. The same migration adds the partial `imap_messages_live_body_progress_idx` on `(account_id, folder_path, id)` for active `IN_WINDOW` rows. Large existing diff --git a/docs/spec-conformance.md b/docs/spec-conformance.md index 4bccf23..87c9a40 100644 --- a/docs/spec-conformance.md +++ b/docs/spec-conformance.md @@ -118,7 +118,9 @@ only after the cause is corrected; a cap-limited message requires a higher `live_bodies_target_count`, and `bodies_pct` use the same completeness test within each returned folder. They also report untracked, missing, and pending folders, which the active account roll-up excludes. Folder targets therefore do -not always sum to the account target. Migration +not always sum to the account target. Live, priority, and per-folder current +body percentages report 100 only when the fetched count reaches the target; +incomplete ratios are floored. Migration `0021_row_accurate_body_progress` adds the partial `imap_messages_live_body_progress_idx`; large existing mirrors must prebuild that exact index concurrently before applying the transactional migration.