Per-kind progressive readiness during initial sync - #1189
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
During the initial informer sync — minutes on large clusters — the whole UI sat behind a splash screen and every read 503'd until the critical set (including pods, typically the slowest LIST) finished. Resource views now open per kind as each informer completes: - k8score: OnInformersStarted hands out the cache mid-Phase-1; KindReadinessFor classifies kinds ready/pending/failed/unavailable from live informer state; GetSyncSnapshot reports per-kind progress without lister walks. RADAR_DEBUG_SYNC_DELAY dev seam simulates slow syncs. - internal/k8s: generation-guarded syncingCache alongside the untouched resourceCache singleton — a context switch mid-construction invalidates every publish/promotion from the stale build instead of resurfacing the old cluster's cache. - server: resource list/get handlers gate per kind (503 kind_sync_pending / kind_sync_failed) and serve from whichever cache exists; this also closes a shipped false-empty where promoted/isEnabled-only kinds could render a partial store as a complete list post-connect. Dynamic/CRD reads keep the connected gate. Connection payload carries a per-kind sync snapshot while connecting. - web: progressive app shell during sync — resource views work per kind (pending kinds stay in loading, never an empty list), other views show a per-kind progress panel with links to ready kinds, header carries global progress. Large-list guard holds while counts are unavailable; warm shared query caches are dropped before the shell can render another cluster's data. Closes #1149
- A manual connection retry cleared everything except the previous attempt's per-kind sync snapshot; drop it so the shell can't mislabel readiness against a fresh attempt. - Events shares the deferred tracking map but syncs in the background with no deadline — the deferred-timeout flag must not classify it as terminally failed while its LIST is legitimately still running. - initialSyncComplete is now set only after successful promotion, so an orphaned construction (context switch mid-build) can't leave the flag true with no cache installed.
e6f384a to
eedcbcc
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit eedcbcc. Configure here.
| return failureCount < 3 | ||
| }, | ||
| retryDelay: (failureCount: number, error: Error) => | ||
| isKindSyncPending(error) ? 2000 : Math.min(1000 * 2 ** failureCount, 30000), |
There was a problem hiding this comment.
Pending kinds flash empty lists
High Severity
selectedKindQueryResult maps loading from TanStack Query's isLoading, but v5 sets isLoading false during retry delays while isPending stays true. Infinite kind_sync_pending retries therefore drop the loader for ~2s gaps and the table falls through to No <kind> found, including long-running Events sync. That breaks the guarantee that an unsynced kind never renders as an empty list.
Reviewed by Cursor Bugbot for commit eedcbcc. Configure here.
| } | ||
| } | ||
| return KindPending | ||
| } |
There was a problem hiding this comment.
Readiness ahead of deferred listers
Medium Severity
KindReadinessFor returns KindReady from live HasSynced, but deferred listers such as ConfigMaps/Secrets still gate on isReady / deferredSynced, which lags the Phase-2 bookkeeping loop. gateResourceRead can allow a request that then hits a nil lister and a plain 503 without kind_sync_pending, so the frontend stops infinite-polling and can briefly show an empty or error state.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit eedcbcc. Configure here.
|
@nadaverell I did a rebase of this branch, please consider if this branch should be merged |
2 participants
Fixes #1149.
What
On large clusters the initial informer sync takes minutes, and until now the entire UI sat behind a splash screen with every read returning 503 until the critical set — including pods, typically the slowest LIST — finished. Radar now becomes usable per kind, as each informer completes: navigate to Deployments while Pods is still listing and the full Deployments view works; the Pods view shows a loading state and appears the moment its data is complete.
Honest bound: the pods view itself can't appear faster than the pods LIST — the win is every other view, plus a legible app with visible per-kind progress instead of an opaque splash.
How it works
Readiness, not reordering. Informers already start in parallel; there is no queue to jump. The change is exposing per-kind sync state and gating reads on it:
k8score:OnInformersStartedhands out the cache mid-Phase-1;KindReadinessForclassifies a kindready / pending / failed / unavailablefrom live informer state;GetSyncSnapshotreports per-kind progress cheaply (no lister walks — the connection endpoint polls sub-second during sync).internal/k8s: a generation-guardedsyncingCachehandle alongside the untouchedresourceCachesingleton (its non-nil ⇒ synced invariant holds everywhere else). A context switch mid-construction invalidates every publish/promotion from the stale build — this also closes a pre-existing race where a construction finishing after a switch could republish the old cluster's cache.server: the resource list/get handlers gate per kind —503 {error_code: kind_sync_pending}while an informer syncs,kind_sync_failedwhen it never will — and serve from whichever cache exists. This also fixes a shipped false-empty bug: post-connect, promoted criticals andisEnabled-only listers (ReplicaSets, HPAs, ServiceAccounts…) could render a partial store as a complete list. Dynamic/CRD reads keep the connected gate (no dynamic cache exists mid-sync). Single-GET serves without relationship enrichment while the topology cache doesn't exist. A disconnected cluster still 503s everywhere — cache handles stand in for connectedness only duringconnecting.web: the splash is replaced by the app shell as soon as informers start. Resource views work per kind — a pending kind stays in its loading state (an unsynced kind never renders as an empty list) and populates automatically. Views needing the full dataset (Home, Topology, Timeline, …) show a per-kind progress panel with links to ready kinds. The header carries "Loading cluster data — x of y ready". The large-list guard holds while counts are unavailable, and warm shared query caches (embedded multi-cluster mounts) are dropped before the shell can show another cluster's data.RADAR_DEBUG_SYNC_DELAY=pods=120s(dev seam) simulates a slow kind for testing/demoing the window.Testing
go test ./...+ k8score module +tsc+ frontend build green.Deliberate scope cuts
/resource-countsisn't progressive yet, so guarded kinds (Pod/Event/ReplicaSet/EndpointSlice) hold their large-list guard until connected — the safe direction; progressive counts is a natural follow-up.Note
Medium Risk
Changes cluster connection gating, cache singleton lifecycle on context switches, and what resource APIs return during sync—high user impact but guarded by generation checks and explicit readiness errors instead of silent wrong data.
Overview
Large clusters used to block the whole UI on a connecting splash until critical informers (often pods) finished listing. This PR exposes per-kind sync state so resource list/get handlers and the web shell can open one kind at a time while others are still pending.
Backend:
k8scoreaddsOnInformersStarted,KindReadinessFor(ready/pending/failed/unavailable),GetSyncSnapshot, and optionalDebugSyncDelays.internal/k8spublishes a generation-guarded mid-syncsyncingCachebefore Phase-1 completes;ResetResourceCachebumpscacheGenerationso stale constructions cannot republish an old cluster’s cache after a context switch. Resource handlers usegateResourceRead/requireConnectedOrSyncingand return503withkind_sync_pendingorkind_sync_failedinstead of serving partial stores as empty lists. Connection status includessyncStatuswhileconnecting.Frontend: When
syncStatusis present, the app shell replaces the splash; resource views load per kind (pending kinds keep polling); Home/Topology/etc. showSyncProgressPaneluntil connected. React Query retries onkind_sync_pending; shared query caches are cleared when progressive sync starts to avoid cross-cluster stale data.Reviewed by Cursor Bugbot for commit eedcbcc. Bugbot is set up for automated code reviews on this repo. Configure here.