Skip to content

fix(storage): stop a full localStorage from hanging the boot splash - #78

Merged
Thunderhawkk merged 6 commits into
Harborofficial:beta-branchfrom
kalashnikxvxiii:fix/localstorage-quota-boot
Nov 6, 2026
Merged

Thunderhawkk merged 6 commits into
Harborofficial:beta-branchfrom
kalashnikxvxiii:fix/localstorage-quota-boot

Conversation

@kalashnikxvxiii

Copy link
Copy Markdown
Contributor

Summary

A localStorage that has reached the WebView quota makes Harbor hang forever on the boot splash. This fixes the crash path and makes the existing storage recovery actually reachable.

  • main.tsx — run proactiveStorageCleanup() at the top of mount(), before anything writes.
  • storage-recovery.ts — trigger the full prune when the store rejects a write, not only when the prunable total passes a size threshold; add the largest refetchable caches to the prunable lists.
  • onboarding.tsx — persist through setItemWithRecovery, like the other callers, instead of a bare setItem during commit.
  • webhook-loop-mount.tsx — drop the now-redundant cleanup effect.
  • New regression test for the quota path.

Why

On a full store every setItem throws. The write in OnboardingProvider's effect was unguarded, and that provider wraps the whole app, so the throw landed during commit and tore the tree down before heroReady could fire onReady. revealRoot() never ran, so #harbor-boot stayed up over a visibility:hidden #root: the app looks frozen on the splash with idle CPU, nothing on stderr, and no error surface, because the error screen React does render is hidden underneath.

The recovery for this already existed but could never run:

  1. proactiveStorageCleanup() was called from a useEffect in WebhookLoopMount, a component that only mounts after the render that dies.
  2. Even when reached, it only called freeStorageSpace() above PROACTIVE_TOTAL_THRESHOLD (3.5 MB of prunable data). A store filled mostly with keys we never prune stays under that while every write already throws, so it no-opped precisely when it was needed. Asking the store whether it still accepts a write is the condition that actually matters.
  3. The biggest consumers were not prunable at all. Added: harbor.ebook.openlibrary.v2.*, harbor.ebook.nyt.v1, harbor.sports.board.v3:*, harbor.mal.hero.v1, harbor.anime.art.v2, harbor.anime.hero.hosted.v5. All are refetchable derived caches. harbor.settings, harbor.profiles.v1, harbor.installed-addons.* and harbor.addonOrderBackups are deliberately left alone.

Measured on a profile that reproduced the hang: cleanup takes localStorage from 3,159,696 to 1,093,786 chars and the app reaches Home.

Verification

vp check src/main.tsx src/lib/storage-recovery.ts src/lib/onboarding.tsx \
         src/components/webhook-loop-mount.tsx tests/storage-quota-boot.test.ts   # pass
tsc --noEmit -p tsconfig.json                                                     # clean
node --experimental-loader ./scripts/node-test-loader.mjs --test \
     tests/storage-quota-boot.test.ts                                             # 2/2 pass
pnpm test                                                                         # 252 failures, identical set with and without this change (pre-existing baseline on beta-branch)
pnpm exec tauri build --no-bundle                                                 # ok

The new test fails on beta-branch without the storage-recovery.ts change and passes with it.

Manual: with the real profile that reproduced the bug, the pre-fix build sits on the splash indefinitely (confirmed past 80 s, across both the system WebKitGTK 2.54.2 and a private 2.52.6); the post-fix build reaches Home in a few seconds. Diagnosis was done by tracing boot stages and Storage.prototype.setItem failures out of the packaged release build.

No Rust changes, so no Cargo checks were run.

Platform Impact

Verified on Linux/Wayland (Hyprland, WebKitGTK 2.54.2). The cause is the WebView localStorage quota, which is not Linux-specific, so Windows and macOS are affected the same way and the fix is platform-neutral. Not retested on those, nor on TV-style input; no input, playback or navigation behavior is touched.

Notes

vp check --fix also rewrote ...(parsed.nudges ?? {}) to ...parsed.nudges in onboarding.tsx — same semantics, and the vp staged pre-commit hook reapplies it on any commit touching that file. Happy to drop that hunk if you prefer.

Checklist

  • This pull request is focused and contains no unrelated refactors.
  • I ran vp check for the changed files.
  • I ran vp run typecheck after TypeScript changes.
  • I ran the relevant Cargo checks after Rust changes. (no Rust changes)
  • I tested affected platforms when the change is platform-specific. (Linux; see Platform Impact)
  • I preserved playback, navigation, and configured hotkey behavior where applicable.
  • I added or updated tests for behavior changes.
  • I removed secrets, tokens, private URLs, and personal data from logs and screenshots.

🤖 Generated with Claude Code

On a localStorage at the WebView quota every setItem throws. The write in
OnboardingProvider's effect was unguarded, so it threw during commit and took
the tree down before heroReady could fire onReady: revealRoot() never ran and
#harbor-boot stayed up forever over a hidden #root. The app looked frozen on
the splash with no error, idle CPU and nothing on stderr.

The recovery for this already existed but could never run:

- proactiveStorageCleanup() was called from a useEffect in WebhookLoopMount, a
  component that only mounts after the render that dies. Call it at the top of
  mount() instead, before anything writes.
- It only reached freeStorageSpace() when the prunable total passed 3.5 MB.
  A store filled mostly with keys we never prune stays under that while every
  write already throws, so it no-opped exactly when it was needed. Ask the
  store whether it still accepts a write instead.
- The largest caches were not prunable at all. Add the refetchable ones:
  ebook OpenLibrary v2, ebook NYT, sports boards, MAL hero, anime art and
  hosted anime hero. Settings, profiles and installed add-ons stay untouched.

OnboardingProvider now persists through setItemWithRecovery like the other
callers, so a full store degrades instead of bricking the boot.

Measured on a profile that reproduced the hang: cleanup takes localStorage
from 3,159,696 to 1,093,786 chars and the app reaches Home.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Thunderhawkk

Copy link
Copy Markdown
Contributor

Thanks for investigating and fixing the boot splash issue. Moving storage recovery earlier in startup and handling the onboarding write through setItemWithRecovery makes sense. However, I found one issue that should be addressed before merging.

1. [P1] Storage recovery can delete persistent user data

The new storageRejectsWrites() check triggers freeStorageSpace() when localStorage rejects writes. However, the existing prunable lists include keys that contain user data rather than disposable caches:

  • harbor.playback-history.v1.* stores playback sources, positions, and continuation information.
  • harbor.lastseason.v1 stores users' last selected seasons.
  • harbor.parental contains legacy parental settings that may still be needed during profile migration.

Since freeStorageSpace() removes all prunable entries, a full localStorage can cause these records to be permanently deleted during startup.

Suggested fix: Separate disposable caches from persistent user data and exclude these keys from automatic eviction. It would also be better to stop pruning once enough space has been recovered rather than deleting every cache unnecessarily.

Please add a regression test confirming that playback history and legacy settings survive storage recovery.

Once this is addressed, the PR should be in good shape for merging.

storageRejectsWrites() is what finally makes freeStorageSpace() run at
boot, which turned a dormant bug into data loss on the first launch with
a full store: the lists held playback history, the last season picked,
live-TV watch counts and the legacy parental key, which still carries
hiddenTabs and pinHash, so recovery could silently unlock parental
controls. Those are out now, and the boot path frees only what it takes
to make the store writable again instead of dropping every cache.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kalashnikxvxiii

Copy link
Copy Markdown
Contributor Author

You're right, thanks for catching it.

Those keys were already in the prunable lists before this PR - what I added was storageRejectsWrites(), which is what finally makes freeStorageSpace() run at boot. So I took a dormant bug and made it fire on first launch. My store was full, I just wanted the app to open so I could watch Edgerunners 2, and once it reached Home I didn't look at what recovery had eaten on the way. Should have read the lists instead of trusting them.

All three are out now, plus a fourth you didn't flag: harbor.iptv.stats.v1 isn't a network cache, it's the user's per-channel watch counts behind Live TV's "Most watched". Nothing regenerates it.

I checked every remaining key against backup.ts and left some prunable on purpose, because its patterns match by prefix - harbor.iptv. also covers hydration.v2 and epgmap.v1, harbor.discover. covers the row caches, harbor.manga. covers manga.cache.* and manga.art.. Those cost a refetch, not a record. The criterion is now a comment above the list so the next addition doesn't repeat this.

On stopping early: proactiveStorageCleanup prunes largest-first and stops at the first write that succeeds. I left freeStorageSpace() itself alone - its four callers reach for it after a big payload has already failed and want all the space they can get, so bailing out on a tiny probe write would make them worse. Say the word if you'd rather have it uniform.

The regression test now seeds the user's records as the largest entries, so a wrong list hits them first, and asserts all four survive; a second test covers the early stop. Both fail if I put the respective bug back, I checked. vp check and tsc --noEmit clean, pnpm test failures identical to the branch baseline.

@Thunderhawkk

Copy link
Copy Markdown
Contributor

Thanks for the follow-up. The previous findings have been addressed, and I agree with keeping freeStorageSpace() unchanged while using the early-stop behavior specifically for boot recovery.

However, I found one more issue with the remaining prunable keys.

1. [P1] Additional user-generated data is still included in storage cleanup

In storage-recovery.ts, two keys remain in PRUNABLE_EXACT despite containing persistent user data rather than disposable caches:

  • harbor.iptv.epgmap.v1: Stores manual EPG channel mappings configured through the Live TV "Match EPG channel" dialog. These mappings are explicitly chosen by the user and cannot be regenerated from the network.
  • harbor.discover.v1: Stores Discover's recorded user interactions and learned affinity profile, including preferences accumulated through browsing, watching, and voting. Deleting it resets that personalization, which cannot simply be refetched.

There's also harbor.surprise.recent.v1, which tracks recently suggested titles to avoid repeatedly recommending the same content. While less critical, it is worth considering whether this should remain prunable.

Suggested fix: Remove the manual EPG mappings and Discover preference records from automatic eviction, and extend the regression tests to verify that these records survive storage recovery.

Checking backup.ts is useful, but its categories contain both persistent user data and recoverable caches, so each key still needs to be evaluated based on how it is populated and whether it can actually be reconstructed.

The rest of the changes look good, including the additional cache entries and early-stop behavior. Once these remaining records are protected, the PR should be ready for merging.

…he live caches

Manual EPG matches come from the Live TV match dialog and Discover's
affinity is built from up to 500 recorded interactions, so neither comes
back from the network. The build rating goes too, and so do the anti-repeat
lists for surprise-me and top picks: a few KB each, nothing to gain by
risking them.

The other half of the same list was pinning cache versions, so seven live
caches had quietly stopped being prunable - jikancatalog2, armkitsucache.v3
and the rest of the anime mapping set, which are exactly the ones that grow.
They are prefixes now, matched whatever the owner bumps them to.

Ordering in the existing tests could leave a wrongly listed key untouched,
so the list has its own guard over the full sweep.
@kalashnikxvxiii

Copy link
Copy Markdown
Contributor Author

Both confirmed and out, along with harbor.build.rating.v1 - the rating the user gives a build from the settings panel, which you didn't flag. harbor.surprise.recent.v1 is gone too, though not as a record: 80 ids with a 14-day TTL, about 3KB. Same for the top-picks ring and visit counter, same mechanism. Anything that small is risk with no payoff.

Two things worth more than the key list, though.

My assertions for those keys passed with the bug still in place. Early-stop frees the largest cache and never reaches the rest, so ordering was deciding the outcome, not the list. There's now an order-independent guard over the full sweep, and I checked it fails for each re-listed key individually.

And going back through every remaining key turned up the opposite problem. Seven live caches weren't prunable at all, because the list pinned the version before the current one:

harbor.jikancatalog → harbor.jikancatalog2
harbor.armkitsucache → harbor.armkitsucache.v3
harbor.anidbtvdbcache → harbor.anidbtvdbcache.v2
harbor.armsrcmalcache → harbor.armsrcmalcache.v2
harbor.extkitsucache → harbor.extkitsucache.v2
harbor.anime_awards.metas.v2 → harbor.anime_awards.metas.v6
harbor.anime.toppicks.cache.v1 → harbor.anime.toppicks.cache.v2

Plus harbor.graph.wikidata.v2, never listed. These are the anime mapping and Jikan catalog caches, which is to say the ones that actually grow. Had they been the top consumers on my machine rather than the hero art and sports boards, recovery would have no-opped again and I'd still be hunting the hang. They're prefixes now, with a test that seeds an unshipped .v99 so the next bump doesn't need this list touched.

One path neither of us looked at: setItemWithRecovery also evicts harbor.snap.*, the exit frames behind Continue Watching cards, which only come back by rewatching. I left it - they're cosmetic, already capped at 80 entries and 2.5MB with a retention setting the user controls, and boot recovery doesn't touch them. Flagging it because it's the largest consumer by design, not as something to change here.

Three keys in the list have no writer left anywhere (harbor.scroll.v1, harbor.discover.events.v1, harbor.manga.popular.v1). Pruning them is a no-op so I left them; say if you'd rather they go.

@comet776 comet776 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head 4dcb55d0 against beta-branch. The core fix is sound: cleanup now runs before createRoot, onboarding writes through setItemWithRecovery, and the new quota tests exercise the real failure shape. I ran the new test file locally: 5/5 pass.

P1 — harbor.streamBadges.v1 is still in the prunable list, and it is user data

src/lib/storage-recovery.ts (line 29) keeps harbor.streamBadges.v1 in PRUNABLE_EXACT. That key is written by src/lib/stream-badges.ts and holds:

  • overrides with user-uploaded badge images — data URLs up to 260 KB (kind-editor-modal.tsx);
  • custom badge rules the user created or imported — regex, colors, names, enabled state.

None of it is refetchable, and the import source URL is not stored with the rules. The key is not a cache.

This is the same class as playback-history, iptv.stats, and discover.v1: it was already in the list, but this PR is what makes it fire at boot. It is especially likely to be hit because freeUntilWritable() prunes largest-first and stops at the first successful probe. A 250 KB badge image can be the largest prunable entry, so it is deleted first, and the loop stops immediately.

I reproduced it against the PR's own quota stub: with streamBadges at 250 KB, mal.hero at 100 KB, and settings at 500 KB, cleanup logs pruned "harbor.streamBadges.v1" and leaves mal.hero intact. The total > 3.5 MB branch (freeStorageSpace) deletes it too. The full-sweep test does not seed it, so it passes.

Suggested fix:

  1. Remove harbor.streamBadges.v1 from PRUNABLE_EXACT.
  2. Add it at ~250 KB to the protected seeds in tests/storage-quota-boot.test.ts and assert it survives both cleanup paths.
  3. Re-audit the rest against "user-created config", not just history and settings.

P2 — the probe only guarantees room for one character

storageRejectsWrites() writes "1". freeUntilWritable() stops when that succeeds, not when the next boot write fits. A store full of many small prunable entries plus large user data can free only a few hundred bytes; the next settings/profile/hero-cache write can still throw. Onboarding is now guarded, so the original crash is fixed, but the comment's "free just enough to get going" is not guaranteed. Probe with a realistic payload (16–64 KB) or free to a fixed headroom, and add a test where the largest prunable entry is smaller than the follow-up write.

P2 — full cleanup now runs in every Tauri window

src/main.tsx:228 runs before the isPip / isModal / isCaptions / isDjDeck / isVideoPip / isHdrOverlay dispatch. Every auxiliary window loads index.html?harbor-* (pip.rs:93, modal_overlay.rs:75, captions.rs:79, dj_deck.rs:17, pip_window.rs:170, hdr_overlay.rs:65) and shares the same localStorage. Opening a modal or PIP can therefore call freeStorageSpace() and wipe more than 3.5 MB of caches the main window is using, causing mid-session refetches.

Previously this only ran in WebhookLoopMount inside the main App. Gate the proactive prune to the main window, or split "recover if unwritable" (safe everywhere) from "proactive prune" (main window only).

P2 — new entries repeat the version-pinning bug the PR fixes

The new comment says "Prefixes, not exact keys, for every cache whose owner version-stamps it", but the PR adds exact harbor.ebook.nyt.v1, harbor.mal.hero.v1, harbor.anime.art.v2, and harbor.anime.hero.hosted.v5. The version-bump test only covers the seven previously converted prefixes. On the next bump these silently stop being prunable.

Several prefixes also still bake in a version: harbor.sports.board.v3:, harbor.ebook.openlibrary.v1./v2., harbor.manga.cache.v1./v2.. Use version-agnostic prefixes and seed an unshipped version for each, the way the test already does with armkitsucache.v99.

P3 — cleanup is unguarded on the boot path

proactiveStorageCleanup() is now called before createRoot with no try/catch. A non-quota storage error (blocked or corrupt storage) would reject mount(); the catch only logs and never reveals the splash. Wrapping the call is cheap insurance.

Verification

  • New tests/storage-quota-boot.test.ts: 5/5 pass locally and in the CI log.
  • Typecheck passed in CI, and the new tests pass in the Frontend job.
  • The Frontend job is still red on unrelated failures, e.g. search-scroll-return.test.mjs (useInstallFeatures mock) and stream-blocker.test.mjs missing src-tauri/resources/harbor-stream-blocker/README.md. Native passed. Worth confirming the beta-branch baseline, but I do not see a diff-related failure.
  • The parsed.nudges formatter hunk is semantically equivalent; keep or drop.
  • The earlier P1s are addressed: playback-history, iptv.stats, iptv.epgmap, discover.v1, lastseason, parental, and build.rating are out of the list.

Verdict: request changes because of the streamBadges data loss. Once that key is protected and the two P2s are addressed, this is a good fix.

…ws out

harbor.streamBadges.v1 holds badge images the user uploaded as data URLs,
up to 260KB each, plus the rules they wrote - none of it refetchable, and
largest-first pruning reaches for it early. Out of the list.

The write probe asked for one character, so recovery could stop with a few
hundred bytes free and the next settings or profile write still threw. It
asks for 32KB now.

Cleanup ran before the window dispatch in main.tsx, so opening a PIP, modal,
captions, DJ or overlay window could evict caches the main window was using.
Gated to the main window.

PRUNABLE_EXACT is gone: one prefix list, nothing with a version stamp, since
the exact/prefix split is what let the pinning happen. The boot call is
wrapped too - a blocked or corrupt store rejecting mount() is the failure
this is meant to prevent.
@kalashnikxvxiii

Copy link
Copy Markdown
Contributor Author

Thorough review, thanks - the window one especially, I'd missed it completely and it's my change that introduced it. Oopsie!

streamBadges confirmed and out, protected on both paths now. Your third suggestion was very helpful: I'd been auditing for "history and settings" when the category is user-created config. Re-checked the rest with that lens - harbor.shows.hero.pool.v2, harbor.heroPool.v1 and harbor.libraryNameRepair.v1. are all TTL caches, so streamBadges was the last one.

The probe is 32KB now rather than one character, sized like a real settings or profile write.

The auxiliary windows are gated behind an isAuxWindow check, so only the main window prunes. Splitting "recover if unwritable" from "proactive prune" would be the more precise fix, but the aux windows write almost nothing, and the main window runs the cleanup moments earlier anyway - happy to split it if you'd rather.

On the version pinning: fair, and a bit embarrassing. I wrote "prefixes, not exact keys" and added four exact version-stamped keys in the same commit. PRUNABLE_EXACT is gone entirely - it's one prefix list now, nothing with a version stamp in it, because the exact/prefix split is what let the pinning happen in the first place. The bump test seeds unshipped versions for the converted prefixes too.

Wrapped the boot call in try/catch. That one stung a little, since a blocked or corrupt store rejecting mount() is exactly the failure this PR is supposed to stop.

On CI: those are the branch baseline. I compared the failing set with and without the diff stashed and it's identical, 251 lines either way. Keeping the parsed.nudges hunk - the pre-commit hook reapplies it.

Each new test fails if I put the respective bug back, which I checked one at a time: streamBadges re-listed drops both the boot and sweep tests, PROBE_CHARS = 1 drops the headroom test, and re-pinning harbor.mal.hero.v1 drops the bump test.

Also built it and checked the window gate for real rather than by construction: the main window reports aux=false cleanup=true, a live harbor-dj window reports aux=true cleanup=false, and the clean build boots to Home.

Preserve the one-time library-name repair marker and the OMDb daily request counter during quota recovery. Keep only rebuildable OMDb rating and miss caches prunable, and test both boot recovery and full-sweep eviction.
Exclude webhook delivery baselines and historical addon-star snapshots from automatic eviction. Add boot quota and full-sweep regressions guarding both records.
@Thunderhawkk
Thunderhawkk merged commit 5ec96db into Harborofficial:beta-branch Nov 6, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants