Fix SQLite contention during import-list sync

docs/superpowers/plans/2026-08-17-import-list-worker-serialization.md

@@ -0,0 +1,97 @@
+# Import-list Worker Serialization Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Ensure one `ImportListSync` command cannot overlap enabled import-list pipelines.
+
+**Architecture:** `FetchAndParseImportListService.Fetch()` will execute each accepted list directly in the existing loop rather than submitting all lists to a long-running `TaskFactory`. Existing result aggregation, failure handling, TMDb mapping, list-movie synchronization, and status updates remain unchanged.
+
+**Tech Stack:** C#, .NET 8, NUnit, FluentAssertions, Moq.
+
+---
+
+### Task 1: Prove that a later list does not start while the first list is blocked
+
+**Files:**
+- Modify: `src/NzbDrone.Core.Test/ImportListTests/FetchAndParseImportListServiceFixture.cs`
+- Test: `src/NzbDrone.Core.Test/ImportListTests/FetchAndParseImportListServiceFixture.cs`
+
+- [ ] **Step 1: Write the failing regression test**
+
+```csharp
+[Test]
+public void should_not_start_the_next_list_until_the_current_list_finishes()
+{
+    using var firstListStarted = new ManualResetEventSlim();
+    using var releaseFirstList = new ManualResetEventSlim();
+    var first = CreateListResult(1, true, true, new ImportListFetchResult());
+    var second = CreateListResult(2, true, true, new ImportListFetchResult());
+
+    first.Setup(x => x.Fetch()).Returns(() =>
+    {
+        firstListStarted.Set();
+        releaseFirstList.Wait();
+        return new ImportListFetchResult();
+    });
+
+    var fetch = Task.Run(() => Subject.Fetch());
+    firstListStarted.Wait(TimeSpan.FromSeconds(1)).Should().BeTrue();
+    second.Verify(x => x.Fetch(), Times.Never());
+
+    releaseFirstList.Set();
+    fetch.Wait(TimeSpan.FromSeconds(1)).Should().BeTrue();
+    second.Verify(x => x.Fetch(), Times.Once());
+}
+```
+
+- [ ] **Step 2: Verify the test is red**
+
+Run:
+
+```bash
+dotnet test src/NzbDrone.Core.Test/Radarr.Core.Test.csproj /p:RunAnalyzers=false --filter 'FullyQualifiedName~FetchAndParseImportListServiceFixture.should_not_start_the_next_list_until_the_current_list_finishes'
+```
+
+Expected: FAIL because the current `TaskFactory.StartNew` immediately invokes the second list's `Fetch()`.
+
+### Task 2: Make all-list processing sequential
+
+**Files:**
+- Modify: `src/NzbDrone.Core/ImportLists/FetchAndParseImportListService.cs`
+- Test: `src/NzbDrone.Core.Test/ImportListTests/FetchAndParseImportListServiceFixture.cs`
+
+- [ ] **Step 1: Replace worker creation with direct execution**
+
+Remove `System.Threading.Tasks`, the task list, and `TaskFactory`. Execute the existing try/catch and `lock (result)` body directly after the blocked-list checks, then remove `Task.WaitAll(...)`.
+
+- [ ] **Step 2: Verify the focused fixture is green**
+
+Run:
+
+```bash
+dotnet test src/NzbDrone.Core.Test/Radarr.Core.Test.csproj /p:RunAnalyzers=false --filter 'FullyQualifiedName~FetchAndParseImportListServiceFixture'
+```
+
+Expected: all fixture tests pass, including the new gating regression.
+
+### Task 3: Verify, publish, and stage production
+
+**Files:**
+- Modify: `docs/superpowers/specs/2026-08-17-import-list-worker-serialization-design.md`
+- Modify: `docs/superpowers/plans/2026-08-17-import-list-worker-serialization.md`
+
+- [ ] **Step 1: Run the focused import-list and queue persistence suites**
+
+```bash
+dotnet test src/NzbDrone.Core.Test/Radarr.Core.Test.csproj /p:RunAnalyzers=false --filter 'FullyQualifiedName~FetchAndParseImportListServiceFixture|FullyQualifiedName~CommandQueueManagerFixture|FullyQualifiedName~CommandQueueFixture'
+```
+
+Expected: all selected tests pass.
+
+- [ ] **Step 2: Build the version-matched Core assembly and update PR #5**
+
+Commit the test, implementation, design, and plan; push `fix/import-list-sync-db-lock` to `johoja12/Radarr`. Confirm PR #5 contains both `7df4bfa` and the new serialization commit.
+
+- [ ] **Step 3: Deploy only Radarr4K and validate staged scheduler cycles**
+
+Create a fresh verified SQLite backup, build the overlay image from the pinned 6.5.1.2032 production image, recreate only `radarr4k`, then check `/ping`, `PRAGMA quick_check`, no active commands, and no fresh locked/busy logs. Enable IDs 4, 13, and 12 one at a time, observing two scheduled cycles for each before proceeding. Disable the active list and stop if any rollback condition occurs.

docs/superpowers/plans/2026-08-18-import-list-sqlite-contention.md

@@ -0,0 +1,260 @@
+# Import-list SQLite Contention Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Eliminate the unnecessary SQLite write pressure that prevents all Radarr4K import lists from coexisting, without changing import-list or library-cleanup semantics.
+
+**Architecture:** Keep the existing serial per-list pipeline and make its persistence delta-based: insert new list mappings, update only mappings whose metadata identity changes, and delete removed mappings. Keep `keepAndUnmonitor`, but write only movies that transition from monitored to unmonitored. Use the current upstream Radarr SQLite busy timeout of one second so short WAL conflicts are waited out rather than returned as errors.
+
+**Tech Stack:** C#, .NET, System.Data.SQLite, NUnit, Moq, FluentAssertions.
+
+---
+
+### Task 1: Test and tune the SQLite connection timeout
+
+**Files:**
+- Create: `src/NzbDrone.Core.Test/Datastore/ConnectionStringFactoryFixture.cs`
+- Modify: `src/NzbDrone.Core/Datastore/ConnectionStringFactory.cs:44-53`
+
+- [ ] **Step 1: Write the failing test**
+
+```csharp
+[Test]
+public void should_configure_sqlite_busy_timeout_for_one_second()
+{
+    var connection = Mocker.Resolve<IConnectionStringFactory>().MainDbConnection.ConnectionString;
+    var builder = new SQLiteConnectionStringBuilder(connection);
+
+    builder.BusyTimeout.Should().Be(1000);
+}
+```
+
+Create the fixture as `DbTest`, which provisions the real `IConnectionStringFactory` and SQLite configuration. Import `System.Data.SQLite`, `FluentAssertions`, `NUnit.Framework`, `NzbDrone.Core.Datastore`, and `NzbDrone.Core.Test.Framework`.
+
+- [ ] **Step 2: Run the test to verify it fails**
+
+Run: `dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter FullyQualifiedName~ConnectionStringFactoryFixture --no-restore`
+
+Expected: the assertion reports the current `BusyTimeout` of `100` instead of `1000`.
+
+- [ ] **Step 3: Make the minimal implementation change**
+
+In `ConnectionStringFactory.GetConnectionString`, change only:
+
+```csharp
+BusyTimeout = 100
+```
+
+to:
+
+```csharp
+BusyTimeout = 1000
+```
+
+- [ ] **Step 4: Run the focused test to verify it passes**
+
+Run: `dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter FullyQualifiedName~ConnectionStringFactoryFixture --no-restore`
+
+Expected: one passing test, zero failures.
+
+- [ ] **Step 5: Commit the completed task**
+
+```bash
+git add src/NzbDrone.Core/Datastore/ConnectionStringFactory.cs src/NzbDrone.Core.Test/Datastore/ConnectionStringFactoryFixture.cs
+git commit -m "fix: wait for transient sqlite locks"
+```
+
+### Task 2: Persist import-list mappings only when they change
+
+**Files:**
+- Create: `src/NzbDrone.Core.Test/ImportListTests/ImportListMovieServiceFixture.cs`
+- Modify: `src/NzbDrone.Core/ImportLists/ImportListMovies/ImportListMovieService.cs:43-57`
+
+- [ ] **Step 1: Write failing unit tests**
+
+Create a `CoreTest<ImportListMovieService>` fixture. Build `ImportListMovie` helpers with a valid `Id`, `ListId`, `MovieMetadataId`, and lazy `MovieMetadata` containing a `TmdbId`. Configure `IImportListMovieRepository.GetAllForLists` for the target list. Add these tests:
+
+```csharp
+[Test]
+public void should_not_update_an_existing_mapping_with_the_same_metadata()
+{
+    GivenExisting(1, 101, 5001);
+    var incoming = GivenIncoming(101, 5001);
+
+    Subject.SyncMoviesForList(new List<ImportListMovie> { incoming }, 1);
+
+    Mocker.GetMock<IImportListMovieRepository>()
+          .Verify(x => x.UpdateMany(It.IsAny<IList<ImportListMovie>>()), Times.Never());
+}
+
+[Test]
+public void should_update_an_existing_mapping_when_its_metadata_identity_changes()
+{
+    GivenExisting(1, 101, 5001);
+    var incoming = GivenIncoming(101, 5002);
+
+    Subject.SyncMoviesForList(new List<ImportListMovie> { incoming }, 1);
+
+    Mocker.GetMock<IImportListMovieRepository>()
+          .Verify(x => x.UpdateMany(It.Is<IList<ImportListMovie>>(x => x.Single().Id == 1 && x.Single().MovieMetadataId == 5002)), Times.Once());
+}
+
+[Test]
+public void should_insert_new_and_delete_removed_mappings()
+{
+    GivenExisting(1, 101, 5001);
+    var incoming = GivenIncoming(202, 6001);
+
+    Subject.SyncMoviesForList(new List<ImportListMovie> { incoming }, 1);
+
+    Mocker.GetMock<IImportListMovieRepository>().Verify(x => x.InsertMany(It.Is<IList<ImportListMovie>>(m => m.Single().TmdbId == 202)), Times.Once());
+    Mocker.GetMock<IImportListMovieRepository>().Verify(x => x.DeleteMany(It.Is<IEnumerable<int>>(ids => ids.Single() == 1)), Times.Once());
+}
+```
+
+- [ ] **Step 2: Run the fixture to verify the no-op test fails**
+
+Run: `dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter FullyQualifiedName~ImportListMovieServiceFixture --no-restore`
+
+Expected: `should_not_update_an_existing_mapping_with_the_same_metadata` fails because the current implementation calls `UpdateMany` for every existing mapping.
+
+- [ ] **Step 3: Implement delta-based persistence**
+
+In `SyncMoviesForList`:
+
+```csharp
+var existingListMovies = GetAllForLists(new List<int> { listId });
+var existingByTmdbId = existingListMovies.ToDictionary(x => x.TmdbId);
+
+var inserts = listMovies.Where(x => !existingByTmdbId.TryGetValue(x.TmdbId, out _)).ToList();
+var updates = new List<ImportListMovie>();
+
+foreach (var listMovie in listMovies.Where(x => existingByTmdbId.TryGetValue(x.TmdbId, out _)))
+{
+    var existing = existingByTmdbId[listMovie.TmdbId];
+    listMovie.Id = existing.Id;
+
+    if (listMovie.MovieMetadataId != existing.MovieMetadataId)
+    {
+        updates.Add(listMovie);
+    }
+}
+
+var deletes = existingListMovies.Where(x => listMovies.All(y => y.TmdbId != x.TmdbId)).Select(x => x.Id).ToList();
+
+if (inserts.Any()) { _importListMovieRepository.InsertMany(inserts); }
+if (updates.Any()) { _importListMovieRepository.UpdateMany(updates); }
+if (deletes.Any()) { _importListMovieRepository.DeleteMany(deletes); }
+```
+
+Keep the returned list and the existing `TmdbId` identity behavior unchanged.
+
+- [ ] **Step 4: Run the fixture to verify all delta cases pass**
+
+Run: `dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter FullyQualifiedName~ImportListMovieServiceFixture --no-restore`
+
+Expected: three passing tests, zero failures.
+
+- [ ] **Step 5: Commit the completed task**
+
+```bash
+git add src/NzbDrone.Core/ImportLists/ImportListMovies/ImportListMovieService.cs src/NzbDrone.Core.Test/ImportListTests/ImportListMovieServiceFixture.cs
+git commit -m "fix: skip unchanged import-list writes"
+```
+
+### Task 3: Update only movies that need the unmonitor transition
+
+**Files:**
+- Modify: `src/NzbDrone.Core/ImportLists/ImportListSyncService.cs:201-227`
+- Modify: `src/NzbDrone.Core.Test/ImportListTests/ImportListSyncServiceFixture.cs:202-257`
+
+- [ ] **Step 1: Write a failing regression test**
+
+Add this test to `ImportListSyncServiceFixture`:
+
+```csharp
+[Test]
+public void should_not_update_already_unmonitored_movies_when_cleaning_library()
+{
+    _importListFetch.Movies.ForEach(m => m.ListId = 1);
+    GivenList(1, true);
+    GivenCleanLevel("keepAndUnmonitor");
+
+    var alreadyUnmonitored = _existingMovies.Select(x =>
+    {
+        x.Monitored = false;
+        return x;
+    }).ToList();
+
+    Mocker.GetMock<IMovieService>().Setup(v => v.GetAllMovies()).Returns(alreadyUnmonitored);
+    Mocker.GetMock<IImportListMovieService>().Setup(v => v.GetAllListMovies()).Returns(_list1Movies);
+
+    Subject.Execute(_commandAll);
+
+    Mocker.GetMock<IMovieService>().Verify(v => v.UpdateMovie(It.IsAny<List<Movie>>(), true), Times.Never());
+}
+```
+
+- [ ] **Step 2: Run the fixture to verify the test fails**
+
+Run: `dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter FullyQualifiedName~ImportListSyncServiceFixture --no-restore`
+
+Expected: the new test fails because the current code sends already-unmonitored movies to `UpdateMovie`.
+
+- [ ] **Step 3: Implement the minimal cleanup guard**
+
+In the `keepAndUnmonitor` branch, require `movie.Monitored` before logging and adding the movie:
+
+```csharp
+case "keepAndUnmonitor" when movie.Monitored:
+    _logger.Info("{0} was in your library, but not found in your lists --> Keeping in library but Unmonitoring it", movie);
+    movie.Monitored = false;
+    moviesToUpdate.Add(movie);
+    break;
+```
+
+Then invoke `_movieService.UpdateMovie(moviesToUpdate, true)` only when `moviesToUpdate.Any()`.
+
+- [ ] **Step 4: Run the focused fixture and preserve the transition behavior**
+
+Run: `dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter FullyQualifiedName~ImportListSyncServiceFixture --no-restore`
+
+Expected: all existing cleanup tests and the new already-unmonitored regression pass.
+
+- [ ] **Step 5: Commit the completed task**
+
+```bash
+git add src/NzbDrone.Core/ImportLists/ImportListSyncService.cs src/NzbDrone.Core.Test/ImportListTests/ImportListSyncServiceFixture.cs
+git commit -m "fix: skip unchanged library cleanup writes"
+```
+
+### Task 4: Verify, publish, deploy, and stage-enable lists
+
+**Files:**
+- Modify: no source files
+
+- [ ] **Step 1: Run the complete targeted regression suite**
+
+Run:
+
+```bash
+dotnet test src/NzbDrone.Core.Test/NzbDrone.Core.Test.csproj --filter 'FullyQualifiedName~ConnectionStringFactoryFixture|FullyQualifiedName~ImportListMovieServiceFixture|FullyQualifiedName~ImportListSyncServiceFixture|FullyQualifiedName~FetchAndParseImportListServiceFixture|FullyQualifiedName~CommandQueueManagerFixture|FullyQualifiedName~CommandQueueFixture' --no-restore
+```
+
+Expected: zero test failures.
+
+- [ ] **Step 2: Build the production image and preserve a database backup**
+
+Build the Radarr image using the established `radarr4k:6.5.1.2032-import-list-delta-persistence-<short-sha>` tag. Before recreating the container, create a timestamped, verified SQLite backup on Synology; do not modify list state during backup.
+
+- [ ] **Step 3: Deploy only Radarr4K and verify readiness**
+
+Replace only the Radarr4K image/container. Verify `/ping`, `PRAGMA quick_check`, WAL journal mode, the running image tag, no started commands, and zero new `Busy` entries after startup.
+
+- [ ] **Step 4: Stage-enable remaining lists without a manual sync**
+
+Keep Indian enabled. Enable Kids through Radarr’s API, then observe two natural scheduler cycles. If both finish without a command backlog or `Busy`, enable Movies and observe two more natural cycles. Do not enable a subsequent list if the current gate fails; immediately re-pause only the list under test.
+
+- [ ] **Step 5: Create the PR and report live verification separately**
+
+Push the branch and update the existing pull request with the design, source changes, targeted test output, image tag, backup confirmation, and each scheduler-cycle result. Do not claim all-list success unless all four staged cycles complete with no fresh lock entries.

docs/superpowers/specs/2026-08-17-import-list-worker-serialization-design.md

@@ -0,0 +1,27 @@
+# Import-list Worker Serialization Design
+
+## Goal
+
+Prevent a single `ImportListSync` command from running multiple enabled import-list pipelines concurrently and contending for Radarr's SQLite database.
+
+## Decision
+
+`FetchAndParseImportListService.Fetch()` will process enabled lists in their existing factory order. A list's fetch, TMDb mapping, import-list-movie persistence, and sync-status update must finish before the next list begins.
+
+The existing `lock (result)` serializes only the result-update block after independently scheduled long-running workers have started. Replacing the worker task fan-out with direct loop execution makes the no-overlap invariant explicit for the entire pipeline, including fetch-start timing and any writes reached from mapping or provider callbacks.
+
+## Scope
+
+Only the all-lists `Fetch()` path changes. `FetchSingleList()` retains its current behavior. No list settings, command-queue behavior, database schema, or provider behavior changes.
+
+## Failure Handling
+
+An exception in one list remains logged and does not prevent the next enabled list from being considered, matching the current worker-body behavior. Failed reports are not persisted; blocked lists continue to set `AnyFailure`.
+
+## Regression Evidence
+
+The focused fixture will make the first list's `Fetch()` wait on a test gate. While it is blocked, the test verifies the second list's `Fetch()` has not begun. This fails on the existing task-fan-out implementation and passes only when the next list is not started until the first pipeline returns.
+
+## Release Validation
+
+Build and run focused import-list and command-queue tests, push the follow-up commit to PR #5, build a production-version-matched overlay, and deploy only Radarr4K. Keep the verified SQLite backup. Re-enable RadarrIndian (4), RadarrKids (13), and Radarr-Movies (12), in that order, one at a time; observe two five-minute scheduled cycles per list before enabling the next. On any fresh SQLite busy/locked entry, over-cadence command, or command accumulation, disable the current list and stop the rollout.