add Soulseek (slskd) release source and download handler

[dskvr]

Summary

Adds Soulseek as a source, using a slskd instance for both search (indexer) and download. Unlike torrent/usenet sources, search and download are the same service here: a result is a file on a specific peer, and only slskd can fetch it, so this ships as one plugin package (shelfmark/release_sources/slskd/) with its own handler rather than a new DownloadClient protocol.

What's included

Indexer — SlskdSource

• Runs the plan's title+author query, then the bare title as a fallback (peers rarely name files with the author); expand_search runs every variant.
• Ebooks: one release per file. Audiobooks: a peer's folder of audio files is grouped into a single multi-file release (archives stay separate), since Soulseek audiobooks are almost always per-chapter files.
• Filters by the configured Supported (Audiobook) Formats, skips locked files, ranks free upload slot → shortest queue → fastest upload speed.
• Custom columns: Peer, Slot (free / queue N / busy), Format, Size; extra sort options for upload speed and queue length.

Downloader — SlskdHandler

• Enqueues every file of a release in slskd and polls transfer state, reporting progress, speed and remote queue position.
• Fails fast on Completed, Errored/Rejected/TimedOut/..., honours cancel, and gives up after a configurable queue timeout when the peer never starts sending.
• Locates completed files at <downloads>/<last remote folder>/<file> (slskd's layout), via the SLSKD_DOWNLOAD_PATH override, or slskd's reported directory passed through Remote Path Mappings for the new slskd client host.
• Multi-file releases are staged into TMP_DIR so unrelated files that happen to share the same per-folder directory are never imported.
• After a successful import the transfer record is removed from slskd and the now-empty folder is deleted.
• The download spec (peer, files, sizes) is persisted in retry_source_context, so retries survive a restart without an in-memory cache.

Settings — new "Soulseek (slskd)" tab: enable, URL, API key, Test Connection (verifies the key and that slskd is logged in to Soulseek), downloads path, search wait, max peer responses, queue timeout, remove-completed toggle.

Small supporting changes — ReleaseProtocol.SOULSEEK, builtin source registration, slskd in the path-mapping client dropdown, frontend protocol type + dot colour, readme and env-var docs, slskd service in docker-compose.test-clients.yml, and scripts/test_clients.py --write-slskd-config to bootstrap a test config.

slskd behaviour worth knowing

• DELETE /transfers/downloads/{user}/{id}?remove=true only removes a completed transfer. On a live one it cancels and leaves a Completed, Cancelled record, so cancellation does a second removal pass.
• Newer slskd versions return the created transfers in the POST /transfers/downloads/{user} body; older ones return an empty 201. Both are handled (fallback looks the transfers up by filename).
• slskd's per-search searchTimeout counts from the last response, so the client also enforces a wall-clock deadline of its own.

Testing

• 148 new tests in tests/slskd/: API client, release building, the full handler lifecycle against a scripted fake slskd (success, multi-file staging, peer failure, cancel, queue timeout, missing/unsafe paths, cleanup), settings, and a queue → task → retry round-trip through the orchestrator.
• tests/slskd/test_live_slskd.py runs against a real instance when SLSKD_TEST_URL / SLSKD_TEST_API_KEY are set (skipped otherwise).
• Full suite: 3172 passed. ruff check, ruff format, basedpyright, vulture and frontend tsc clean.
• Verified end to end against a live slskd 0.26.0 logged in to Soulseek: a search for A Study in Scarlet returned 37 releases from real peers; the handler downloaded one (size verified on disk), cleanup removed the record; cancelling a 15-file audiobook mid-transfer left no records behind in slskd.

Setup for users

1. Run slskd, logged in to Soulseek, with an API key in slskd.yml (web.authentication.api_keys.<name>.key, role readwrite).
2. Mount slskd's downloads folder into the Shelfmark container.
3. Settings → Soulseek (slskd): enable, enter URL + API key, Test Connection. If the folder is at a different path inside Shelfmark, set Downloads Path or add a Remote Path Mapping for client slskd.

[Copilot]

🟡 Changes recommended

Critical import errors and unresolved search, transfer cleanup, and configuration issues block approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds Soulseek/slskd as a searchable release source and download handler with settings, retry support, cleanup, UI integration, documentation, and tests.

Changes:

• Adds the slskd API client, source, release grouping, and transfer handling.
• Adds configuration, path mappings, frontend support, Docker setup, and documentation.
• Adds unit, integration, and optional live-instance tests.

File summaries

File	Summary and final review notes
tests/slskd/test_source.py	Tests search, filtering, grouping, and ranking.
tests/slskd/test_settings.py	Tests settings and connection actions.
tests/slskd/test_orchestrator_integration.py	Tests queue, task, and retry integration.
tests/slskd/test_live_slskd.py	Optional live slskd integration tests.
tests/slskd/test_handler.py	Tests download lifecycle and cleanup.
tests/slskd/test_api.py	Tests the slskd API client.
tests/slskd/conftest.py	Provides slskd test fixtures and stubs.
tests/slskd/__init__.py	Defines the test package.
src/frontend/src/utils/colorMaps.ts	Adds the Soulseek protocol color.
src/frontend/src/types/index.ts	Adds the Soulseek protocol type.
shelfmark/release_sources/slskd/source.py	Implements search and release construction. Moderate (2 votes): expanded searches can lose audiobook files during deduplication. Moderate (2 votes): timeout values below 5 seconds are accepted despite slskd rejecting them. Moderate (1 vote): enabled configurations without an API key are treated as available.
shelfmark/release_sources/slskd/settings.py	Defines Soulseek settings and connection testing. Moderate (1 vote): explicitly blank form values can fall back to persisted URL or API key values.
shelfmark/release_sources/slskd/handler.py	Implements transfer polling, staging, cancellation, and cleanup. Critical (1 vote): invalid exception-list syntax prevents import. Moderate (1 vote): filename lookups can target unrelated transfers. Moderate (2 votes): cancellation during file location can skip cleanup, including the line 571 path. Moderate (1 vote): missing files can leave other transfers running.
shelfmark/release_sources/slskd/api.py	Implements the slskd REST client. Critical (1 vote): invalid exception-list syntax prevents import. Moderate (3 votes): search timeout is sent in milliseconds instead of seconds.
shelfmark/release_sources/slskd/__init__.py	Initializes the slskd plugin.
shelfmark/release_sources/__init__.py	Registers the protocol and builtin source.
shelfmark/config/settings.py	Adds slskd path-mapping configuration.
scripts/test_clients.py	Adds slskd test configuration bootstrap support.
readme.md	Documents Soulseek setup.
docs/environment-variables.md	Documents slskd environment variables.
docker-compose.test-clients.yml	Adds the slskd test service.

Review details

Suppressed comments (5)

shelfmark/release_sources/slskd/handler.py:474

• This lookup returns every non-removed transfer whose filename matches the release, including older failed/completed attempts and another task's transfer for the same peer and filename. _snapshot already handles multiple records per filename, so cancellation and post-import cleanup can cancel/delete unrelated records; retain and use only the IDs created for this download (or the newest fallback record per file).

        wanted = set(spec.filenames)
        return [
            str(t["id"])
            for t in transfers
            if t.get("id") and str(t.get("filename") or "") in wanted and not t.get("removed")

shelfmark/release_sources/slskd/handler.py:573

• When post-processing fails, the orchestrator keeps task.staged_path so a retry can reuse the downloaded file, but this branch discards the only cleanup references. A later retry can therefore import the staged file successfully without removing its slskd transfer record or source folder; persist the transfer IDs or reconstruct cleanup from the task before dropping them.

        refs = self._cleanup_refs.pop(task.task_id, None)
        if refs is None or not success:
            return

shelfmark/release_sources/slskd/handler.py:419

• When one file in a multi-file release disappears, this returns after the grace polls without cancelling the other matching transfers. The task reports an error while the remaining downloads continue in slskd, leaving an orphaned transfer consuming peer resources; perform the same release cleanup before returning.

                if missing_polls >= MAX_MISSING_POLLS:
                    status_callback("error", "Transfer disappeared from slskd")
                    return None

shelfmark/release_sources/slskd/settings.py:25

• Using or makes an explicitly blank value from the current form fall back to persisted configuration. If a user clears a saved URL or API key and clicks Test Connection before saving, this tests the old value and can report success even though the form will save an unusable blank value; distinguish missing form keys from explicitly blank values.

    raw_url = str(current_values.get("SLSKD_URL") or config.get("SLSKD_URL", "") or "")
    api_key = str(current_values.get("SLSKD_API_KEY") or config.get("SLSKD_API_KEY", "") or "")

shelfmark/release_sources/slskd/source.py:485

• is_available() treats an enabled URL with no API key as configured, even though the Soulseek settings action requires a key and the normal slskd authentication setup rejects headerless requests. The source then creates an empty-key client, catches the resulting authorization failure in search(), and presents an empty result instead of an unavailable/misconfigured source. Require the same non-empty key here (as Prowlarr does at shelfmark/release_sources/prowlarr/source.py:1224-1230).

    def is_available(self) -> bool:
        if not config.get("SLSKD_ENABLED", False):
            return False
        return bool(normalize_http_url(str(config.get("SLSKD_URL", "") or "")))

• Files reviewed: 20/21 changed files
• Comments generated: 6
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

A syntax error blocks API imports, and several handler, configuration, availability, query, and grouping issues remain unresolved.

Review details

Suppressed comments (8)

Previously missed (1) — in code that hasn't changed since the last review.

shelfmark/release_sources/slskd/settings.py:25

• Using or treats an explicitly cleared form field as missing and silently reuses the saved URL/key. That makes it impossible for Test Connection to validate the values the user just entered (or cleared); distinguish a missing key from a present empty value before falling back to config.

shelfmark/release_sources/slskd/api.py:166

• This except clause is invalid Python 3 syntax, so importing shelfmark.release_sources.slskd.api raises a SyntaxError and prevents the new source/handler from loading at all. Wrap the exception types in parentheses.

        except requests.exceptions.RequestException, SlskdError:

shelfmark/release_sources/slskd/handler.py:419

• When only part of a multi-file release disappears, this branch gives up without cancelling the transfers that are still present. Those files can continue downloading in slskd after Shelfmark has marked the task failed; the other terminal/error branches call _cancel_transfers before returning. Cancel the release here as well (the call is harmless when all records have disappeared).

            if snap.found < len(spec.files):
                missing_polls += 1
                if missing_polls >= MAX_MISSING_POLLS:
                    status_callback("error", "Transfer disappeared from slskd")
                    return None

shelfmark/release_sources/slskd/handler.py:377

• Once _poll has reported success, cancellation while _wait_for_files is locating the completed files takes this branch without calling _cancel_transfers. The completed records (and potentially the files) are therefore left behind even though the task reports cancelled; perform the same removal pass here before returning.

            if local_paths is None:
                if cancel_flag.is_set():
                    status_callback("cancelled", "Cancelled")
                return None

shelfmark/release_sources/slskd/source.py:485

• is_available() treats an enabled source with a URL but no API key as usable, so the normal search flow includes it and then performs unauthenticated requests that fail instead of reporting the source as unconfigured. The settings action and setup both require an API key; include a non-empty SLSKD_API_KEY in this availability check (or make the client auth-optional consistently).

    def is_available(self) -> bool:
        if not config.get("SLSKD_ENABLED", False):
            return False
        return bool(normalize_http_url(str(config.get("SLSKD_URL", "") or "")))

shelfmark/release_sources/slskd/source.py:349

• _plan_queries flattens each variant into an author query plus fallback, then truncates that list to three strings. With multiple localized title variants, later variants (and even the second variant's bare-title fallback) are never searched, so expand_search=True does not run every variant as the surrounding comment promises. Let the wall-clock deadline cap the work, or cap variants before expanding each one.

    return unique[:MAX_QUERY_VARIANTS]

shelfmark/release_sources/slskd/source.py:277

• An empty directory is a valid result from split_remote_path for a bare filename, but this key groups every root-level audio match from the peer into one audiobook. Unrelated bare files returned by one search response can therefore be downloaded together; use a per-file key when peer_file.directory is empty (or skip grouping for those files).

            by_directory: dict[str, list[PeerFile]] = {}
            for peer_file in matching:
                by_directory.setdefault(peer_file.directory, []).append(peer_file)

shelfmark/release_sources/slskd/source.py:472

• With expand_search=True, the title+author and bare-title searches can return different files from the same peer directory. directory_release uses (username, directory) as its source id, so this loop discards the later release wholesale and can produce an audiobook missing chapters found by the other query. Merge the files for the same peer/directory across query results before deduplicating releases.

            for release in build_releases(responses, content_type=content_type, formats=formats):
                if release.source_id in seen_ids:
                    continue
                seen_ids.add(release.source_id)
                releases.append(release)

• Files reviewed: 20/21 changed files
• Comments generated: 0 new
• Review effort level: Lite

[splitsec2]

@dskvr I opened #1428, which is this PR rebased onto current main with some changes on top. Your two commits are the first in it with your authorship kept, and the plugin, handler, API client, settings tab and tests are yours. The main change is that downloads are polled by transfer id into a per-task folder, because another app clearing slskd's completed list made finished downloads disappear for the original handler. If you'd rather keep #1335 going and take those changes as commits here, say so and I'll close #1428.

[dskvr]

Superceded by #1428