Skip to content

feat(cli): add database conversion command for SQLite migration - #1327

Open
krbrs wants to merge 14 commits into
ShokoAnime:masterfrom
krbrs:databaseconverter
Open

feat(cli): add database conversion command for SQLite migration#1327
krbrs wants to merge 14 commits into
ShokoAnime:masterfrom
krbrs:databaseconverter

Conversation

@krbrs

@krbrs krbrs commented May 21, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR should be merged after #1326.

This PR adds database conversion support for migrating supported Shoko databases into SQLite through the shared SystemService startup lifecycle.

Since SQLite now performs well even with large Shoko databases, users may want to migrate existing installations to SQLite, which is also the default database backend for Shoko.

The converter currently supports one-way migration into SQLite from:

  • SQL Server
  • MySQL/MariaDB

Changes

Conversion mode

  • add --convert-db conversion mode to Shoko.CLI
  • move conversion execution into the shared SystemService startup lifecycle
  • make conversion startup host-agnostic for:
    • Shoko.CLI
    • Shoko.TrayService
    • future hosts
  • detect conversion mode from raw startup args position-independently
  • run conversion after early initialization and plugin setup, but before hosted services/web endpoints start
  • skip LateStart() during conversion mode

Source and target handling

  • default the source database from the current ServerSettings.Database
  • allow optional source overrides through:
    • --source-type
    • --source-connection-string
  • default the SQLite target path to Shoko's normal SQLite location
  • allow optional target override through:
    • --target-file
  • keep conversion scoped one-way to SQLite only

SQLite target bootstrap

  • create the target database through Shoko's SQLite bootstrap and migration path
  • migrate all supported application data except the Versions table, since migration metadata is backend-specific
  • isolate conversion startup in a temporary Shoko home so Quartz/bootstrap/runtime initialization does not touch the user's real Shoko home or image/cache directories

Data migration

  • copy shared application tables and columns from the source database into SQLite

Verification

  • verify migrated data after copy
  • compare copied tables between source and target
  • fail with explicit errors when verification detects a mismatch

Safety and guardrails

  • fail early if the source database is not already upgraded to the schema version expected by the running Shoko build
  • fail clearly for unsupported conversion directions
  • keep conversion scoped to supported same-version scenarios
  • require --overwrite before replacing an existing target SQLite file

Result

Shoko now has a practical migration path for users who want to move an existing SQL Server or MySQL/MariaDB installation to SQLite.

This is especially useful now that SQLite performs well enough for larger libraries while remaining the default Shoko backend.

@krbrs
krbrs marked this pull request as ready for review May 21, 2026 13:42
@krbrs
krbrs marked this pull request as draft May 21, 2026 20:54
Run conversion mode before hosted services and web endpoints start, and wrap early startup in a temporary conversion Shoko home.

This keeps Quartz, bootstrap, and other startup-time filesystem/database initialization from touching the user's real Shoko home during conversion.

Source settings are copied before isolation, while the target SQLite file still uses the resolved conversion output path.
@krbrs
krbrs marked this pull request as ready for review May 22, 2026 08:26
revam and others added 5 commits May 22, 2026 12:36
But keep the override for the db fixture.
- switch the database migration fixture from --config to the supported --home override
- remove bare convert-db detection and require --convert-db explicitly
- update conversion parser tests to use --home instead of --config
- keep parser coverage for false positives on option values
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
7 Security Hotspots

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants