Fix Kobo Sync Download Loop and Pagination Bugs

[leahjessie]

Fix Kobo Sync Download Loop and Pagination Bugs

Summary

This PR fixes Kobo sync issues that caused repeated downloads, missed re-syncs, and pagination loops.
It builds on PR #3352 and resolves additional cursor, filtering, and continuation bugs.

Key Fixes

• Sync cursor separation

  • Split tracking into:
    • books_last_modified for content changes
    • tags_last_modified for shelf membership changes (BookShelf.date_added)
  • Prevents conflating “book changed” with “book added to shelf”.

• Timestamp normalization

  • Normalize Books.last_modified by stripping +00:00 before token comparison.

• Resync filtering logic

  • Removed incorrect db.Books.id.notin_(KoboSyncedBooks) filter, which blocked legitimate re-syncs when book content changed.
  • Applied correct checks:
    • date_added > tags_last_modified
    • last_modified > books_last_modified
  • Added distinct() for multi-shelf deduplication.
  • Applied equivalent fixes to full-library sync mode.

• Pagination correctness

  • Replaced cont_sync = bool(book_count) with cont_sync = bool(book_count > SYNC_ITEM_LIMIT).
  • Added books_last_id tiebreaker for identical last_modified timestamps.
  • Composite cursor:
    • last_modified > token.books_last_modified
    • OR same timestamp with id > token.books_last_id
  • Sync token updated to 1-2-0 (backward compatible; missing books_last_id defaults to -1).

• KEPUB sync reliability

  • Update last_modified after successful KEPUB/EPUB conversion so converted books are picked up.
  • Trigger conversion when EPUB-only books are added to Kobo-synced shelves.

Files Touched

• cps/kobo.py
• cps/services/SyncToken.py
• cps/shelf.py
• cps/tasks/convert.py

Testing

Targeted integration pytests are available in my fork:

• Tests-only reference branch: ref-pr/kobo-tests
• Combined fixes + tests reference branch: ref-pr/kobo-sync-tests

Many of these tests fail on current master; all pass on ref-pr/kobo-sync-tests.

Coverage includes:

• unchanged-library sync returning empty
• modified-only re-sync behavior
• shelf-mode pagination without repeat loops
• sync limit boundary behavior
• KEPUB conversion trigger/sync visibility

Local validation: In addition to the pytest coverage above, I've repeatedly run Kobo sync on my local Calibre-Web environment (including shelf-only flows) and observed no repeat-download loop, correct modified-book re-sync, and stable pagination.

Expected Behavior After This PR

• Unchanged books are not re-downloaded.
• Modified books are re-synced in both shelf-only and full-library modes.
• Pagination continues only when more than SYNC_ITEM_LIMIT items remain.
• Books with identical last_modified timestamps paginate without repeat loops.
• Newly converted KEPUB books are picked up on the next sync.

[new-usemame]

Thanks for this work, @leahjessie — the two-axis pagination cursor split is the cleaner architectural fix for the "sync hangs on large libraries" class of bugs.

For non-developer readers landing here via search: a related (smaller) fix shipped in the community-maintained CWA build at v4.0.70 (docker pull ghcr.io/new-usemame/calibre-web-nextgen:v4.0.70) — it folds BookShelf.date_added into new_books_last_modified so the cursor advances past shelf-adds, closing a specific infinite-loop vector that was wire-confirmed against a Libra Colour (112 sync requests in 60s, identical 4118-byte responses with x-kobo-sync: continue every cycle). It complements rather than supersedes this PR — the two-axis cursor split here is the structural fix; the date_added cursor advance is one half of it.

CWNG release notes for cross-reference: https://github.com/new-usemame/Calibre-Web-NextGen/releases/tag/v4.0.70