[vavallee]

Summary

PR 2 of Phase 2 for #2832. Imports stop pushing to Calibre inline. They queue each imported ebook file in calibre_deliveries (the ledger from #2841) and kick a new worker, which delivers with retry and backoff. A book imported while Calibre is closed is now delivered on its own once Calibre is reachable, within about a minute.

Refs #2832

Worker (internal/calibre/delivery_worker.go). Deliverer with Enqueue, Kick and RunDeliveries. It is registered by scheduler.WithCalibreDeliverer as calibre-deliver, running @every 1m. The tick and a kick share one sync.Mutex taken with TryLock, so passes never overlap. Kicks and ticks both run in the jobs group, so shutdown drains a pass that is in flight. A pass works like this:

• Mode off: returns.
• Reads DueBatch(now, 50). An empty queue returns before any HTTP call.
• Resolves the client through AdderResolver. In plugin mode it runs HealthDetail with a 5s timeout. If the bridge is unreachable or degraded, the pass returns and no row is touched.
• Delivers the rows one at a time. Each ledger write is its own statement, and no transaction is held across a Calibre call.
• Per row: re-reads the book and the book_files row. It skips the row if the book is gone, the file row is gone, or the file is missing on disk. It builds metadata at delivery time with BuildMetadata, taking the edition from the row's edition_id or matching one by extension (editionForFile, factored out of the syncer), plus the series and the cover.
• Outcomes:
  • Success: MarkDelivered(id, "added", library). The library is what the plugin's health reports, or library_path for calibredb.
  • 409: MarkDelivered(id, "already"). Ownership for the metadata PATCH now comes from DeliveredByCalibreID: an earlier delivered row for this book whose target library matches, or is empty on a backfilled row. book.CalibreID == id no longer decides it.
  • bad_format / path_forbidden: terminal. Anything else backs off 1m, 5m, 15m, 1h, 6h, then 24h, and goes terminal on the 8th attempt.
  • Transport error, cancelled ctx, a 503 after the client's own retries, or ErrDisabled: the pass stops and no attempt is counted.
• Logging: one INFO calibre delivery pass line with counts when anything happened. WARN on a row's first failure (calibre delivery: add failed, will retry) and on giving up (calibre delivery: giving up on a book, with book id and code).

books.calibre_id. It now means the id in the library at calibre.library_path, and the model field documents this. A delivery writes it only through the new SetCalibreIDIfUnset (WHERE calibre_id IS NULL), only when the book is not Calibre origin (isCalibreOrigin), and only when library_path is empty or equals the target (sameLibraryPath, factored out of sameCalibreLibrary). Existing values are never nulled or replaced.

Importer. pushToCalibre, pushCalibreAdd, reconcileExistingCalibreBook and the Scanner's cover code are gone. The ebook loop calls enqueueCalibreDelivery for each imported file. It passes the book_files id, the download's EditionID when it resolved to a real edition of the book, and the path; the format is the lower case extension. The worker is kicked once after the loop. The audiobook path queues nothing. External import mode is unchanged (it returns earlier), and so is the CWA mirror. With the integration off nothing is queued.

Covers. Cover resolution now lives in calibre.CoverSource.PathFor and is shared by the worker and Push all. Push all never set CoverPath before; it now sends covers (Syncer.WithCovers). Nothing else in Push all changes. That is PR 3.

Plugin client. A rejection is now a typed *PluginError{Status, Code, Detail}, read with PluginErrorCode(err), and the error string is unchanged. HealthState gains Library.

Concurrency with Push all. A Push all run and the worker can push the same book at the same time. That is safe: the bridge dedupes on the bindery identifier, so the second push comes back as a 409 and is recorded as already. In calibredb mode, calibredb add without --duplicates skips a book it already holds, but Push all is plugin only anyway.

Deviations from the plan

• DueBatch runs before the health probe. Without that, an idle queue would cost a request to the Calibre host every minute. The probe still runs before any row is touched.
• A 503 is treated like unreachable: no attempt is counted and the pass stops. The client has already retried it for about 30s, and it means Calibre is busy (db_unavailable), not that the book is bad.
• A row's first failure logs at WARN and later retries at DEBUG. With DEBUG only, a bad remap would stay invisible for about a day and a half.
• Ledger writes after a successful add use context.WithoutCancel, so a delivery that completes during shutdown is still recorded.
• Kicks that arrive while a pass is running are dropped, and the one minute tick picks those rows up. There is no rerun flag.

How it was verified

• go build ./... && go vet ./... pass. GOOS=windows go build ./... passes. go test ./cmd/... ./internal/... exits 0, with the exit code read directly and not piped.
• golangci-lint run on calibre, importer, scheduler, db, models and cmd: 0 issues.
• The contract suite ran against a fresh bindery-plugins checkout (BINDERY_PLUGIN_SRC=... go test ./internal/calibre/ -run Contract) and passed. The plugin's documented codes (bad_format, path_forbidden, db_unavailable on 503) match the codes the worker branches on.
• New tests:
  • Worker: unreachable or degraded bridge leaves rows untouched with attempts 0; an empty queue sends no probe; mode off; the adder is resolved per pass; success sets calibre_id under the rule (7 cases, including Calibre origin, a different library and an existing id); 409 ownership from the ledger (4 cases); terminal codes; the backoff schedule and the 8th attempt; a transport error stops the pass; a 503 is not an attempt; ErrDisabled; a missing file is skipped; calibredb mode; overlapping kick and tick deliver once; metadata is built at delivery time; the real PluginClient against httptest, both down and up.
  • Cover helper and the Push all cover.
  • SetCalibreIDIfUnset.
  • Importer: one row per ebook file, and the import returns while the adder is blocked; audiobooks queue nothing; mode off queues nothing; one kick per download.
  • Scheduler: the job is registered every minute only when a deliverer is wired.
• Fail before, on origin/main with main API variants of the tests:
  • The importer test fails on both counts: the import waited 6s on Calibre (adder called 2 times inline) and queued rows = 0.
  • The Push all cover test fails: bulk push sent no cover.
  • On main, the 409 ownership rule is the one TestPushToCalibre_409UpdatesARowBinderyOwns encodes (book.CalibreID == id). The new "books.calibre_id alone is not ownership" case asserts the opposite.
• Mutation checks. Each mutant passed go vet, and its named test failed:
  • Terminal on the 9th attempt instead of the 8th, and a shifted backoff index: OtherErrorsBackOffThenGiveUp.
  • Terminal codes ignored: TerminalPluginCodesFailAtOnce.
  • Unreachable early return removed: UnreachableBridgeLeavesRowsUntouched.
  • calibre_id guard with the origin check, the library check or the existing-id check removed: SuccessMarksDeliveredAndRecordsCalibreID.
  • TryLock removed, and TryLock turned into a blocking Lock: OverlappingPassesDoNotDoubleDeliver.
  • Ownership read from books.calibre_id: 409MarksAlreadyAndOwnershipComesFromTheLedger.
  • A transport error counted as an attempt: TransportErrorStopsThePassWithoutAnAttempt.

🤖 Generated with Claude Code

https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9

[codecov]

Codecov Report

❌ Patch coverage is 75.64470% with 85 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/calibre/delivery_worker.go	74.60%	40 Missing and 25 partials ⚠️
internal/importer/scanner_handoff.go	61.53%	8 Missing and 2 partials ⚠️
internal/db/books.go	50.00%	2 Missing and 2 partials ⚠️
internal/calibre/cover_path.go	82.35%	2 Missing and 1 partial ⚠️
internal/calibre/syncer.go	85.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!