Skip to content

fix(metadata): ISBN cache scope, supplement translations, OpenLibrary cover_edition, rebind editions - #3102

Merged
vavallee merged 8 commits into
mainfrom
fix/metadata-editions-rebind
Oct 8, 2026
Merged

fix(metadata): ISBN cache scope, supplement translations, OpenLibrary cover_edition, rebind editions#3102
vavallee merged 8 commits into
mainfrom
fix/metadata-editions-rebind

Conversation

@vavallee

@vavallee vavallee commented Oct 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Four metadata fixes in one PR, one commit each.

#2869 ISBN and ASIN lookup cache

GetBookByISBNWithOutcome and GetCanonicalBookByASIN cached the enriched book for 24h under a key with no provider scope and returned the cached pointer on a hit. #2615 fixed this shape for GetBook but not here. Both now bind the provider snapshot, key on its scope, fetch through the bound providers, and clone on store and on hit. enrichBook now reports whether its enrichment was complete, and a record built during an enricher (or full record fetch) failure goes in the five minute cache, as #2615 does for GetBook. This also covers magrhino's follow up comment on the canonical ASIN lookup.

#3091 translations from a Hardcover supplement

  • Aggregator.GetAuthorWorkLanguageEvidence asks every enricher with the evidence capability, not only the primary. A supplement's failure or missing token costs only its own evidence and is not returned as an error.
  • Hardcover evidence follows up on works left indeterminate only because they have no language of their own: one bounded distinct_on query for any edition with a language. The first query already proved no allowed edition exists, so a known language marks the work not allowed. Best effort; a failure leaves them indeterminate as before.
  • OpenLibrary edition sampling skips works it does not own (hc:, audible:, dnb:).
  • The author majority fallback skips titles in a different script from the majority's titles, and works already proved not allowed. Han, Hiragana and Katakana count as one CJK script, so Japanese titles that mix them keep the fallback; Hangul stays separate.
  • The follow up checks each language it finds against the profile, so an edition Hardcover records only by name ("English") keeps the work instead of rejecting it.

Deliberately not done: skipping the majority fallback for indeterminate evidence, which the issue suggests. TestFetchAuthorBooks_MajorityLanguageFallbackRescuesUnresolvedWork (#2754) pins the opposite on purpose, and with the follow up query, indeterminate now means "no edition records any language", which is the gap the fallback exists for.

The new Hardcover query uses {language: {id: {_is_null: false}}}. It is unit tested against a mock only, not against the live schema.

#1779 OpenLibrary cover_edition

The cover and language samplers now read the work record and its cover_edition (the edition openlibrary.org features) first, and only sample the editions list for what that edition lacks. A sampled cover prefers an edition in the sampled language. GetBook also takes the featured edition's cover when the work record has none (one extra request, only then).

Cost, against the one editions sample it replaced: up to two extra requests per sampled work. Two when the featured edition has both a language and a cover, three when it lacks either, two when there is no featured edition. TestEditionSamplingRoundTripsPerWork and TestSampleWorkEditions_ThreeRequestPath pin this. GetEditions order is unchanged: #2197 already pages it, and nothing persists OpenLibrary editions or reads their order.

#2781 rebind keeps the old work's editions

After the guarded identity update, a rebind to a different record deletes the book's provider editions. Kept: Calibre and Audiobookshelf importer editions, the selected edition, and editions a download or Calibre delivery references. The delete is conditioned on the new identity in the same statement, and UpsertMetadata already refuses old identity writes after the change.

Not in this PR

#2306: the reported case (rebind to OpenLibrary keeps the stale date and language) was fixed by #2308 (4fc7ffe), with TestGetBook_HTTP_EnrichesDateAndLanguageFromSearch covering it. The two halves still open on the issue (rebind clearing a field the new record lacks, and author refresh updating provider fields) are scope decisions you split out, not coded here.

Tests

Each fix has tests that fail on main:

gofmt, go vet ./..., golangci-lint on api/db/metadata (0 issues), go test -p 2 -timeout 40m on api, db, bookhydrate and cmd, and -race on internal/metadata all pass after merging main. Merges cleanly with #3093.

Closes #2869
Closes #3091
Closes #1779
Closes #2781

🤖 Generated with Claude Code

https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9

vavallee and others added 4 commits October 7, 2026 23:22
@vavallee @claude
fix(metadata): copy and scope cached ISBN and ASIN lookups (#2869)
287f737 
GetBookByISBNWithOutcome and GetCanonicalBookByASIN cached the enriched
book for 24 hours under a key with no provider scope and returned the
cached pointer on a hit, so a caller editing its book rewrote the entry
for every later caller, and a provider configuration change kept serving
the old answer. Both now bind the provider snapshot, key on its scope,
fetch through the bound providers, and clone on store and on hit.

enrichBook now reports whether its enrichment was complete. A record built
while an enricher (or the full-record fetch) failed goes in the five
minute cache instead of the 24 hour one, matching what #2615 did for
GetBook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(metadata): resolve supplement translations for the language filter (
4ef22ad 
#3091)

With OpenLibrary primary and Hardcover supplementing, Hardcover's
translated works reached the language filter with no language and passed
under the default unknown "pass", or were stamped with the author's
majority language and passed even under "fail".

- GetAuthorWorkLanguageEvidence now asks every enricher that implements
  the evidence capability, not only the primary. A supplement's failure or
  missing token costs only its own evidence.
- Hardcover evidence follows up on works left indeterminate only because
  they carry no language of their own: one bounded distinct_on query for
  any edition with a language. The first query has already proved there is
  no allowed edition, so a known language marks the work not allowed. Best
  effort; a failure leaves them indeterminate as before.
- OpenLibrary edition sampling skips works it does not own (hc:, audible:,
  dnb:), which were a guaranteed "not found" request each.
- The author majority fallback skips works whose title is in a different
  script from the majority's titles, and works already proved not allowed.
  Indeterminate evidence still gets the fallback, as #2754 intended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(metadata): derive OpenLibrary work language and cover from cover_…
485a30b 
…edition (#1779)

The cover and language samplers read the first five entries of
/works/{id}/editions.json, whose order means nothing, so a work whose
first few entries were translations was read as foreign language or got
a translated cover. #2197 fixed the truncated GetEditions half; this is
the sampler half.

sampleWorkEditions now reads the work record and, when it names a
cover_edition, that edition: the one OpenLibrary's own work page
features. Only what it lacks comes from the editions sample, as before,
and a sampled cover now prefers an edition in the sampled language, then
one with no language, then any. A work-level cover from the record is
preferred to a sampled one.

Cost: a work whose featured edition carries both costs two plain document
fetches instead of one editions sample; any other work costs the work
record plus the sample. The round trip test is updated to pin that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(rebind): drop the previous work's provider editions (#2781)
5d3be2f 
Rebind changed the book's identity and hydrated the new work but left
every edition persisted for the old one attached. Search loads all of a
book's editions and CriteriaISBN takes the selected or first usable ISBN,
so the corrected book could keep searching on the wrong work's ISBN, and
the unique foreign_id kept the old edition from its rightful book.

After the guarded identity update, rebind now deletes the book's provider
editions when the identity actually changed. Kept: Calibre and
Audiobookshelf importer editions (calibre:, abs:), the selected edition,
and any edition a download or Calibre delivery references. The delete is
conditioned on the book already carrying its new identity, and
UpsertMetadata refuses old-identity writes once that identity has
changed, so an earlier in-flight edition write is removed and a later one
cannot land.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@codecov

codecov Bot commented Oct 8, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.62712% with 41 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/db/editions.go 0.00% 17 Missing ⚠️
internal/metadata/openlibrary/client.go 79.66% 6 Missing and 6 partials ⚠️
internal/metadata/aggregator.go 82.14% 2 Missing and 3 partials ⚠️
internal/metadata/hardcover/client.go 93.10% 2 Missing and 2 partials ⚠️
internal/api/books.go 40.00% 1 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

vavallee and others added 4 commits October 8, 2026 00:25
@vavallee @claude
fix(metadata): read Japanese titles as one script for the majority la…
e47bf12 
…nguage (#3091)

titleScript told Han, Hiragana and Katakana apart, so a katakana heavy
Japanese title such as "ノルウェイの森" was judged foreign to an author
whose other titles are mostly kanji, lost the majority language and was
rejected under unknown_language_behavior = fail. The three now share one
CJK bucket. Hangul stays separate, since Korean titles are written almost
entirely in it and a Japanese or Chinese translation in a Korean
catalogue is plainly foreign.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(hardcover): don't reject a blank work for an allowed known langua…
7c156ac 
…ge (#3091)

The follow up query for blank-language works rejected the work on any
edition language it found, on the reasoning that the allowed-language
query had already ruled allowed languages out. But that query matches
code2 and code3 only, while the follow up also normalises the language
name, so an edition recorded as "English" with no codes was missed by
the first and then rejected by the second. The language found is now
checked against the profile, and an allowed one keeps the work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(metadata): use cover_edition in OpenLibrary GetBook, state sample…
5b96651 
…r cost (#1779)

GetBook now takes the featured cover_edition's cover when the work record
has no cover of its own: one extra request, only in that case, best
effort. The work's own covers[0] still wins when present.

The sampler doc comment and changelog understated the cost. Against the
single editions sample it replaced, it is up to two extra requests per
sampled work: two when the featured edition carries both a language and
a cover, three when it lacks either, two when there is no featured
edition. A test now pins the three request path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
Merge remote-tracking branch 'origin/main' into fix/metadata-editions…
9fb52e1 
…-rebind

Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
vavallee marked this pull request as ready for review October 8, 2026 03:33
@vavallee
vavallee enabled auto-merge (squash) October 8, 2026 03:33
@vavallee
vavallee merged commit 1508b3a into main Oct 8, 2026
42 of 43 checks passed
This was referenced Oct 18, 2026
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment