Skip to content

fix(metadata): author middle initials, cross provider author duplicates, ABS match by file (#2881, #2117, #1691) - #3106

Merged
vavallee merged 6 commits into
mainfrom
fix/author-identity
Oct 8, 2026
Merged

vavallee merged 6 commits into
mainfrom
fix/author-identity

Conversation

@vavallee

@vavallee vavallee commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

Three author and book identity fixes, one commit each.

Closes #2881
Refs #2117
Refs #1691

#2881: different middle initials matched as one author

MatchAuthorName("J. R. Smith", "J. T. Smith") was FuzzyAuto. Two causes: the glued initials form jr was scored as a spelled out forename beginning with J, so authorInitialsCompatible("jr", "j t") passed; and the whole string Jaro-Winkler fallback read kelly l against kelly m as a typo (+1.5).

  • Given names are lined up position by position (authorInitialsCompare). Two initials that disagree after an equal forename score 0 (ambiguous with an equal surname, so a person can confirm). A disagreeing first initial, or a middle conflict after a forename that only abbreviates, is a conflict (-3). Initial against a different whole word further in still falls to the string comparison, which keeps "Ursula K. Le Guin" matching "Ursula Le Guin".
  • A missing initial where both sides are only initials ("A. Smith" against "A. B. Smith") scores 0 instead of auto matching.
  • Field scoring decides which words are initials from the name as written: in a mixed case name a short word splits into letters only when it is in capitals ("JRR") or has no vowel ("Jrr", "Tj"). The exact and swap tiers use the public variant sets exactly as on main, so "Jrr Tolkien", "Ra Salvatore" and "Tolkien, Jrr" stay Exact for alias binding, the bulk import skip, the Calibre alias rule and the sort name dedupe (review follow up).
  • A dropped initial is still accepted where a title backs it: AuthorMatchResult.DroppedInitial and ConfirmedByTitle() accept the ambiguous band only when one side leaves out initials and nothing disagrees. Used by canonical book matching, enrichment candidate pick, series author agreement, the ASIN metadata map and the Hardcover series author score. Not used by trustedAuthorNameMatch in api/series.go, because matchGlobalAuthorByName resolves an author across the whole library by name alone.

Table test TestMatchAuthorNameMiddleInitials covers both directions (each pair checked both ways): conflicting and missing initials must not auto match; Tolkien dotted, spaced, run together, all caps, all lower case, sort name and spelled out, plus Jemisin, Klune, Hackwith, Martin, Lewis, Rowling, Le Guin and Maas keep their current band.

#2117: duplicate authors across providers

#2610 already closed the prevention half (#2332: SafeToBind on the CSV, Readarr and Goodreads importers, provider label from the foreign id). Two concrete defects remained:

  1. Re-running an author import duplicates an author bound to another provider. resolveAndCreateAuthor only recognised an existing author by the foreign id the search returned. A row bound to dnb: during an outage before fix(migrate): guard CSV, Readarr and Goodreads author binds against a failed primary (#2332) #2610, or added on a Hardcover primary, was missed once OpenLibrary answered with its own id. Both importers now read the library once per run (libraryAuthors) and skip a row whose name is an exact MatchAuthorName pairing with an existing author, before any provider call. Nothing is merged or relinked; similar but different names are still added. The interactive add flow already had this via findCanonicalAuthorMatch.
  2. Hardcoded openlibrary labels on the remaining fallback paths: fetchAuthorForCreate (three fallbacks), addBookCore when GetAuthor fails, relinkExistingAuthorToUpstream with an unlabelled record, and the ABS manual author mapping. DNB has no author lookup by id at all, so every DNB author added from a search result was labelled OpenLibrary. All now use models.AuthorProviderFromForeignID.

Left open on #2117, hence Refs:

  • Reconciling rows already duplicated. That is a merge and needs the user; Merge on the author page remains the path, and the import now says nothing changed.
  • The Add Book by ISBN author resolution is still unguarded by SafeToBind (noted in fix(migrate): guard CSV, Readarr and Goodreads author binds against a failed primary (#2332) #2610 as its own follow up).
  • author profile refresh: metadata lookup failed ... dnb does not support author lookup by ID still logs at WARN on every refresh of a DNB author. Log noise, not identity.
  • The Goodreads importer still dedupes authors by id only. It attaches books to the author, so a name match there is closer to a merge decision and I left it.

#1691: ABS import mints a twin of an owned book

The retitle case in the report is already fixed on main: provenance by ABS item id, then the abs: key also found as a book identifier after a relink (#2134), written before the relink (#3038). Verified by a probe: retitle and re-import updates the same row.

What remained is the issue's ask 2 and the dangerous tier 3 (owned plus wanted twin). A book Bindery had before the ABS item was ever imported (downloaded itself, or an ABS item that was re-created) is found by neither ABS key, so a differently spelled title created a second row. That row cannot take the file since #2937 (another book tracks it), so it stays wanted beside the owned one. New rung findBookByItemFiles before the title match: the item's ebook file and audiobook folder, after abs.path_remap, looked up in book_files (BookRepo.GetByTrackedPath, live owners only). Guards (review follow up): the book's author must be the item's author (same row, or Exact or FuzzyAuto name), the book must have no owner or the importing owner, and a file matched book keeps its own author and title, on later imports too: migration 106 adds abs_provenance.keep_identity, which the file match sets, the upsert keeps while the link names the same book, and the provenance and abs: key paths honour.

Also: Review duplicates (detection only, user confirms) now reads the filing form, so "Trace of Death, A" groups with "A Trace of Death" (dropArticle).

Left open on #1691, hence Refs:

  • The ingestion key indexer.CanonicalDedupKey still treats "Trace of Death, A" and "A Trace of Death" as different titles, so a catalogue sync can still create such a twin. That key is persisted (books.dedup_key) and backs uniqueness, so changing it needs a migration and a decision about rows that would newly collide. Not attempted.
  • "Chapter House Dune" against "Chapterhouse: Dune" is not a title rule anyone can safely write; the file rung covers it when a file is tracked.

Fail before evidence

Each set of new tests was run against main's code for the files it covers.

#2881 (main's author.go):

--- FAIL: TestMatchAuthorNameMiddleInitials/middle_initial_differs
    MatchAuthorName("J. R. Smith", "J. T. Smith") = FuzzyAuto (weight 5.5), want FuzzyAmbiguous
--- FAIL: .../middle_initial_differs,_issue_example   "A. B. Smith" / "A. C. Smith" = FuzzyAuto
--- FAIL: .../middle_initial_differs_after_a_forename "Philip K. Dick" / "Philip J. Dick" = FuzzyAuto
--- FAIL: .../middle_initial_differs,_long_surname    "Robert B. Parker" / "Robert A. Parker" = FuzzyAuto
--- FAIL: .../middle_initial_differs,_run_together    "JR Smith" / "J. T. Smith" = FuzzyAuto
--- FAIL: .../missing_initial                          "A. Smith" / "A. B. Smith" = FuzzyAuto
--- FAIL: .../missing_initial,_reversed                "J. R. Smith" / "J. Smith" = FuzzyAuto

The same person rows pass on main and after the fix.

#2117 (main's migrate/authors.go, csv.go, readarr.go, api/authors.go, api/add_book_core.go, abs/import_upserts.go):

--- FAIL: TestImportCSVAuthors_ExistingAuthorUnderAnotherProviderIsSkipped (dnb row, hardcover row, same name spelled differently)
    result = {Requested:1 Added:1 Skipped:0 ...}, want the row skipped
    authors = 2, want 1: a second author was created for the same person
--- FAIL: TestImportReadarr_ExistingAuthorUnderAnotherProviderIsSkipped
--- FAIL: TestAddBook_DNBDirectInsertSucceeds
    author metadata_provider = "openlibrary" for dnb:gnd:123120802, want dnb
--- FAIL: TestFetchAuthorForCreate_LabelsProviderFromForeignID (dnb, hc)
--- FAIL: TestCovImpResolveManualAuthor/creates_without_metadata_labels_the_id's_provider
    metadata_provider = "openlibrary" for dnb:gnd:118540238, want dnb

TestImportCSVAuthors_SimilarNameIsNotSkipped is a control and passes on both.

Review follow ups, run against the previous branch head: TestImporter_FileMatchRequiresTheSameAuthor (book moved to a new Frank Herbert row and retitled), TestImporter_FileMatchStaysWithinOneOwner (bob's book linked by an ownerless import), TestImporter_MatchesExistingBookByTrackedFile (title replaced), TestMatchAuthorNameTitleCaseInitialsStayExact ("Jrr Tolkien" FuzzyAuto, "Tj Klune" vs "T. K. Klune" FuzzyAuto), TestHardcoverAuthorScoreAcceptsDroppedInitial (15, want 20) all fail; TestImporter_FileMatchKeepsIdentityOnLaterImports fails on the head before migration 106 (run 2 retitles the book to "Chapter House Dune"); ConfirmedByTitle did not exist.

#1691 (main's abs/import_upserts.go, duplicates/duplicates.go):

--- FAIL: TestImporter_MatchesExistingBookByTrackedFile/ebook and /audiobook
    book 2 "Chapter House Dune" status=wanted
    book 1 "Chapterhouse: Dune" status=imported
    books = 2, want the ABS item matched to the book that already tracks its file
--- FAIL: TestArticleKey/filing-form-a     ArticleKey("Trace of Death, A") = "traceofdeatha"
--- FAIL: TestScanGroupsFilingFormTitles   Scan = 0 groups, want 2

Test plan

  • gofmt -l internal/ clean, go vet on touched packages
  • golangci-lint run on textutil, migrate, api, abs, db, duplicates: 0 issues
  • go test -p 2 -timeout 40m on textutil, duplicates, db, abs, migrate, api: ok
  • go test on the other MatchAuthorName callers (metadata, importer, calibre, recommender, db) after the Author matching treats different middle initials as the same author #2881 commit: ok
  • Docs: docs/Migrating-From-Readarr-Wiki.md (name dedupe), docs/abs_import.md (match ladder), docs/User-Guide-Wiki.md (duplicate rule); changelog fragments for each issue
  • Migration 106 (abs_provenance.keep_identity, reserved for this PR)

🤖 Generated with Claude Code

https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9

vavallee and others added 3 commits October 7, 2026 23:23
)

MatchAuthorName scored "J. R. Smith" against "J. T. Smith" as FuzzyAuto.
The glued initials form "jr" was compared as if it were a spelled out
forename beginning with J, and the whole string Jaro-Winkler fallback read
"kelly l" against "kelly m" as a typo.

Given names are now lined up position by position. Two initials that
disagree after an agreeing forename score nothing (ambiguous with an equal
surname, so a person can confirm), a disagreeing first initial is a
conflict, and a missing initial where both sides are only initials is no
longer enough to auto match.

Matching now decides which words are initials from the name as written: in
a mixed case name only a capitalised short word ("JRR", "KL") is split into
letters, so "Ann Leckie" is no longer Exact with "A. N. N. Leckie". The
public NormalizeAuthorNameWithVariants output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…der id (#2117)

The Readarr import and the author list import recognised an existing
author only by the foreign id the search returned. An author bound to DNB
during an OpenLibrary outage (before #2610), or added on a Hardcover
primary, was missed once OpenLibrary answered with its own id, and a
second author was created for the same person.

Both importers now read the library once per run and skip a row whose
name is an exact MatchAuthorName pairing with an author already there,
whichever provider it is linked to. Nothing is merged or relinked. A
similar but different name is still added.

The remaining hardcoded "openlibrary" author labels are replaced with the
provider the foreign id belongs to: the add author and add book fallbacks
taken when the provider lookup fails (always for DNB, which has no author
lookup by id), the upstream relink with an unlabelled record, and the ABS
manual author mapping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
)

The book match ladder was provenance, then the abs: key, then the
normalized title. A book Bindery had before the ABS item was ever
imported (one it downloaded itself, or one whose ABS item was re-created)
is found by neither key, so a title spelled differently ("Chapter House
Dune" against "Chapterhouse: Dune") created a second row. That row could
not take the file, which another book tracks (#2937), so it stayed wanted
beside the owned one.

A new rung before the title match looks up the item's ebook file and
audiobook folder, after the path remap, in book_files and links the item
to the live book that tracks them. A book owned by another user is not
offered.

Review duplicates also reads the library filing form, so "Trace of Death,
A" groups with "A Trace of Death". Detection only, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@codecov

codecov Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

vavallee and others added 2 commits October 8, 2026 00:44
Review of the file rung found two ways it could reach the wrong book.

The match was not limited to the item's author, and the matched book then
went through applyBookFields, which set AuthorID to the ABS author and
replaced the title. A book by Brian Herbert tracking a file that an item
credited to Frank Herbert also points at was moved to a new Frank Herbert
row and retitled. The rung now needs the book's author to be the item's
author (the same row, or an Exact or FuzzyAuto name), and a file matched
book keeps its own author and title; the other fields follow the usual
rules. Enrichment is given the book's author.

The owner guard only skipped when both owners were set, and authors an
ABS import creates or links have none, so any user's book matched. A book
is now offered only when it has no owner or the importing owner's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
…rm a dropped initial (#2881)

Review follow ups to the middle initials fix.

Title case runs of initials ("Jrr Tolkien", "Ra Salvatore", "Tj Klune",
"Tolkien, Jrr") had dropped from Exact to FuzzyAuto, which broke every
Exact only caller: alias binding, the bulk import skip, the Calibre alias
rule and the sort name dedupe. The exact and swap tiers go back to the
public variant sets, unchanged from main. Only the field scoring uses the
case aware forms, and a short word now reads as initials there when it is
in capitals or has no vowel, so "Jrr Tolkien" still matches the spelled
out name while "Tj Klune" against "T. K. Klune" is no longer an auto match.
"Ann Leckie" against "A. N. N. Leckie" is Exact again, as on main.

#2881 asked that a dropped initial stay acceptable when a title backs it.
AuthorMatchResult gains DroppedInitial and ConfirmedByTitle, which accepts
the ambiguous band only when one side leaves out initials and nothing
disagrees. The callers that have already matched the title or an ASIN use
it: canonical book matching, enrichment candidate pick, series author
agreement, the ASIN metadata map and the Hardcover series author score.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
vavallee marked this pull request as ready for review October 8, 2026 03:44
…1691)

The file match kept the book's own author and title on the first import
only. It records ABS provenance, so the next import found the item through
that link (or the abs: key) and ran applyBookFields, which retitled the
book and moved it to the ABS author.

Migration 106 adds abs_provenance.keep_identity. The file match writes it,
the upsert keeps it while the link names the same book, and an import that
follows a link carrying it keeps the book's author and title. Links made
from the item itself are unchanged.

The #2881 fragment now also names a full first name with dropped initials
("George Martin" against "George R. R. Martin") as matching where the title
agrees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
vavallee enabled auto-merge (squash) October 8, 2026 03:58
@vavallee
vavallee merged commit 09e9ad3 into main Oct 8, 2026
40 checks passed
magrhino added a commit to magrhino/bindery that referenced this pull request Oct 8, 2026
vavallee#3106 skips a row whose author is already in the library by name before
any provider is asked. A rerun after a daily hold matched its committed
authors there, so it never reached the foreign-id branch that queues an
author whose catalogue the hold interrupted, and those catalogues stayed
empty. Both branches now share skipExistingAuthor: a never-populated
catalogue is queued again, a deliberately emptied one stays skipped, and a
failed marker read counts once as an error.

Signed-off-by: magrhino <12738722+magrhino@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Author matching treats different middle initials as the same author

1 participant