Skip to content

feat(metadata): BnF as an opt-in primary provider (#3144) - #3150

Merged
vavallee merged 14 commits into
mainfrom
feat/3144-bnf-provider
Oct 12, 2026
Merged

feat(metadata): BnF as an opt-in primary provider (#3144)#3150
vavallee merged 14 commits into
mainfrom
feat/3144-bnf-provider

Conversation

@vavallee

@vavallee vavallee commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

Adds the Bibliothèque nationale de France as an opt-in primary metadata provider, modelled on DNB and Nasjonalbiblioteket, so a French library gets French edition titles ("Alice au pays des merveilles") instead of the original English titles OpenLibrary and Hardcover carry. Requested by @mnivoliez.

Closes #3144.

Live behaviour was exercised with about 22 requests to BnF (18 by curl while choosing the schema, then one pass of the finished client: GetAuthor, SearchBooks with its name lookups, GetBookByISBN), plus 2 during the review fixes (NFC against NFD). It has not been tested by a human in the app. The author catalogue walk was checked live query by query (author2bib, the creation date name query), not end to end.

Record schema: UNIMARC, not Dublin Core

From Explain and real searchRetrieve responses in both schemas (title+author, bib.fuzzyIsbn with an ISBN-10, ARK lookup, author2bib, authority records):

Need dublincore unimarcXchange (chosen)
Title Alice au pays des merveilles / Lewis Carroll ; édité par Edmond Bronstein, edition and responsibility mixed in 200 $a alone (older converted records still need trimming, handled)
Author vs translator text role in at least three spellings: (Auteur du texte), . Auteur du texte, Auteur du texte / Autrice du texte relator code $4 (070 author, 730 translator, 100 adapted from, 330 dubious), or text $c on 2026 records
Identity none $3 authority record number, ISNI in $o
Original title none 454 $t, or the 500 uniform title
Language, ISBN, publisher, year, pages yes, as text 101, 010/073, 214/210, 215
Summary source not stated 330 $2 (éditeur, Electre), so third party text can be skipped

What the June 2026 change ("Noemi") looks like today: a record created 2026-06-01 has a mostly blank leader, puts both author and translator in 702 with the role as text ($c Auteur du texte, $c Traducteur) and no $3 link, and links a French uniform title (500 $3). A query proved such a record is not returned by bib.author2bib for its author. The parser accepts both shapes, every field is optional (missing means empty, a garbage or truncated body is an error, never a panic), zero records is an empty result, and an SRU diagnostic (BnF sends them with HTTP 200) is an error. If BnF changes the format again, unknown fields are ignored; the risk is fewer fields filled, not a crash.

What it does
Search books bib.anywhere all "<term>" and bib.doctype any "a", 30 records, folded into works
Search authors person authority records, aut.accesspoint all "<term>" and aut.type any "pep", kept only when the heading itself matches every word (Lewis Carroll yes, Johnny Carroll via his variant "Johnny Lewis" no)
Author bnf:author:<authority record number>; variants become AlternateNames (filtered by LatinAliasBinds as for NB)
Author catalogue bib.author2bib pages (10 x 100 max), plus a name query for records created since 2026-06-01 that BnF has not linked yet; never reported Complete
Book bnf:<ARK name> (e.g. bnf:cb48780493c), the earliest entered French edition, stable across new printings; GetBook adds the work's other editions with one more search
ISBN bib.fuzzyIsbn (ISBN-10, ISBN-13 and EAN all match, verified live)
Unlinked authors linked by exact name from another result first, then by authority lookup when exactly one person has that heading (2 lookups per search at most, a lookup matching more than the 20 records it reads is inconclusive); otherwise left unlinked, never guessed. The catalogue walk adopts an unlinked record found by name only after one lookup shows no namesake
Elision L'outsider kept as is in queries and titles; L’outsider and L'outsider group; "Jean d'" + "Ormesson" displays as "Jean d'Ormesson"
Language 101 $a through NormalizeLanguageCode
Original title Book.OriginalTitle, which the search title code already uses
Series not parsed (225/410 are publisher collections, not series)

Licence requirements

Requirement Where it is met
1a. Docs section with sources, review date, licence, credit line, redistribution note docs/third-party-data.md section "Bibliothèque nationale de France", Reviewed 2026-10-11; sources include the catalogue CGU, BnF's reuse conditions page, the data.bnf.fr licence page and the Etalab text. The licence condition is stated as applying to every reuse, a self-hosted display included, met by the credit plus the date
1b. Visible credit where the provider is chosen en.json metadataProviderHint and metadataProviderBnfCredit; MetadataTab.tsx shows the credit when BnF is selected and says where the date is shown; test in MetadataTab.primaryProvider.test.tsx
1c. Source record identifiable, retrieval date shown book foreign ID is the ARK (records.go recordARK), linked to catalogue.bnf.fr/ark:/12148/... from the Links menu (metadataSource.ts). The date is now displayed: a BnF book's Metadata source row and a BnF author's header read "Data from the BnF, Licence Ouverte / Open Licence 2.0, retrieved " (common.bnfSourceCredit), the date being lastMetadataRefreshAt, else createdAt, both already in the API response; no new column or migration. Tests in BookDetailPage.test.tsx, AuthorDetailPage.test.tsx, metadataSource.test.ts. A book's date is when it was added or rebound: an author refresh stamps the author, not each book
1d. README and User Guide README.md line 158; docs/User-Guide-Wiki.md lines 787 to 803
2. No endorsement "BnF does not endorse Bindery" in the credit, the docs section and testdata/SOURCE.md
3. No misleading source or date docs section bullet "No endorsement, nothing misattributed"; dates are Bindery's own fetch times
4. No Dewey authorities, no third party music records only aut.type any "pep" is ever queried, no 676 or subject field read; bookFilter (cql.go) on every bibliographic search and isBookRecord (records.go) drops sound recordings, music, serials and integrating resources (leader position 7 s, i), periodical issues and authority records; tests TestParseRecord_DropsWhatIsNotABook, TestGetAuthorWorksSnapshot
5. No covers 856 is never read, no cover provider implemented; tests assert empty ImageURL; docs say so. Every 856 field is now stripped from the redistributed fixtures (testdata/SOURCE.md records it)
6. Summaries recordDescription (records.go) keeps only unsourced or éditeur summaries, skips Electre and any other third party; stored as the description only; fixtures have the summary text replaced (testdata/SOURCE.md). Keeping a 330 with no $2 is a judgment call, stated as such in the docs
7. Personal data docs section bullet "Names are personal data"
8. No new dependency encoding/xml and the standard library only; go.mod, go.sum and package files unchanged
9. Opt in, zero traffic otherwise cmd/bindery/metadata_providers.go primaryMetadataProvider, constructed only when primary, no enricher, no startup probe; TestPrimaryMetadataProvider
10. No copied code written from the SRU Explain response and the records themselves

The fixtures are real BnF responses (retrieved 2026-10-11), which the repo now redistributes, so internal/metadata/bnf/testdata/SOURCE.md carries the credit line and lists the two changes made (summaries replaced, 856 fields removed).

Rate limit design

Guard Value Where Test
Start spacing 1 request a second, no burst pacer.go line 30 TestPacer_SpacesRequestStarts
Concurrency 2 in flight, slot held until the body is closed pacer.go line 31 TestPacer_CapsConcurrentRequests
Concurrency slot release held until the response body is closed pacer.go releasingBody TestPacer_SlotHeldUntilBodyClosed
Shared one pacer and one refusal gate for every client in the process, compared with the package defaults (a nil gate fails) New() TestNewClientsShareOnePacerAndGate
Deadline a slot past the caller's deadline is refused at once as ErrRateLimited (as Hardcover's pacer does) pacer.go, classify TestPacer_RefusesPastTheDeadlineAsRateLimited
Retry-After / backoff shared providerhttp.Do: 429, 502, 503, 504 retried 3 times (4 requests), Retry-After honoured up to 30 s, jittered exponential backoff otherwise, the gate holds every BnF request during a refusal (#3100); other 4xx not retried client.go get TestRetryAfterIsHonoured, TestPersistent429IsRateLimited, TestOther4xxIsNotRetriedOrMarked
Error marking 429 and pacing refusal: ErrRateLimited; 5xx and network failures: ErrUnavailable classify TestServerErrorsAreUnavailable, TestNetworkFailureIsUnavailable
Timeout 20 s per request, pacing wait included client.go line 85 TestHungServerHitsTheTimeout
Body cap 4 MiB, cut off body fails to decode client.go maxResponseBytes TestOversizedBodyIsCutOff
XML nesting refused past 64 levels, before the decoder's element stack grows (a 4 MiB nest of <a> allocated about 256 MiB before) client.go decodeXML, depthLimiter TestDeeplyNestedBodyIsRefused (measures allocation)
Page caps 10 pages of 100 for a catalogue, 2 for the name query, 2 name lookups per search, 1 namesake lookup per catalogue walk (only when there is something to adopt) client.go constants TestPageCapStopsARunawayCursor, TestLinkUnlinked_BoundedLookups, TestAdoptUnlinked_NamesakeCheck
CQL user text is composed to NFC, then becomes one quoted term with " \ * ? ^ ( ) < > = / and control characters replaced by spaces, capped at 200 runes cql.go TestCQLTerm (includes x" or bib.title any "*, NUL, DEL, C1 markers, NFD input)

Why remove rather than escape: sent bib.title all "x\" or bib.title any \"*", BnF echoed the query as bib.title all "x" or bib.title any "*", so its handling of a backslash escape could not be confirmed. Word searches lose nothing when punctuation goes.

Why 1 a second: BnF publishes nothing and every install is an independent client. A search is 1 to 3 requests and fits the aggregator's 8 s budget; opening or adding one book is up to 4; an author refresh is up to 15 requests (the author's record, then the authority record, up to 10 catalogue pages, up to 2 pages of unlinked records, 1 namesake check), around 15 s. With MinPages or SkipMissingISBN set, refresh used to call GetEditions per new work, which for BnF is a full GetBook (up to 4 requests each, so 100 to 200 requests and about 3 minutes for Lewis Carroll); BnF works are now judged on the editions the catalogue already returned, with no extra request.

Mutation checks

Each guard disabled on a compiling tree, the named test run:

Guard disabled Result
pacer bypassed caught: SpacesRequestStarts, CapsConcurrentRequests, RefusesPastTheDeadline
deadline refusal caught
page cap tripled caught: PageCapStopsARunawayCursor
CQL character removal caught: TestCQLTerm
doctype filter emptied caught: FrenchEditionTitle..., GetBookByISBN
record type filter caught: DropsWhatIsNotABook
transport errors not marked unavailable caught: NetworkFailure, HungServer
pacing refusal not marked rate limited caught
body cap removed caught: OversizedBodyIsCutOff
author role ignored caught: Credits_Roles
third party summary kept caught: ThirdPartySummaryIsDropped, GetBookByISBN
elision join caught
control characters kept in titles caught
name linking skipped, ambiguous heading linked, lookups unbounded caught: LinkUnlinked, LinkUnlinked_BoundedLookups
snapshot reported complete caught
reconciliation bnf skip removed caught: NBUsesTheWorksOwnEditions (now runs nb and bnf)
bnf: prefix removed from models caught: models and classifier tests
bnf removed from accepted settings caught: settings and boot resolution tests
After review: serial level check removed caught: DropsWhatIsNotABook
After review: ", by" cut without the name check; "sic" kept as original; leading [sic] dropped without the name check; whole word match loosened caught: SplitTitleStatement_ResponsibilityAfterComma
After review: adoption without sameName; without the namesake lookup; inconclusive lookup accepted caught: AdoptUnlinked_NamesakeCheck
After review: gate: defaultGate removed from New(); per client pacer caught: NewClientsShareOnePacerAndGate
After review: French preference; workOf; control character replacement; slot released at headers; case "bnf" caught: FrenchEditionRepresentsTheWork, GetBook_AddsTheWorksOtherEditions, CQLTerm, Pacer_SlotHeldUntilBodyClosed, PrimaryMetadataProvider
After review: nesting limit lifted caught: DeeplyNestedBodyIsRefused (426 MiB)
After review: BnF skip in the refresh prefetch removed caught: FetchAuthorBooks_BnFWorksJudgedOnTheirOwnEditions
After review (web): bnf in AddToLibraryModal, AuthorMetadataLinkModal label, authorProviderKey, the book and author credit caught: the new tests in each file

Touchpoints compared with the NB PR (#2995)

All of #2995's touchpoints are covered: provider package, main.go, settings_handler.go, catalogue_reconciliation.go (+ test), models/author.go and book.go (+ tests incl. book_identity_test.go), MetadataTab.tsx, metadataSource.ts (+ test), AddToLibraryModal.tsx, AuthorMetadataLinkModal.tsx, authorMetadata.ts, en.json, README, ARCHITECTURE, User Guide, third-party-data, changelog fragment. Also touched: aggregator_foreign_id_classifier_test.go (routing and SafeToBind for bnf:), cmd/bindery/metadata_providers_test.go, settings_handler_test.go, internal/calibre/identifiers.go, and a new MetadataTab.primaryProvider test. Not needed: a series file (no series parsing). docs/API.md lists no provider values, so it is unchanged.

Not in this PR

  • BnF as an enricher supplying a French search title for books from other providers. Not small: it would need a stored per book search title or alias fed into indexer search, and BnF wired as an enricher, which breaks the zero traffic opt in unless gated by a new setting. Follow up.
  • Covers: none from BnF, data.bnf.fr or Gallica, by design.
  • Summaries are not claimed as open data; third party ones are skipped.
  • Catalogue completeness: while BnF's migration runs (to the end of 2026) records created since June 2026 are only found by name, so reconciliation treats a BnF catalogue as partial and never offers removals from it.
  • Audiobooks and e-book only editions outside doctype "a" are not read.
  • No live test is committed (the fixtures are real responses).
  • Representative ID drift. A work's ID is its earliest entered French edition among the records fetched, so a book added from a 30 record search (for example bnf:cb35611382j per the review) can carry a different ID from the same work in the catalogue walk (bnf:cb37615303n). Not reproduced live here. Author refresh does not duplicate it: the title index (seenTitles) matches it to the existing row. What it does not do: the ID lookup misses, so the work counts as new for the MinPages/SkipMissingISBN filters, and a work that fails one is reported as skipped instead of updated. Matching on edition IDs would fix that for every provider and is left for a follow up.
  • A book's credit date is when it was added or rebound; an author refresh stamps the author, not each book, so a book's date does not move when a refresh reads it again.
  • Search time linking on common names: a name lookup that matches more than the 20 person records it reads is treated as inconclusive, so an unlinked "Martin, Jean" stays unlinked rather than risk a namesake.

Changes after independent review

Each defect was reproduced by a test that failed before its fix; guards were then disabled on a compiling tree to show the test catches them.

# Finding Result
1 Serial records passed as books Fixed: leader position 7 s and i rejected; hand built serial (cas) and integrating resource records fail before, pass after
2 English statements of responsibility and ["sic"] Fixed: , by / , par cut only when what follows names one of the record's people or is a plain cue ("the author of"); "Ruby, by Moonlight" kept; a leading name flagged [sic] dropped when it misspells one of the record's people; bracketed sic never an original title; 12 cases including the four real titles
3 adoptUnlinked had no namesake check Fixed: one authority lookup before adopting, nothing adopted when another person has the heading or the lookup is inconclusive; dropping sameName now fails
4 Gate test passed with two nil gates Fixed: compared with defaultGate and defaultPacer; removing the gate now fails
5 Surviving mutations Fixed: French representative, workOf, control characters (NUL, SOH, DEL, C1; vertical tab, form feed, NEL and U+2028 are whitespace to strings.Fields so they cannot exercise the replacement and are kept only as a guard), case "bnf" (switch moved to a tested function), pacer slot held until body close, and the three web survivors
6 No NFC Fixed with tests. Live check (2 requests): BnF folds both forms itself, NFC and NFD each matched 638 records, so it changes no results today. No go.mod or go.sum change
7 Author refresh cost Fixed: BnF works judged on their attached editions in the MinPages/SkipMissingISBN prefetch, as reconciliation does; same verdicts, no GetEditions call
8 XML nesting, title folding, BOM, representative ID Fixed: nesting limit (allocation measured), titles folded with textutil.FoldForSearch, dead BOM skip removed (encoding/xml skips it; the BOM test passes without it). Representative ID drift confirmed by reading, reconciled by title, documented above
9 No date displayed Fixed: dated credit on BnF book and author pages from existing fields, one new en.json string
10 Docs Fixed: licence condition applies to all reuse, sources added, unsourced summary judgment stated, request costs, 856 stripped from fixtures
11 PR body This update

Checklist

  • Tests added or updated
  • Doc-update gate cleared (docs/, README.md, godoc)
  • Wiki pages updated (User Guide)

Test plan

  • go build ./..., go vet on touched packages
  • golangci-lint run --concurrency 2 on bnf, metadata, models, calibre, api, cmd/bindery: 0 issues
  • go test -p 2 ./internal/metadata/... ./internal/models/ ./internal/calibre/ ./cmd/bindery/ ./internal/api/
  • go test -race ./internal/metadata/bnf/ (count 3); after review -count=20 and -race -count=5
  • cd web && npm run typecheck, eslint on touched files (one existing warning in MetadataTab.tsx line 76, unchanged)
  • npx vitest run MetadataTab, metadataSource, authorMetadata, AddToLibraryModal, AuthorMetadataLinkModal; after review also BookDetailPage, AuthorDetailPage and the settings pages
  • Full Go suite and full vitest (CI)
  • Manual run in the app with BnF as primary

🤖 Generated with Claude Code

https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9

vavallee and others added 4 commits October 11, 2026 14:08
@vavallee @claude
feat(metadata): BnF SRU client (#3144)
0f3c843 
A read-only client for the Bibliothèque nationale de France catalogue
général through its public SRU 1.2 service, asking for UNIMARC
(unimarcXchange). Dublin Core was the other candidate: it folds the
statement of responsibility into the title, gives the role as text in at
least three spellings, and carries no authority links, so it cannot tell
the author from the translator reliably.

- Books are works folded from edition records: editions sharing a title,
  or a translation's original title (454, a bracketed original, or the
  uniform title), group together per author. The work is named after its
  French edition and its ID is the ARK of the earliest entered French
  record, so it is stable across new printings.
- Authors are BnF person authority records (bnf:author:<record number>).
  Only relator 070, or the 2026 text role "Auteur du texte", is the
  author; translators, illustrators and adapted-from authors never are.
  Records BnF has not linked to an authority yet are linked by exact name
  when one person has that heading.
- Only printed text records are read (bib.doctype "a" in the query, and
  the record's own type checked again), so music and sound recordings never
  get through. No covers, no Dewey or subject authorities.
- An author's catalogue is never reported complete: records created since
  BnF's June 2026 model change are not linked to their author yet, and are
  found by a bounded name query.

Politeness: one request a second, two in flight, for the whole process,
through a pacer in the HTTP transport so retries are paced too. The shared
provider loop honours Retry-After and holds the provider on a refusal; a
429 or a local pacing refusal surfaces as ErrRateLimited, a 5xx or network
failure as ErrUnavailable. 20 s per request, 4 MiB per body, at most 10
pages for a catalogue and 2 for the name query, 2 name lookups per search.
User text is reduced to one quoted CQL term with the special characters
removed rather than escaped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
feat(metadata): BnF as an opt-in primary provider (#3144)
f915649 
metadata.primary_provider accepts "bnf". Like Nasjonalbiblioteket it is
wired only when chosen, never as an enricher, so an install that did not
choose it sends BnF nothing.

The "bnf:" prefix is known to the foreign ID classifiers. Before, it fell
through to openlibrary: with OpenLibrary primary a bnf: author would have
been routed to the OpenLibrary client, and SafeToBind would have taken a
BnF match for the primary's after an OpenLibrary failure.

Catalogue reconciliation skips the per work editions lookup for bnf works,
as for dnb and nb: each one is a full GetBook, at one request a second.
Calibre identifiers carry the bare ARK.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
feat(web): BnF in the primary provider setting, with its credit (#3144)
68a168a 
The primary provider select offers BnF. The Licence Ouverte asks for
credit to the source and the retrieval date, so the setting's description
names it, and choosing BnF shows the credit line and that BnF does not
endorse Bindery. BnF book IDs link to their catalogue record by ARK in the
Links menu; authors have no link, as their ARK needs a check character
Bindery does not store. Only en.json gets the new strings.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
docs: BnF provider, its licence terms and the changelog fragment (#3144)
eb19669 
docs/third-party-data.md gains a Bibliothèque nationale de France
section: the Licence Ouverte 2.0 and its credit line, where the retrieval
date lives, no endorsement, the CGU carve outs (music records, Dewey
authorities) and how Bindery keeps clear of them, no covers, summaries
not claimed as open data, names as personal data, and the pacing. README,
ARCHITECTURE and the User Guide list BnF as an opt-in primary.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@codecov

codecov Bot commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.24764% with 112 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/metadata/bnf/client.go 78.93% 43 Missing and 28 partials ⚠️
internal/metadata/bnf/records.go 91.83% 13 Missing and 11 partials ⚠️
internal/metadata/bnf/works.go 95.76% 6 Missing and 2 partials ⚠️
internal/metadata/bnf/pacer.go 88.63% 4 Missing and 1 partial ⚠️
internal/metadata/bnf/types.go 95.23% 1 Missing and 1 partial ⚠️
web/src/components/AddToLibraryModal.tsx 0.00% 0 Missing and 1 partial ⚠️
web/src/components/AuthorMetadataLinkModal.tsx 0.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

vavallee and others added 10 commits October 11, 2026 14:38
@vavallee @claude
fix(metadata): BnF drops serials and cuts English responsibility stat…
c54a5b2 
…ements (#3144)

bib.doctype "a" includes periodicals, and a serial title record carries no
461 link, so a search for "Le Monde diplomatique" read ten serial records
as books. Leader position 7 "s" (serial) and "i" (integrating resource)
are now rejected; monographs, collections, component parts and the blank
level of a 2026 record are kept.

Title statements such as "Sylvie and Bruno, by Lewis Carroll" kept their
statement of responsibility, and `L. Carrol ["sic"]. Alice au pays des
merveilles` became the title "L. Carrol" with the original title "sic".
A ", by" or ", par" with no space before the comma is now cut only when
what follows names one of the record's people or is a plain cue ("the
author of"), so "Ruby, by Moonlight" keeps its comma. A leading name
flagged [sic] is dropped when it is a misspelling of one of the record's
people, and a bracketed sic is never an original title.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(metadata): BnF adopts an unlinked record only when no namesake ex…
48bdd44 
…ists (#3144)

The catalogue walk's name query adopted every record crediting, without an
authority link, an author named exactly like the heading. Two people named
"Martin, Jean" are two authority records, so the other one's new books
joined this author's catalogue. Adoption now makes one authority lookup,
only when there is something to adopt, and adopts nothing unless this
person is the only one with that heading.

Both that check and the search time linking now read one page of 20 and
treat a lookup that matched more records than it read as inconclusive:
the second namesake could be on the page not read. Before, linking read
5 records and could miss it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
test(metadata): cover BnF guards that mutations survived (#3144)
beec362 
Each test below fails with its guard disabled on a compiling tree:

- the shared refusal gate and pacer are compared with the package
  defaults, so a New() without the gate (two nil gates are equal) fails
- the French edition represents a work even when another language's
  edition was entered earlier and comes first
- GetBook adds the work's other editions (workOf)
- cqlTerm replaces control characters strings.Fields does not split on
  (NUL, SOH, DEL, the C1 markers)
- the pacer's concurrency slot is held until the response body is closed

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
test(cmd): wire the primary metadata provider through a tested functi…
d7a897b 
…on (#3144)

The provider switch sat inline in main(), so dropping its "bnf" case left
every test green while a BnF setting booted OpenLibrary. It moves to
primaryMetadataProvider, unchanged, with a test that each accepted value
builds that provider.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(metadata): compose BnF query terms to NFC (#3144)
014167c 
cqlTerm passed decomposed input through, so "Les Misérables" was sent
as e plus a combining accent and counted the accent against the 200 rune
cap. It is composed first now. golang.org/x/text is already a direct
dependency; go.mod and go.sum are unchanged.

Checked live with one request each: BnF folds both forms itself (both
queries were echoed as "Les Miserables" and matched 638 records), so this
changes no results today. It keeps the query in the form BnF's records use
and the cap counting letters.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(metadata): bound BnF XML nesting and fold accents in work grouping (
708bd94 
#3144)

A 4 MiB body of nested elements grew encoding/xml's element stack to
about 256 MiB before failing. Decoding now goes through a token reader
that refuses nesting past 64 levels (a real SRU response nests about
eight), and the test measures the allocation.

The byte order mark skip is removed: encoding/xml already skips a leading
BOM, and the BOM test passes without it.

Work grouping keyed titles on CanonicalDedupKey alone, which keeps accents
and ligatures, so "Les Misérables" and "les miserables", or "Œuvres" and
"Oeuvres", became two works. The key now also goes through
textutil.FoldForSearch, the fold the search keys use.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
fix(api): judge BnF works on their own editions during author refresh (…
891e08f 
…#3144)

With MinPages or SkipMissingISBN set, author refresh called GetEditions
for every new work. For BnF that is a full GetBook: the ARK lookup, up to
two name lookups and the work search, at one request a second, so a
refresh of Lewis Carroll (about 49 works) cost 100 to 200 requests and
around three minutes.

BnF works arrive from the author catalogue with every edition attached,
the same records the lookup would fetch again, so the prefetch uses them
as catalogue reconciliation already does. A work with none attached still
goes to the lookup. The filters reach the same verdicts; the test checks
both filters and that no lookup is made.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
feat(web): show the BnF credit with its retrieval date (#3144)
d062e97 
The Licence Ouverte / Open Licence 2.0 that BnF's records are under asks
for the source and the date, and no date was shown anywhere. A BnF book's
Metadata source row and a BnF author's header now read "Data from the
BnF, Licence Ouverte / Open Licence 2.0, retrieved <date>". The date is
the row's own: lastMetadataRefreshAt, else createdAt, both already in the
API response; no column or migration. The Settings credit now says where
the date is shown.

Also covered by tests now: the BnF label in the link metadata dialog,
authorProviderKey for bnf: ids, and bnf as a primary the add dialog
honours. Each fails with its line removed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
docs: BnF licence conditions, sources, cost and fixtures after review (…
bd94ee2 
…#3144)

- third-party-data.md: the Licence Ouverte condition applies to every
  reuse, a self-hosted display included, and is met by the credit and the
  date together; says where each is shown and which stored date it is.
  Adds BnF's reuse conditions page and the data.bnf.fr licence page to the
  sources, states that an unsourced 330 summary is kept as a judgment call,
  and gives the request cost of a search, a book and an author refresh.
- Every 856 field is removed from the redistributed fixtures (cover image
  references and links to other catalogues): nothing reads 856 and Bindery
  fetches no BnF image. SOURCE.md records the change; the tests pass on the
  stripped fixtures.
- User Guide and the changelog fragment mention the dated credit on book
  and author pages and the namesake check.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee @claude
Merge main into feat/3144-bnf-provider
e1dcfe4 
No conflicts. Build, vet and the metadata, models, calibre and cmd tests
and the web typecheck and component tests pass on the merged tree.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
@vavallee
vavallee merged commit ad01e3d into main Oct 12, 2026
41 checks passed
@vavallee vavallee mentioned this pull request Oct 12, 2026
Merged
12 of 15 tasks
vavallee added a commit that referenced this pull request Oct 12, 2026
@vavallee @claude
docs: bring documentation up to date with what merged since v1.41.0
4c7d7ca 
Provider lists in the README, the roadmap Delivered and Next up sections,
the In Library meaning for partly owned dual format books, ratings, the
Authors list have / total count and the author page sort and filters, the
series field in the book edit table, the availableBookCount and rating
fields, the rating sort values and the primary provider values in the API
reference, and the one time dedup key recompute on first boot.

Documents #3125, #3095, #3088, #3134, #3090, #3127, #3150, #3158, #3116,
#2915 and #2994. No new environment variables, dependencies or breaking
behaviour shipped, so DEPLOYMENT.md env tables, THIRD_PARTY_LICENSES.md and
docs/upgrade-v1.md are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
vavallee added a commit that referenced this pull request Oct 12, 2026
@vavallee @claude
docs: bring documentation up to date with what merged since v1.41.0
dea7903 
Provider lists in the README, the roadmap Delivered and Next up sections,
the In Library meaning for partly owned dual format books, ratings, the
Authors list have / total count and the author page sort and filters, the
series field in the book edit table, the availableBookCount and rating
fields, the rating sort values and the primary provider values in the API
reference, and the one time dedup key recompute on first boot.

Documents #3125, #3095, #3088, #3134, #3090, #3127, #3150, #3158, #3116,
#2915 and #2994. No new environment variables, dependencies or breaking
behaviour shipped, so DEPLOYMENT.md env tables, THIRD_PARTY_LICENSES.md and
docs/upgrade-v1.md are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fJcCVbNnKsmj2MAAMwWj9
Signed-off-by: vavallee <vavallee@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

BnF (Bibliothèque nationale de France) metadata provider, promotable to primary, for French-language libraries

1 participant

@vavallee