Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelog.d/2235-min-edition-count.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
### Added
- **Minimum edition count on metadata profiles** ([#2235](https://github.com/vavallee/bindery/issues/2235)): an opt in **Min editions** floor under Settings, Metadata that keeps thin, one off records out of an author's catalogue when it syncs. A work is judged on the best edition count among the works sharing its title, so one well known record is enough to keep the title. Only OpenLibrary reports edition counts, so the filter only acts on OpenLibrary records: a work with no known count passes, and a Hardcover catalogue is never touched. Books already in your library are never dropped. It is off by default (0). Thanks gchahcg.
7 changes: 5 additions & 2 deletions docs/User-Guide-Wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -671,8 +671,11 @@ When metadata is wrong, you have three levels of fix:
never overwrite them ([guide](Metadata-Editing-Wiki.md)).
2. **Re-bind** the book, or **relink** the author ("Find better match"), to a
different provider record when the match itself is wrong.
3. A **metadata profile** (languages, minimum page count, skip part books)
filters what a catalogue sync lets in.
3. A **metadata profile** (languages, minimum page count, minimum edition
count, skip part books) filters what a catalogue sync lets in. Filling a
series skips every metadata profile filter today, the edition count included
([#2208](https://github.com/vavallee/bindery/issues/2208)), so a filled
series can still bring in thin works.

Box sets need no setting. A work whose title plainly names a bundle ("... Box
Set", "3 Books Set", "Carton of 10 Signed Copies") is dropped from every
Expand Down
72 changes: 70 additions & 2 deletions internal/api/authors.go
Original file line number Diff line number Diff line change
Expand Up @@ -1875,6 +1875,7 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
skipPartBooks := h.resolveSkipPartBooks(ctx, author)
skipMissingDate := h.resolveSkipMissingDate(ctx, author)
minPages, skipMissingISBN := h.resolveEditionFilters(ctx, author)
minEditionCount := h.resolveMinEditionCount(ctx, author)
var languageEvidence map[string]metadata.AuthorWorkLanguageEvidence
if !singleWork {
var languageEvidenceErr error
Expand Down Expand Up @@ -2133,7 +2134,7 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
// the write. Both were invisible, which made Total minus everything else
// look like a hole.
var added, matched, failed, skippedLang, skippedJunk, skippedMediaType, skippedNotAccepted, skippedExcluded int
var skippedPartBooks, skippedMissingDate, skippedMinPages, skippedMissingISBN int
var skippedPartBooks, skippedMissingDate, skippedMinPages, skippedMissingISBN, skippedThinCluster int
// Names of the first few language-rejected works, reported to the user
// alongside the count (#1889): "65 books skipped" is alarming, but it is
// the titles and their language codes that tell them whether the profile
Expand All @@ -2144,6 +2145,20 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
// again for these filters specifically in PR review, vavallee).
var skippedPartBooksSample, skippedMissingDateSample []models.AuthorSyncSkippedBook
var skippedMinPagesSample, skippedMissingISBNSample []models.AuthorSyncSkippedBook
var skippedThinClusterSample []models.AuthorSyncSkippedBook
// clusterEditions maps a normalised title to the highest known edition
// count among the works carrying that title, built as the candidates
// accumulate below. It backs the MinEditionCount filter in the loop that
// follows (#2235): a title whose works all report fewer editions than the
// profile's floor is the noise shape that filter drops. It is only
// populated when the filter is actually on, so a profile without it pays
// nothing. A title absent from the map, or mapped to 0, means "no work in
// the cluster reported an edition count" — unknown, not zero — and the
// filter lets it through, the same unknown-passes semantics MinPages uses.
var clusterEditions map[string]int
if minEditionCount > 0 {
clusterEditions = make(map[string]int, len(books))
}
// candidates accumulates every work that survives the free (in-memory)
// filters below and would otherwise reach the MinPages/SkipMissingISBN
// check. Splitting the loop here lets the edition lookups those two
Expand Down Expand Up @@ -2220,6 +2235,9 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
continue
}

if clusterEditions != nil && b.EditionCount > clusterEditions[normalizedTitle] {
clusterEditions[normalizedTitle] = b.EditionCount
}
candidates = append(candidates, b)
}

Expand Down Expand Up @@ -2317,6 +2335,37 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
continue
}

// Filter works whose title cluster is thinner than the profile's
// MinEditionCount floor (#2235). Edition count only separates real
// works from one-off noise once records are grouped by title, so the
// judgement is made on the cluster's best-known count, not each work's
// own: a title with at least one well-editioned work is a real book
// even if this particular record is thin. The cluster map was built in
// the loop above from works that already survived the junk and
// language filters.
//
// The exemptions mirror the other discovery filters: a book the user
// already owns (existing != nil) is maintained, never dropped, and a
// single-work run is exempt per #1612 — an explicit add of one
// specific work must not be vetoed by catalogue-sync heuristics.
// A cluster with no known edition count (map miss, or 0) is unknown,
// not zero, and passes: only OpenLibrary search results populate
// EditionCount, so a Hardcover-primary author's catalogue — where
// every work reports none — must survive a profile that turned this
// filter on.
if existing == nil && !singleWork && minEditionCount > 0 {
clusterKey := strings.ToLower(strings.TrimSpace(b.Title))
if count, ok := clusterEditions[clusterKey]; ok && count > 0 && count < minEditionCount {
skippedThinCluster++
if len(skippedThinClusterSample) < authorSyncSkippedSampleLimit {
skippedThinClusterSample = append(skippedThinClusterSample, models.AuthorSyncSkippedBook{Title: b.Title})
}
slog.Debug("skipping work from a title cluster below the edition-count floor",
"title", b.Title, "foreignId", b.ForeignID, "clusterEditions", count, "minEditionCount", minEditionCount)
continue
}
}

// MinPages / SkipMissingISBN both need edition data (page count and
// ISBN live on Edition, not Book) fetched in the prefetch pass above.
// A book with no entry in editionsByForeignID is either not gated by
Expand Down Expand Up @@ -2702,6 +2751,8 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
SkippedMinPagesSample: skippedMinPagesSample,
SkippedMissingISBN: skippedMissingISBN,
SkippedMissingISBNSample: skippedMissingISBNSample,
SkippedThinCluster: skippedThinCluster,
SkippedThinClusterSample: skippedThinClusterSample,
AllowedLanguages: allowedLangs,
UnknownLanguageFail: unknownFail,
SkippedLanguageSample: skippedLangSample,
Expand All @@ -2719,6 +2770,7 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
"skipped_part_books", skippedPartBooks,
"skipped_missing_date", skippedMissingDate,
"skipped_min_pages", skippedMinPages, "skipped_missing_isbn", skippedMissingISBN,
"skipped_thin_cluster", skippedThinCluster,
"total", len(books),
}
// The metadata filters dropping works is the surprising case and stays at
Expand All @@ -2730,7 +2782,7 @@ func (h *AuthorHandler) runCatalogueSync(ctx context.Context, author *models.Aut
// filter dropping a work is a setting doing its job, while a create that
// lost its write is the run failing at the thing it exists to do.
if failed+skippedLang+skippedJunk+skippedMediaType+skippedPartBooks+skippedMissingDate+
skippedMinPages+skippedMissingISBN > 0 {
skippedMinPages+skippedMissingISBN+skippedThinCluster > 0 {
slog.Warn("author books synced", logArgs...)
return added, nil
}
Expand Down Expand Up @@ -3509,6 +3561,22 @@ func (h *AuthorHandler) resolveEditionFilters(ctx context.Context, author *model
return p.MinPages, p.SkipMissingISBN
}

// resolveMinEditionCount returns the author's effective metadata profile's
// MinEditionCount floor (#2235). Defaults to 0 (filter disabled) on any
// lookup failure, so an unresolvable profile never turns into unexpected
// catalogue loss.
func (h *AuthorHandler) resolveMinEditionCount(ctx context.Context, author *models.Author) int {
id := models.DefaultMetadataProfileID
if author.MetadataProfileID != nil {
id = *author.MetadataProfileID
}
p, err := h.profiles.GetByID(ctx, id)
if err != nil || p == nil {
return 0
}
return p.MinEditionCount
}

// anyEditionHasISBN reports whether any edition carries an ISBN-13 or
// ISBN-10. Returns false for a nil or empty slice — a work with no editions
// to check has no ISBN to confirm.
Expand Down
Loading
Loading