fix(importer): reject a download whose file is in a language the profile does not allow (#2998)

changelog.d/2998-reject-disallowed-language.md

@@ -0,0 +1,6 @@
+### Changed
+- **A download in a language your profile does not allow is now rejected instead of relabelling the book** (#2998). When a release name did not say its language, a foreign edition could slip past the search filter, and at import Bindery read the EPUB's language and quietly switched the book to it, so an English only library filled up with Swedish and Dutch books that looked imported and were never searched again. Now, when the author's metadata profile lists the languages it allows and the downloaded EPUB declares one outside that list, nothing is imported: the Queue row says which language the file declares and which the profile allows, the release is blocklisted so the next search picks another, and the book stays Wanted. Files that declare no language, or an allowed one in any spelling (en, en-US, eng), import as before, an EPUB that declares several languages is allowed when any of them is, a release with both an allowed and a disallowed EPUB imports the allowed one, and profiles that allow any language keep the old relabelling. Manual import and Match to book are never refused, and books already in your library are not touched. Thanks to foobarbigtime for the report.
+
+### Fixed
+- **Norwegian Bokmål and Nynorsk count as Norwegian** (#2998). Books and files tagged nb, nob, nn or nno no longer fail a profile that allows Norwegian. Thanks francisrath.
+- **Every two letter language code is recognised** (#2998). Books and files tagged with codes such as uk, he or sk were not recognised as Ukrainian, Hebrew or Slovak, so a Ukrainian EPUB could import under an English only profile. Chinese tagged cmn or yue now counts as Chinese.

cmd/bindery/main.go

@@ -406,6 +406,10 @@ func main() {
 	// before importing it, and blocklist a release rejected that way so the
 	// next search does not grab it again (#1782).
 	importScanner.WithFormatEnforcement(qualityProfileRepo, blocklistRepo)
+	// Same for the file's declared language against the author's metadata
+	// profile: a downloaded EPUB in a language the profile does not allow is
+	// rejected and blocklisted instead of relabelling the book (#2998).
+	importScanner.WithLanguageEnforcement(metadataProfileRepo, blocklistRepo)
 	importScanner.WithJobs(bgJobs) // drain manual scans on shutdown (#1458)
 
 	// Grimmory push pipeline (#826). Config is loaded live per push, so the

docs/Troubleshooting-Wiki.md

@@ -54,7 +54,7 @@ If your platform can't run a different command, set `BINDERY_DB_FK_CHECK=report`
 
 ## A grabbed book never imports
 
-The book shows **grabbed** in History and reaches 100% in your download client, but Bindery never imports it. Usually one of two causes.
+The book shows **grabbed** in History and reaches 100% in your download client, but Bindery never imports it. Usually one of the causes below.
 
 ### qBittorrent 5.x with an old Bindery
 
@@ -64,6 +64,12 @@ You may see `add torrent failed: {"added_torrent_ids":...}` or `failed to send t
 
 **Fix:** upgrade to Bindery **1.12.1 or later** (the current release is recommended). Bindery 1.11.0 and earlier cannot complete torrent grabs against qBittorrent 5.x at all — it is a hard incompatibility. After upgrading, re-grab anything that was stuck.
 
+### The file is in a language your profile does not allow
+
+The Queue row is **importBlocked** with a message like `file declares Swedish (swe), but the metadata profile "Standard" allows only English (eng)`. The release name did not say its language, so it passed the search filter, but the EPUB inside declares one the author's metadata profile does not allow. Bindery placed nothing, added the release to the Blocklist and left the book Wanted, so the next search picks a different release on its own. See [the file's language is checked at import](User-Guide-Wiki.md#from-wanted-to-your-library) for the full rule.
+
+If the file is what you want after all, press **Match to book** on the row and pick the book, or import it through **Import → From a folder**; a manual import is never refused for its language. To accept more languages for this author from now on, add them to the metadata profile, or set it to any language. Releases already blocklisted for this stay there until you remove them in **Settings → Blocklist**.
+
 ### Bindery cannot read the completed files
 
 **Start with Diagnose.** In **Settings → Download clients**, press **Diagnose** on the client. It works out where Bindery's grabs actually land for ebooks and for audiobooks (the save path Bindery sends, the category's folder, or the client default, and it says which), applies the path remap the same way the importer does, checks that Bindery can read and write the result, and tries a hardlink into each library folder. The panel puts the first thing to fix at the top and shows the paths side by side: where the client puts the files, which remap applied, and where Bindery looks. **Copy report** copies the check results and paths without the host, port or username, so it is safe to paste into an issue or Discord.

docs/User-Guide-Wiki.md

@@ -408,6 +408,49 @@ a CWA ingest folder, Grimmory's BookDrop, an Audiobookshelf library scan,
 webhooks. The three ways of reaching Calibre or CWA are easy to mix up; the
 [Calibre integration guide](Calibre-Integration-Wiki.md) tells them apart.
 
+**The file's language is checked at import.** The language filter at grab
+time can only read the release name, so a release that does not say its
+language gets through it. When the download arrives, Bindery reads the
+language the EPUB declares (its `dc:language`) and compares it with the
+languages the book may be in: the allowed languages on the author's metadata
+profile, or, when that profile allows any language, the preferred search
+language if it is set to English. That is the same rule the release name was
+held to, so a file is never judged more strictly than its name would have
+been.
+
+- **The file is in an allowed language**, declares no language, or declares
+  one that is not a specific language (`und`, `mul`): it imports as normal.
+  Every language an EPUB declares counts, so a bilingual edition tagged
+  French and English is allowed under an English profile. Region tags are
+  ignored, so `en-US` and `en_GB` count as English, Bokmål and Nynorsk (`nb`,
+  `nob`, `nn`, `nno`) count as Norwegian, and `cmn` and `yue` count as
+  Chinese. Every two letter ISO 639-1 code is recognised.
+- **The release holds EPUBs in both allowed and disallowed languages**: the
+  allowed ones are imported and the others are left out, the same way a
+  disallowed format inside a release is left out.
+- **Every file is in a language the profile does not allow**: the release is
+  treated as the wrong release. Nothing is placed, the Queue row is blocked
+  with a message naming both languages ("file declares Swedish (swe), but the
+  metadata profile "Standard" allows only English (eng)"), the release is
+  added to the Blocklist so the next search picks a different one, and the
+  book stays Wanted. The failure is in History and is sent to your webhooks
+  like any other failed import.
+- **The profile allows any language** (and the preferred search language is
+  not English): nothing is rejected. When the file's language differs from
+  the book's, the book is relabelled to the file's language and History
+  records a "language corrected" row, because the file on disk is the edition
+  you actually have.
+
+To keep a rejected file anyway, use **Match to book** on the Queue row, or
+import it through **Import → From a folder**. A manual import is your own
+choice and is never refused for its language. If you locked a book's language
+by editing it, a file in that language is accepted for that book even when the
+profile does not allow it. Only downloads are checked: a library scan,
+adopting files already in your library, and books imported before this check
+existed are left alone. With the import mode set to **External**, Bindery
+hands the download to your other tool without opening its files, so nothing
+is checked there either. Audiobooks are not checked yet.
+
 If the library app downstream reads sidecar metadata, turn on **Write a
 metadata.opf sidecar** in Settings → General (off by default). Bindery then
 writes a Calibre style `metadata.opf` next to each imported ebook and

internal/importer/epubmeta.go

@@ -8,6 +8,7 @@ import (
 	"io"
 	"path"
 	"path/filepath"
+	"slices"
 	"strings"
 
 	"github.com/vavallee/bindery/internal/isbnutil"
@@ -24,6 +25,11 @@ type EpubMetadata struct {
 	Author   string
 	ISBN     string // normalised (digits only); ISBN-13 preferred over ISBN-10
 	Language string // ISO 639-2/B code from dc:language, normalised ("en" → "eng")
+	// Languages is every dc:language the OPF declares, normalised, in document
+	// order with duplicates dropped. Language is its first entry. A bilingual
+	// edition declares both, and the import language check (#2998) must see
+	// the second one too.
+	Languages []string
 }
 
 // IsEpubFile reports whether path is an EPUB we can read embedded metadata from.
@@ -206,8 +212,13 @@ func parseOPFMetadata(r io.Reader) (EpubMetadata, error) {
 				// First dc:language wins. Normalise to the ISO 639-2/B code the
 				// language filter and metadata profiles use so a value like "en"
 				// or "en-US" matches an "eng" profile (#1160).
-				if meta.Language == "" {
-					meta.Language = models.NormalizeLanguageCode(val)
+				if code := models.NormalizeLanguageCode(val); code != "" {
+					if meta.Language == "" {
+						meta.Language = code
+					}
+					if !slices.Contains(meta.Languages, code) {
+						meta.Languages = append(meta.Languages, code)
+					}
 				}
 			}
 		case xml.EndElement:

internal/importer/epubmeta_fuzz_test.go

@@ -5,6 +5,7 @@ import (
 	"bytes"
 	"os"
 	"path/filepath"
+	"reflect"
 	"strings"
 	"testing"
 	"time"
@@ -116,7 +117,7 @@ func FuzzReadEpubMetadata(f *testing.F) {
 		if err != nil {
 			// Error path is the fallback-to-filename-parsing contract: it must
 			// hand back a zero value, not partially-populated metadata.
-			if meta != (EpubMetadata{}) {
+			if !reflect.DeepEqual(meta, EpubMetadata{}) {
 				t.Fatalf("non-zero metadata %+v alongside error %v", meta, err)
 			}
 			return

internal/importer/epubmeta_test.go

@@ -4,6 +4,7 @@ import (
 	"archive/zip"
 	"os"
 	"path/filepath"
+	"slices"
 	"testing"
 
 	"github.com/vavallee/bindery/internal/isbnutil"
@@ -95,6 +96,33 @@ func TestReadEpubMetadata_LanguageRegionSubtag(t *testing.T) {
 	}
 }
 
+// TestReadEpubMetadata_AllLanguages: Language stays the first dc:language for
+// existing callers, and Languages carries every one, normalised and deduped,
+// for the import language check (#2998).
+func TestReadEpubMetadata_AllLanguages(t *testing.T) {
+	opf := `<?xml version="1.0"?>
+<package xmlns="http://www.idpf.org/2007/opf">
+  <metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
+    <dc:title>Le Petit Prince</dc:title>
+    <dc:language> </dc:language>
+    <dc:language>fr</dc:language>
+    <dc:language>en-GB</dc:language>
+    <dc:language>fr-CA</dc:language>
+  </metadata>
+</package>`
+	p := writeTestEpub(t, "content.opf", opf)
+	meta, err := ReadEpubMetadata(p)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if meta.Language != "fre" {
+		t.Errorf("Language = %q, want %q (the first declared)", meta.Language, "fre")
+	}
+	if want := []string{"fre", "eng"}; !slices.Equal(meta.Languages, want) {
+		t.Errorf("Languages = %v, want %v", meta.Languages, want)
+	}
+}
+
 func TestReadEpubMetadata_PrefersAutCreator(t *testing.T) {
 	// First creator is an illustrator; the role="aut" creator must win.
 	opf := `<?xml version="1.0"?>