Search products across five open databases, and fix the broken OFF search (epic #43, backend)

[untraceablez]

Part of #43. Backend half; the frontend PR that renders the grouped results follows.

The bug

The product name search shipped in #59 called the Open Food Facts v2 API with a search_terms parameter. v2 has no such parameter — it was silently ignored, so the endpoint returned an arbitrary slice of the 4.6M-product database:

GET /api/v1/barcode/search?q=peanut+butter
→ Fromage Blanc Nature, Sidi Ali (bottled water), Perly
→ or {"results":[]}   (intermittently)

The existing tests didn't catch it because they mock the transport — the payload shape was right, only the request was wrong. There's now a regression test asserting the OFF query goes to Search-a-licious with q, and that search_terms never comes back.

Open Food Facts is now queried through Search-a-licious (search.openfoodfacts.org), which is relevance-ranked and returns the image URLs the suggestion list needs. Its legacy cgi/search.pl endpoint is currently serving "Page temporarily unavailable" under load, so it isn't used as a fallback.

Multi-source search

Search fans out concurrently to five sources and groups the hits:

Source	Endpoint	Adds
Open Food Facts	Search-a-licious	packaged food
Open Beauty Facts	cgi/search.pl	toiletries, cosmetics
Open Products Facts	cgi/search.pl	general household goods
Open Pet Food Facts	cgi/search.pl	pet food
USDA FoodData Central	FDC API	generic/whole foods

The lower-traffic OFF siblings have no Search-a-licious index, but their CGI search works fine. USDA covers the generic/whole foods (bananas, raw) that the barcode-keyed databases don't carry.

Each group carries its own "see more" link. Sources returning nothing are omitted, and a source that times out, rate-limits or dies degrades to an empty group rather than failing the whole search.

Live output for q=peanut+butter:

== Open Food Facts       Peanut Butter (Peanut Butter & Co) …
== Open Beauty Facts     Fwee Lip Balm Peanut Butter Carmel …
== Open Products Facts   Chocolate Peanut Butter (MYFITNESS) …
== Open Pet Food Facts   Peamutt Butter, Peanut Butter & Banana Cookies (Wagg)
== USDA FoodData Central Peanut Butter, Peanut Butter (Wild Oats) …

API changes

• GET /barcode/search gains groups: [{source, label, results, search_url}]. Suggestions now carry source, source_label and id alongside barcode.
• New GET /barcode/product?source=&id= — selecting a suggestion needs the source, because USDA foods are keyed by FDC id rather than barcode. Unknown sources 400 with the supported list.
• GET /barcode/{barcode} and the flat results/search_url fields are kept, so the currently deployed frontend keeps working until it moves to the grouped shape.

Config

USDA_FDC_API_KEY defaults to the rate-limited DEMO_KEY (~30 req/hr/IP); a free key comes from https://fdc.nal.usda.gov/api-key-signup. Setting it empty drops USDA from search entirely.

Testing

• 417 passed — full backend suite, no regressions.
• 100% line coverage on all three touched source files (barcode_service.py 172, product_sources.py 30, api/v1/barcode.py 26).
• Verified live against all five real APIs, plus source-aware lookup for off / obf / opff / usda, the 400 on an unknown source, and the legacy /barcode/{barcode} route.

🤖 Generated with Claude Code