feat: harden provider onboarding and resolution

README.md

@@ -33,8 +33,21 @@ git clone https://github.com/maxjustships/nomnomcli
 cd nomnomcli
 python3 -m pip install -e .
 nomnom --version
+nomnom setup
+nomnom doctor --json
 ```
 
+Run `nomnom setup` in an interactive terminal before the first food log. It probes Open Food
+Facts, explains that OFF needs no key, shows the official USDA signup URL, validates the USDA key
+with a minimal FoodData Central request, and only then stores it locally. Follow with
+`nomnom doctor --json` to verify current provider reachability instead of assuming setup worked.
+
+USDA credentials are local user configuration, never repository or database content. The default
+path is `$XDG_CONFIG_HOME/nomnomcli/config.toml` or `~/.config/nomnomcli/config.toml`, written with
+owner-only `0600` permissions. `NOMNOM_USDA_KEY` is the non-interactive/CI option and takes
+precedence over the stored key. Avoid putting it in commands, logs, or checked-in environment
+files.
+
 ## Resolve and log food
 
 ```sh
@@ -48,7 +61,7 @@ Resolution is deterministic and ordered:
 2. exact match in the user's `food_cache`;
 3. token-overlap search in that cache;
 4. Open Food Facts free-text search;
-5. USDA FoodData Central, only when `NOMNOM_USDA_KEY` is set;
+5. USDA FoodData Central, when a setup key or `NOMNOM_USDA_KEY` is configured;
 6. actionable JSON error—never a guessed food.
 
 Open Food Facts candidates need at least 0.5 normalized token overlap between the query and
@@ -60,16 +73,37 @@ Successful API results are cached in the user's database, so the same food can r
 later. Existing v0.2 cache records, logs, and recipes are preserved when v0.3 opens the database.
 `nomnom search QUERY` searches this user cache; it is not a packaged food catalog.
 
+Open Food Facts search goes directly through its official REST endpoint,
+`https://api.openfoodfacts.org/api/v2/search`, with a descriptive nomnomcli User-Agent. Direct
+`requests` calls are intentional: the official REST contract is small and transparent, while a
+third-party OFF SDK would add an unofficial dependency without adding an authenticated workflow or
+schema capability. nomnom applies bounded backoff to HTTP 429 and 5xx responses, safely honors a
+numeric `Retry-After`, and reports a typed retryable `openfoodfacts_unavailable` error after retries
+are exhausted. It never silently falls back to the unstable `world` search host.
+
 ### Enable USDA fallback
 
-Get a free key at <https://fdc.nal.usda.gov/api-key-signup.html>, then set it in the environment:
+Run the guided flow (recommended):
+
+```sh
+nomnom setup
+nomnom doctor --json
+```
+
+Get a free key at the official signup page
+<https://fdc.nal.usda.gov/api-key-signup.html>. For non-interactive/CI use only, set it in the
+environment:
 
 ```sh
 export NOMNOM_USDA_KEY="your-key"
 ```
 
-Without the key, a food that OFF cannot resolve returns `usda_key_required`, the same setup URL,
-and the environment variable name. USDA matches cache with `source=usda` and their `fdc_id`.
+Without a key, a food that OFF cannot resolve returns `usda_key_required`, the setup command, and
+the same signup URL. USDA search requires complete positive kcal/protein/fat/carbs, scores query
+token overlap together with data type and category, prefers Foundation and SR Legacy, and enforces
+a confidence floor. Weak matches return `usda_low_confidence` with candidate alternatives and are
+never cached. Accepted matches cache `source=usda`, `fdc_id`, and any returned serving-field
+provenance.
 
 Set `NOMNOM_OFFLINE=1` to prevent all remote food lookup. Set `NOMNOM_DISABLE_OFF=1` to skip OFF
 while retaining USDA when its key is configured.
@@ -119,8 +153,10 @@ The agent must not translate by inventing nutrition values or silently substitut
 
 The parser accepts kilograms, grams, millilitres, pieces, fractions, and explicit per-piece grams.
 English and Russian size words such as `small` and `небольшой` remain valid syntax, but a size word
-does not supply a packaged estimate. Piece grams come only from the resolved cached/pinned/API food
-record. When that field is absent, `piece_weight_unknown` asks for grams.
+does not supply a packaged estimate. Piece grams come only from explicit user input or serving data
+on the resolved cached/pinned/API food record. Assumptions identify the provider, source field, and
+returned value. When serving data is absent, structured `piece_weight_unknown` asks for exact
+grams. Explicit grams always win, including `3 pieces FOOD at 38g` → `114g`.
 
 ```sh
 nomnom log --parse "bread 2 pieces at 40g" --json
@@ -150,11 +186,17 @@ an `error` object and exit with status 2. Important codes include:
 
 - `off_low_confidence`: inspect `candidate` and `alternatives`; retry more specifically or pin a
   verified label.
+- `usda_low_confidence`: inspect the FDC candidate/data type/category and retry more specifically;
+  no near match was cached.
+- `usda_invalid_nutrition`: every USDA candidate lacked one or more complete positive core values.
 - `usda_key_required`: configure the free FDC key or pin verified values.
 - `piece_weight_unknown`: ask for grams or add a verified `--piece-grams` value.
 - `alias_target_not_found`: add/resolve the exact cached target or remove the stale alias.
 - `openfoodfacts_unavailable` / `usda_unavailable`: retry later or use a manual label.
 
+Provider-unavailable errors include a `retryable` boolean. `nomnom doctor --json` always emits only
+provider `configured`/`reachable` state and USDA `key_source`; it never includes credential values.
+
 Successful logs are stored immediately. Agents should show the returned names, grams, confidence,
 alternatives, and assumptions before treating the resolution as confirmed.
 

install.sh

@@ -66,4 +66,5 @@ else
 fi
 
 say "nomnomcli installation complete."
-say "Next: run 'nomnom --help', then 'nomnom search buckwheat'."
+say "Next: run 'nomnom setup' in an interactive terminal before the first food log."
+say "Then verify provider readiness with 'nomnom doctor --json'."

nomnomcli/cli.py

@@ -11,6 +11,7 @@
 from nomnomcli.errors import NomnomError
 from nomnomcli.foods import FoodRepository
 from nomnomcli.models import scale_food, total_items
+from nomnomcli.onboarding import doctor_report, setup_providers
 from nomnomcli.parser import parse_free_text
 from nomnomcli.recipes import fetch_recipe, recipe_portion, save_recipe
 
@@ -60,6 +61,11 @@ def _build_parser() -> argparse.ArgumentParser:
     parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
     commands = parser.add_subparsers(dest="command", required=True)
 
+    commands.add_parser("setup", help="configure and validate nutrition providers")
+
+    doctor = commands.add_parser("doctor", help="probe provider readiness")
+    doctor.add_argument("--json", action="store_true", help="machine-readable JSON output")
+
     log = commands.add_parser("log", help="resolve and store food")
     form = log.add_mutually_exclusive_group(required=True)
     form.add_argument("--parse", metavar="TEXT", help="comma-separated food phrases")
@@ -112,6 +118,35 @@ def _build_parser() -> argparse.ArgumentParser:
 
 
 def _run(args: argparse.Namespace) -> int:
+    if args.command == "setup":
+        print("Open Food Facts: free, no account or key; branded packaged foods.")
+        print("USDA FoodData Central: free API key; generic/raw foods and fallback.")
+        print("USDA signup: https://fdc.nal.usda.gov/api-key-signup.html")
+        result = setup_providers(interactive=sys.stdin.isatty())
+        off_status = (
+            "reachable"
+            if result["providers"]["openfoodfacts"]["reachable"]
+            else "unreachable"
+        )
+        print(f"Open Food Facts (no key): {off_status}")
+        print(
+            "USDA: reachable; key source "
+            f"{result['providers']['usda']['key_source']}"
+        )
+        if result.get("config_path"):
+            print(f"Saved secure provider config: {result['config_path']} (0600)")
+        return 0
+
+    if args.command == "doctor":
+        result = doctor_report()
+        if args.json:
+            print(_json_output(result))
+        else:
+            for provider, status in result["providers"].items():
+                state = "reachable" if status["reachable"] else "unreachable"
+                print(f"{provider}: {state}")
+        return 0
+
     with connect() as connection:
         repository = FoodRepository(connection)
         if args.command == "alias":

nomnomcli/config.py

@@ -0,0 +1,105 @@
+from __future__ import annotations
+
+import json
+import os
+import tempfile
+import tomllib
+from collections.abc import Mapping
+from contextlib import suppress
+from dataclasses import dataclass
+from pathlib import Path
+
+from nomnomcli.errors import NomnomError
+
+USDA_ENV_VAR = "NOMNOM_USDA_KEY"
+
+
+@dataclass(frozen=True, slots=True)
+class Credential:
+    value: str
+    source: str
+
+
+class ProviderConfig:
+    """Read and persist provider settings outside the repository and user database."""
+
+    def __init__(
+        self,
+        *,
+        environ: Mapping[str, str] | None = None,
+        config_path: str | Path | None = None,
+    ) -> None:
+        self._environ = os.environ if environ is None else environ
+        self._config_path = Path(config_path).expanduser() if config_path else None
+
+    @property
+    def path(self) -> Path:
+        if self._config_path is not None:
+            return self._config_path
+        root = self._environ.get("XDG_CONFIG_HOME")
+        base = Path(root).expanduser() if root else Path.home() / ".config"
+        return base / "nomnomcli" / "config.toml"
+
+    def usda_credential(self) -> Credential | None:
+        environment_key = self._environ.get(USDA_ENV_VAR, "").strip()
+        if environment_key:
+            return Credential(environment_key, "environment")
+        stored_key = self._stored_usda_key()
+        return Credential(stored_key, "user_config") if stored_key else None
+
+    def _stored_usda_key(self) -> str | None:
+        if not self.path.exists():
+            return None
+        try:
+            payload = tomllib.loads(self.path.read_text(encoding="utf-8"))
+            value = payload.get("providers", {}).get("usda", {}).get("api_key")
+        except (OSError, tomllib.TOMLDecodeError, AttributeError) as exc:
+            raise NomnomError(
+                "provider_config_invalid",
+                f"Provider configuration is invalid: {self.path}",
+                details={"path": str(self.path), "action": "Run nomnom setup again"},
+            ) from exc
+        if value is None:
+            return None
+        if not isinstance(value, str):
+            raise NomnomError(
+                "provider_config_invalid",
+                "USDA api_key in provider configuration must be a string",
+                details={"path": str(self.path), "action": "Run nomnom setup again"},
+            )
+        return value.strip() or None
+
+    def store_usda_key(self, api_key: str) -> Path:
+        key = api_key.strip()
+        if not key or any(ord(character) < 32 for character in key):
+            raise NomnomError("usda_key_invalid", "USDA API key must not be empty")
+
+        destination = self.path
+        destination.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
+        with suppress(OSError):
+            destination.parent.chmod(0o700)
+        content = f"[providers.usda]\napi_key = {json.dumps(key, ensure_ascii=False)}\n"
+        temporary_name: str | None = None
+        try:
+            descriptor, temporary_name = tempfile.mkstemp(
+                prefix=".config.toml.", dir=destination.parent
+            )
+            os.fchmod(descriptor, 0o600)
+            with os.fdopen(descriptor, "w", encoding="utf-8") as temporary:
+                temporary.write(content)
+                temporary.flush()
+                os.fsync(temporary.fileno())
+            os.replace(temporary_name, destination)
+            temporary_name = None
+            destination.chmod(0o600)
+        except OSError as exc:
+            raise NomnomError(
+                "provider_config_write_failed",
+                f"Could not securely write provider configuration: {destination}",
+                details={"path": str(destination)},
+            ) from exc
+        finally:
+            if temporary_name is not None:
+                with suppress(FileNotFoundError):
+                    Path(temporary_name).unlink()
+        return destination

nomnomcli/db.py

@@ -8,7 +8,7 @@
 from datetime import datetime, timedelta
 from pathlib import Path
 
-LATEST_SCHEMA_VERSION = 3
+LATEST_SCHEMA_VERSION = 4
 V1_TABLES = frozenset({"food_cache", "log_entries", "recipes"})
 
 LATEST_SCHEMA = (
@@ -25,7 +25,9 @@
     barcode TEXT,
     brand TEXT,
     lookup_query TEXT,
-    alternatives_json TEXT
+    alternatives_json TEXT,
+    piece_grams_source TEXT,
+    piece_grams_source_value TEXT
 )""",
     """CREATE TABLE log_entries (
     id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -101,7 +103,20 @@ def _migrate_v2_to_v3(connection: sqlite3.Connection) -> None:
     )
 
 
-MIGRATIONS = {1: _migrate_v1_to_v2, 2: _migrate_v2_to_v3}
+def _migrate_v3_to_v4(connection: sqlite3.Connection) -> None:
+    columns = _column_names(connection, "food_cache")
+    additions = {
+        "piece_grams_source": "ALTER TABLE food_cache ADD COLUMN piece_grams_source TEXT",
+        "piece_grams_source_value": (
+            "ALTER TABLE food_cache ADD COLUMN piece_grams_source_value TEXT"
+        ),
+    }
+    for column, statement in additions.items():
+        if column not in columns:
+            connection.execute(statement)
+
+
+MIGRATIONS = {1: _migrate_v1_to_v2, 2: _migrate_v2_to_v3, 3: _migrate_v3_to_v4}
 
 
 def _initialize_database(connection: sqlite3.Connection) -> None:

nomnomcli/errors.py

@@ -17,3 +17,24 @@ def as_dict(self) -> dict:
         if self.details:
             error["details"] = self.details
         return {"error": error}
+
+
+class ProviderUnavailableError(NomnomError):
+    """A provider failure whose retryability is explicit in the public contract."""
+
+    def __init__(
+        self,
+        provider: str,
+        code: str,
+        message: str,
+        *,
+        retryable: bool,
+        details: dict | None = None,
+    ) -> None:
+        super().__init__(
+            code,
+            message,
+            details={"provider": provider, "retryable": retryable, **(details or {})},
+        )
+        self.provider = provider
+        self.retryable = retryable