Release version 0.4.0 with min/max inclusion layer enhancements

CHANGELOG.md

@@ -5,7 +5,26 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.4.0] - 2026-09-22
+
+### Added
+- Minimum/maximum inclusion layer. Nutrient composition alone is not enough to formulate on: the optimizer could return diets that met every nutrient target yet could not be extruded or produce growth, such as bran-heavy rations. Two new tables carry the limits a formulator would otherwise apply by hand — `fasa_core/config/nutrient_limits.csv` (bounds on dietary nutrient levels, by ASNS spec code) and `fasa_core/config/ingredient_limits.csv` (bounds on how much of a single ingredient may go in, by FICD code).
+- Optional `nutrient_limits` and `ingredient_limits` inputs on `/formulate`, for callers that already know the bounds they want. An entry overrides the table for the code it names.
+- `recipe[]` lines now carry the limit applied to each ingredient (`min_inclusion_percent`, `max_inclusion_percent`, `limit_source`), and `composition[]` lines carry `source`, saying whether a target came from the species requirement or from the limit layer.
+- `infeasibility.bound_conflicts` explains, in plain language, ingredient limits that cannot add up to a full batch — the one failure the existing conflict report cannot describe.
+- `/validate-recipe` now checks a recipe's inclusion rates, not only its composition. Optional `species`, `stage`, `production_system`, `processing_method`, `premix_enabled` and the two limit inputs scope the check; `in_limits`, `inclusion_checks`, `nutrient_checks`, `total_inclusion_percent` and `warnings` report the result. Existing payloads are unaffected.
+- `/ready` also loads the two limit tables, so a malformed table is caught at deploy time rather than on the first request.
+
+### Changed
+- An ingredient given a `0` maximum is left out of the recipe even when it is priced, and the response names it in a warning — so "we should not recommend this at all" is distinguishable from "too expensive today".
+- Where a limit and the species requirement bound the same nutrient, the tighter of the two applies. A limit can never relax an ASNS requirement.
+
+### Fixed
+- A recipe returned by `/formulate` no longer reads as out of spec when sent straight back to `/validate-recipe`. Both endpoints now score achieved-versus-target through the same code, with a tolerance that allows for the rounding in the published inclusion percentages.
+
+### Notes
+- Both tables ship with documented headers and **no data rows** — the values are still being compiled, so formulations are unchanged until the columns are filled. Activating them needs no code change.
+- Ingredient limits are written and sent as a mass fraction of total feed (`0.15` = 15%) but reported back as a percentage (`max_inclusion_percent: 15.0`). Nutrient limits use the ASNS unit of their spec code and are not rescaled.
 
 ## [0.3.0] - 2026-07-21
 

README.md

@@ -42,10 +42,11 @@ Project changes are tracked in [`CHANGELOG.md`](CHANGELOG.md).
 - **Linear programming engine** (PuLP + HiGHS) that solves a digestibility-aware least-cost feed formulation against the active ASNS constraints for a chosen species/stage/production-system tuple.
 - **Premix-aware constraint masking**: vitamins and trace minerals are assumed satisfied by a fixed-rate vitamin/mineral premix (default 0.5%), keeping the LP focused on macros, amino acids, digestible energy, Ca, P, fatty acids, and toxin ceilings.
 - **Hard ceilings on toxins / anti-nutrients**: aflatoxin B, gossypol, phytic acid, glucosinolates, tannins, etc. are enforced as Maximum constraints regardless of premix or override settings.
+- **Min/max inclusion layer**: a second layer of bounds on top of composition data, because nutrient tables alone let the LP build diets that pass every target yet cannot be extruded or grown on. `fasa_core/config/nutrient_limits.csv` adds Minimum/Maximum bounds on dietary nutrient levels (the tighter of ASNS and the limit binds); `fasa_core/config/ingredient_limits.csv` bounds how much of any single ingredient may go in. Both ship empty pending the data — filling the columns activates them with no code change — and either can be supplied per request via `nutrient_limits` / `ingredient_limits`.
 - **Opt-in advisory caps**: collective binder mass cap and fish-meal cost-share cap are available as optional inputs but are *not* applied by default. Toxicity and ASNS nutrient limits remain the only fixed constraints; callers that want to force a sustainability or pellet-quality ceiling can pass `max_fishmeal_cost_share` and/or `max_binder_inclusion` explicitly.
 - **Hard-fail with IIS reporting**: when the constraint set is infeasible at the supplied prices/pool, a deletion-filter algorithm extracts a minimal Irreducible Inconsistent Subset of constraints and returns it to the caller so the miller knows exactly why the recipe could not be built.
 - **PAFF benchmark gate**: independent recomputation of PAFF reference recipes' nutrient composition acts as the correctness test for the data-loading and crosswalk pipeline.
-- **FastAPI surface** with `/formulate`, `/supported`, `/validate-recipe`, and `/health` endpoints.
+- **FastAPI surface** with `/formulate`, `/supported`, `/validate-recipe`, and `/health` endpoints. `/validate-recipe` scores a caller-supplied recipe against the same inclusion limits and constraint set the optimizer uses, so a hand-edited formulation can be checked without re-running the LP.
 - **Architectural hooks** (no-ops in MVP) for the post-MVP non-additive interaction layer (Hua & Bureau 2012) and chance-constrained variability handling.
 
 ## Layout
@@ -62,13 +63,16 @@ fasa_engine/
 │   │   ├── crosswalk.json              ASNS spec code -> FICD parameter (+ unit factor)
 │   │   ├── premix_mask.json            which spec codes the premix covers
 │   │   ├── ingredient_pool_africa.csv  country-tagged ingredient pool (KE/NG/ZM)
+│   │   ├── nutrient_limits.csv         min/max bounds on dietary nutrient levels
+│   │   ├── ingredient_limits.csv       min/max bounds on single-ingredient inclusion
 │   │   └── defaults.py                 numeric defaults (premix rate, caps, etc.)
 │   ├── data_loader.py                  ASNS / FICD / PAFF loaders (cached)
 │   ├── crosswalk.py                    spec → FICD parameter resolver
 │   ├── ingredient_pool.py              pool filter
+│   ├── inclusion_limits.py             min/max inclusion layer (nutrient + ingredient)
 │   ├── constraint_builder.py           solver-agnostic LP constraint emission
 │   ├── optimizer.py                    PuLP+HiGHS solve, IIS via deletion filter
-│   ├── validator.py                    independent composition recompute + PAFF gate
+│   ├── validator.py                    recipe scoring (composition + inclusion limits) + PAFF gate
 │   └── models.py                       pydantic request/response schemas
 ├── fasa_api/
 │   └── main.py                         FastAPI app
@@ -125,10 +129,10 @@ uvicorn fasa_api.main:app --reload --port 8000
 The API is self-documented at `GET /docs`. Endpoints:
 
 - `GET /health` liveness probe
-- `GET /ready` readiness probe (validates data files + FICD preload)
+- `GET /ready` readiness probe (validates data files, FICD preload, and the inclusion-limit tables)
 - `GET /supported` discover valid `species`, `production_system`, and `stage` strings
 - `POST /formulate` run the optimization
-- `POST /validate-recipe` recompute composition for an explicit recipe
+- `POST /validate-recipe` score an explicit recipe: recompute composition and check its inclusion rates against the limit layer
 
 Notes:
 - Browsers issue **GET** requests; `GET /formulate` will return **405 Method Not Allowed** because `/formulate` is **POST-only**.
@@ -157,10 +161,19 @@ For external teams integrating this API into other systems, see [`docs/integrati
     "10018": 1.50,
     "62134": 0.80,
     "62138": 0.10
-  }
+  },
+  "nutrient_limits": { "PA05": { "max": 8.0 } },
+  "ingredient_limits": { "31605": { "max": 0.15 } }
 }
 ```
 
+`nutrient_limits` (ASNS spec code → bound in that code's ASNS unit, e.g. `%` for fibre,
+`kcal/kg` for energy) and `ingredient_limits` (FICD code → mass **fraction** of total feed
+in `[0, 1]`, so `0.15` = 15%) are optional per-request overrides of the inclusion-limit
+layer. They override the configured CSVs for the codes they name. Where ASNS already
+states the same bound, the tighter value binds. Note that the response echoes an
+ingredient bound as a **percentage** (`max_inclusion_percent: 15.0`), not a fraction.
+
 `country` is optional (ISO-2: `KE`, `NG`, `ZM`; discover valid values via `/supported`). When supplied, each recipe line is flagged `locally_available`. It is an advisory highlight only — the optimizer always draws on the full pool (millers routinely buy imported soy, premix, etc.), so a country is **never** a hard filter.
 
 Example call with auth:
@@ -174,12 +187,12 @@ curl -X POST "http://127.0.0.1:8000/formulate" \
 
 ### Reading the response (high level)
 
-- **`recipe`**: ingredient inclusions (percent of final feed) + cost breakdown. If `batch_size_kg` was supplied on the request, each line also carries `quantity_kg`. If `country` was supplied, each line carries `locally_available` (`true`/`false`); it is `null` when no `country` was requested.
+- **`recipe`**: ingredient inclusions (percent of final feed) + cost breakdown. If `batch_size_kg` was supplied on the request, each line also carries `quantity_kg`. If `country` was supplied, each line carries `locally_available` (`true`/`false`); it is `null` when no `country` was requested. Each line also echoes the inclusion box that was applied (`min_inclusion_percent`, `max_inclusion_percent`, `limit_source`).
 - **`batch_size_kg` / `premix_quantity_kg` / `total_cost`**: echoed only when `batch_size_kg` was supplied; otherwise `null`.
-- **`composition`**: per-spec achieved vs target, including toxin ceilings.
+- **`composition`**: per-spec achieved vs target, including toxin ceilings. `source` says whether the target came from ASNS alone or from the inclusion-limit layer.
 - **`status`**:
   - `optimal`: solution exists and is least-cost under constraints
-  - `infeasible`: no solution; see `infeasibility.iis_codes` / `iis_explanations` for why
+  - `infeasible`: no solution; see `infeasibility.iis_codes` / `iis_explanations` for why, or `infeasibility.bound_conflicts` when the ingredient inclusion limits alone cannot add up to the feed mass
 
 ### Error envelope
 
@@ -202,7 +215,7 @@ Business/auth errors are returned with:
 - Container command binds to `0.0.0.0:$PORT`.
 - Required data files are bundled under `data/`.
 - `/health` is liveness and public.
-- `/ready` verifies data artifacts and FICD preload.
+- `/ready` verifies data artifacts, FICD preload, and that the inclusion-limit tables parse.
 - Auth is controlled by:
   - `FASA_REQUIRE_AUTH` (default: `true`)
   - `FASA_API_TOKEN` (required when auth enabled)
@@ -257,7 +270,7 @@ Configure:
 
 ## Modeling notes
 
-1. **Decision variables** `x_i ∈ [0, max_inclusion_i]` = mass fraction of each priced+available ingredient.
+1. **Decision variables** `x_i ∈ [min_inclusion_i, max_inclusion_i]` = mass fraction of each priced+available ingredient; the box comes from the inclusion-limit layer and defaults to `[0, 1]`.
 2. **Objective** `min Σ price_i × x_i` (USD/kg or local currency, supplied at runtime).
 3. **Mass balance** `Σ x_i = 1 − premix_rate` so the premix takes a fixed slice.
 4. **Nutritional constraints** generated 1:1 from active ASNS rows for the given (species, system, stage):
@@ -266,8 +279,9 @@ Configure:
    - The `dig_*_fish_percent` and `dig_p_*_percent` FICD columns are bound directly to the corresponding ASNS digestible-nutrient codes — *digestibility is baked into the constraint LHS, not bolted on after*.
 5. **Energy column selection** is data-driven: ASNS itself carries the species-appropriate energy code (Tilapia ⇒ ED02 DE-Omni, African Catfish ⇒ ED01 DE-Carni). The crosswalk then maps that code to the pelleted vs. extruded FICD variant per the request's `processing_method`.
 6. **Hard toxin ceilings** are NEVER masked, even via override (TX01–TX16 always emit Maximum constraints).
-7. **Soft warnings** are appended (not enforced) when any single ingredient exceeds 40 % inclusion.
-8. **On infeasibility**, a deletion-filter IIS is returned so the user can see which constraints conflict.
+7. **Nutrient inclusion limits** join the ASNS row set before emission, so they inherit unit conversion, the composition report, and IIS reporting. Where ASNS states the same direction the tighter value binds; a code the layer bounds is never masked by the premix.
+8. **Soft warnings** are appended (not enforced) when any single ingredient exceeds 40 % inclusion.
+9. **On infeasibility**, a deletion-filter IIS is returned so the user can see which constraints conflict. Ingredient bounds that cannot sum to the feed mass are caught before the solve and reported separately as `bound_conflicts`.
 
 ## Post-MVP roadmap (hooks already in place)
 

docs/architecture.md

@@ -15,7 +15,7 @@ The project provides an API that helps formulate fish feed recipes based on:
 ## Main building blocks
 
 - `fasa_api/`: API layer (receives requests and returns responses)
-- `fasa_core/`: core logic (data loading, constraints, optimization, validation)
+- `fasa_core/`: core logic (data loading, constraints, inclusion limits, optimization, validation)
 - `data/`: reference CSV files used by the core logic
 - `.github/workflows/`: CI, deployment, and release automation
 
@@ -43,6 +43,13 @@ The project provides an API that helps formulate fish feed recipes based on:
   `/formulate` flags which ingredients are *locally available* in the response; it never
   restricts the optimization. The pool is regenerated from source lists by
   `scripts/build_ingredient_pool.py`.
+- Two hand-maintained tables hold the min/max inclusion layer:
+  `fasa_core/config/nutrient_limits.csv` (bounds on dietary nutrient levels) and
+  `fasa_core/config/ingredient_limits.csv` (bounds on single-ingredient inclusion).
+  They express what a mill can actually formulate and process, which composition data
+  alone does not. Both ship without data rows pending the values; filling the columns
+  activates the layer with no code change, and clients can supply bounds per request
+  in the meantime.
 
 ## Versioning and changes
 

docs/integration.md

@@ -70,7 +70,27 @@ Recommended client behavior:
 - Optionally send `country` (ISO-2: `KE`/`NG`/`ZM`) to have each recipe line flagged
   `locally_available` (`true`/`false`; `null` when no `country` is sent). This is an
   advisory highlight only — the optimizer is never restricted to local ingredients.
+- Optionally send `nutrient_limits` and/or `ingredient_limits` to bound dietary nutrient
+  levels (ASNS spec code -> `{min, max}` in that code's ASNS unit) or single-ingredient
+  inclusion (FICD code -> `{min, max}` as a mass fraction of feed). These override the
+  engine's configured limit tables for the codes they name; where ASNS already states the
+  same bound, the tighter value binds. Each `recipe` line echoes the box applied
+  (`min_inclusion_percent`, `max_inclusion_percent`, `limit_source`) and each
+  `composition` line carries a `source` naming where its target came from.
+- **Watch the units on ingredient limits:** you send a **fraction** (`{"max": 0.15}`) and
+  get back a **percentage** (`"max_inclusion_percent": 15.0`). Out-of-range request values
+  are rejected with 422, so `15` sent for 15% fails loudly, but rendering `0.15` as "0.15%"
+  fails silently — convert in both directions. Nutrient limits are not rescaled: they go in
+  and come back in that spec code's ASNS unit.
 - Handle `status` in response (`optimal`, `infeasible`, `error`).
+- On `infeasible`, check `infeasibility.bound_conflicts` before `iis_codes`: when it is
+  non-empty the ingredient inclusion limits alone cannot add up to the feed mass and no
+  IIS was computed.
+- To check a recipe the user edited (or one from elsewhere) without re-running the LP,
+  post it to `/validate-recipe` with the same `species`/`stage`/`production_system` and
+  any limit overrides. It returns `in_limits`, per-ingredient `inclusion_checks`, and
+  `nutrient_checks` in the same shape as `/formulate`'s `composition`. All fields other
+  than `fractions` are optional, so existing callers are unaffected.
 - Log request IDs (if the calling client or service provides them) for easier debugging.
 
 ## 7) Upgrade playbook