From 392fae334c080c3fbb6223b363332f3deeee4c0c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 20:35:30 +0000 Subject: [PATCH 01/12] docs(plans): draft the Phase 4 plan (forecasts, tariffs, planner, battery) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/plans/phase-4.md | 339 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 339 insertions(+) create mode 100644 docs/plans/phase-4.md diff --git a/docs/plans/phase-4.md b/docs/plans/phase-4.md new file mode 100644 index 0000000..8ea464e --- /dev/null +++ b/docs/plans/phase-4.md @@ -0,0 +1,339 @@ +# Phase 4 – Forecasts, tariffs, planner, battery + +Status: **draft, waiting for the maintainer's approval** (questions in §6; bold defaults apply +where he does not answer) +Source of truth: [`docs/BRIEF.md`](../BRIEF.md) §3 (two control layers), §4.1 (battery, weather), +§4.2, §4.3, §4.4, §4.5 (battery rules of the surplus distribution), §4.8, §4.9 (plan view, only +the part Phase 4 needs), §4.10 steps 4–6 and 10, §5, §8, §9, §10; +[`docs/roadmap.md`](../roadmap.md) Phase 4; ADR-0003, 0004, 0006, 0011, 0012; +[`docs/research/integrations.md`](../research/integrations.md), +[`kostal-plenticore-modbus.md`](../research/kostal-plenticore-modbus.md), +[`solver-on-aarch64.md`](../research/solver-on-aarch64.md), +[`device-integrations.md`](../research/device-integrations.md); the "Phase 4 follow-ups" and the +`UNVERIFIED` table in [`docs/STATUS.md`](../STATUS.md). +Branch: `claude/zealous-mayer-j5oda9` (session branch, see §3 and question 1). + +Phase 4 gives SunCore foresight. Adapters read PV forecasts and prices from the integrations the +user already runs, a load forecast is learnt from Home Assistant's statistics, and a planner +(HiGHS LP/MILP with a heuristic fallback) computes a 24–48 h schedule in 15-minute steps. The plan +fills the `Plan` contract the Phase 3 controller already consumes (`DeviceBand`, `BatteryBand`, +`grid_import_limit_w`), so the controller keeps the last word in real time and the safety gate +stays the only writer. The battery becomes a controllable asset through battery adapters, and +every plan interval carries structured reasons that the UI renders as "Why?". **Shadow mode stays +the default; the battery starts in shadow like every device.** + +Phase 3 is implemented but its acceptance (shadow run and one device to automatic on the +maintainer's system) is still open. Phase 4 is built on top of `main` in parallel; nothing in +Phase 4 changes the Phase 3 write paths of devices (question 2). + +## 1. Goals and acceptance criteria + +Phase 4 is done when (roadmap Phase 4 milestones): + +1. **Forecast adapters:** Solcast (attributes or the read-only query action, never the Solcast + API), Forecast.Solar (core, via `energy/solar_forecast`), Open-Meteo Solar Forecast (HACS), + a generic attribute adapter (mapping per template), and the built-in fallback behind an + explicit opt-in (question 5). Several sources at once are blended by measured skill; a bias + calibration per hour of day and season is learnt; clipping (inverter limit, export limit) is + recognised; accuracy (MAE, bias per source and horizon) is stored. +2. **Load forecast:** base-load profile from HA long-term statistics (`recorder/statistics_during_period`), + weekday/holiday (optional `workday` binary sensor), optional temperature from a `weather` + entity (`weather.get_forecasts`); exponential smoothing; the draw of controlled devices is + subtracted (no double counting). +3. **Tariffs:** dynamic import from Tibber, Nord Pool, ENTSO-E, EPEX Spot (and Octopus if its + source can be verified), fixed and HT/NT (time windows, seasonal); price formula + (spot → end price with grid fee incl. time-variable grid fees §14a module 3, levies, margin, + VAT); 15- and 60-minute grids; export fixed, tiered by kWp share (blended rate) and none; + negative-price region rule (`de`: no remuneration → avoid export). +4. **Planner:** HiGHS model (`highspy`), heuristic fallback (always available, also when the + solver times out or the model is infeasible, with a UI hint), warm start from the previous + plan, time budget (default 8 s, MIP gap 1 %), runs every 15 min and on events. Benchmark: + the maintainer's reference scenario solves in < 10 s on aarch64 (CI arm runner as proxy, + see R3). +5. **Battery strategies:** self-consumption (default), target SoC by time (P10 PV in the target + constraint), grid charging on a dynamic tariff only when the spread exceeds efficiency loss + + degradation + margin (option "night only", already in `goals`), grid charging on a fixed + tariff only with an HT/NT spread or in an emergency (off by default), discharge lock, reserve + separate from BMS drift, max-SoC limit, periodic calibration charge, cycle counter, + degradation cost in currency/kWh. +6. **Battery adapters:** generic HA entities (template kind `battery`: number/select/switch/ + actions), Kostal Plenticore through the core `kostal_plenticore` integration (min SoC as + discharge lock/reserve), Kostal Modbus §3.5 limits only with the verified register map and + behind question 3. Read-back after every write, write only on change, release = hand the + battery back to the inverter's own control. +7. **Explanations:** every plan interval carries `Reason(key, args)` (price, forecast, target, + limit, rule); the UI renders them in a plan view and in the Now view's "next actions". +8. **`control-safety` review** of every change to `safety/`, the controller's battery path and + the battery adapters' write paths, documented in the PR (invariant table). + +Exit (roadmap): plan with battery strategy and "Why?" on the maintainer's system, battery +control in shadow mode, then assisted. + +## 2. Work packages + +Owner = subagent that implements it. Budget rule (CLAUDE.md, your instruction): **Fable only for +`optimizer-engineer` (planner, forecast mathematics, battery economics), `control-safety` +(controller battery path, safety, battery write paths) and the final `reviewer`**; Sonnet for +adapters, backend, simulator, frontend and docs; Haiku for catalogue sync. At most two subagents +in parallel. + +### WP1 – Verify interfaces · `ha-integration` (skills `verify-ha-api`, `new-adapter`) + +Read from the GitHub sources (the doc sites are unreachable, see `research/README.md`) and record +in `research/integrations.md` / `research/ha-websocket-api.md`, each fact with commit and path: + +- HA: `recorder/statistics_during_period` (request/response, `period`, `types`, units), + `energy/solar_forecast` (response shape, `wh_hours`), `weather.get_forecasts` (response, + `hourly`), `workday` binary sensor, `number.set_value`, `select.select_option`, + `modbus.write_register` (fields, multi-register values) for question 3. +- Integrations: Forecast.Solar `energy.py`; Open-Meteo Solar Forecast (domain, attributes); + Nord Pool `get_prices_for_date`; ENTSO-E and EPEX Spot attributes; Octopus `rates`; + `kostal_plenticore` `Battery:MinSoc`/charging mode entities (translation keys, ranges). +- Built-in fallbacks: Open-Meteo `global_tilted_irradiance`/`tilt`/`azimuth` and its terms; + Energy-Charts `/price` and aWATTar `marketdata` and their terms. If unreachable they stay + `UNVERIFIED` and are not shipped (question 5). + +### WP2 – Contracts · `optimizer-engineer` (Fable), reviewed by `architect` + +- `core/series.py`: `TimeSeries` (aware start/step, values, unit), `QuantileSeries` + (P10/P50/P90, monotone), `PriceSeries` (import/export per interval, resolution 15/60 min, + source, `price_includes`), resampling to the 15-minute planning grid (energy-preserving for + power/energy, step-hold for prices). +- `core/forecast.py`: `ForecastSource` protocol (`fetch(window) -> QuantileSeries`), accuracy + record; `core/tariff.py`: `PriceSource` protocol, `PriceFormula`, HT/NT windows; + `core/battery.py`: `BatteryModel` (capacity, efficiencies, limits, reserve, BMS-drift margin, + degradation cost), `BatteryAction` for the gate (charge/discharge limits, min SoC, grid charge + power) – **`BatteryAction` is reviewed by `control-safety`**. +- `Plan` v2 (additive to `core/control.py`): per interval the expected flows (PV P10/P50/P90, + load, import, export, battery power, SoC), prices, the battery band, reasons; `Plan.source` + (`solver`/`heuristic`/`default`) and solver statistics. The controller keeps reading + `DeviceBand`/`BatteryBand`/`grid_import_limit_w` unchanged. +- Config documents (additive, migrated): `ForecastConfig` v2 (several sources, weights auto/ + manual, safety discount), `TariffConfig` v3 (formula components, HT/NT windows, seasonal), + section `battery` (adapter template id, entities, model parameters, strategy options), + `goals` v2 (night window for grid charging, calibration interval, conservative planning on/off). +- Template schema v1 additions (additive, schema v1 changelog): kind `battery` (ADR-0012 already + names it), forecast `series` mapping for the generic attribute adapter, tariff HT/NT. + +### WP3 – Storage and statistics · `backend-engineer` + +- `/data/timeseries.db` (ADR-0004; `backup_exclude` in both `config.yaml`, via `ha-integration`): + own migrations; tables for measurements (controller resolution 48 h, 1-min 30 days, 15-min + unlimited), forecasts versioned by issue time, prices; nightly downsample + vacuum; rebuild of + 15-min history from `recorder/statistics_during_period` when the file is missing. +- `suncore.db` migration `0004`: `plan_snapshot` (compressed JSON, last N plans + retention), + learnt parameters (forecast bias tables, load profiles, blending weights). Schema 4: 0.4.x + refuses newer databases → export before update (release note). +- Rate limiter per (domain, service) with try-acquire/deadline for `call_action_response` + (follow-up A3 from Phase 1) before the first adapter calls an action; boundary rule 7 admits + the adapter wiring. + +### WP4 – Tariff adapters · `backend-engineer` (skill `new-adapter`), maths by `optimizer-engineer` + +`suncore/tariff/`: adapters Tibber (`tibber.get_prices`), Nord Pool, ENTSO-E, EPEX Spot, +Octopus (only if WP1 verifies it), generic price-list attribute, fixed and HT/NT; price formula; +export fixed/tiered (blended rate by kWp share)/none; negative-price rule from the region +template; refresh on the integration's update, never more than the rate limiter allows. Fixtures +copied from the verified source shapes. New templates per adapter with `source` and a conformance +scenario (`new-device-template` skill). + +### WP5 – PV forecast · `device-adapters` (adapters), `optimizer-engineer` (Fable, blending/calibration) + +`suncore/forecast/pv/`: Solcast (`detailedForecast` attributes, `query_forecast_data` as +fallback), Forecast.Solar, Open-Meteo Solar Forecast, generic attribute, built-in fallback (only +after question 5); blending weighted by recent skill (inverse MAE over 14 days, minimum weight, +manual override); bias calibration per hour of day × season × clear-sky index bucket (bounded, +shrinks to 1 with little data); clipping detection (inverter/export limit plateaus excluded from +the error statistics); accuracy bookkeeping in `timeseries.db`. + +### WP6 – Load forecast · `optimizer-engineer` (Fable) + +`suncore/forecast/load.py`: house base load = house − controlled devices (their estimates from +the Phase 3 power models); profile per weekday type (workday/weekend/holiday) × 15-min slot with +exponential smoothing; temperature regression term when a weather entity is mapped; P10/P90 from +residuals; cold start from HA statistics (WP3), else a flat profile from the current mean with a +wide band and a UI hint. No ML model in Phase 4 (aarch64 footprint). + +### WP7 – Planner · `optimizer-engineer` (Fable) + +`suncore/planner/`: `model.py` (sparse matrices for `highspy`, Protocol for the backend so tests +can use a reference solver), `heuristic.py`, `explain.py`, `run.py`. + +- Variables per interval: import, export, battery charge/discharge (separate efficiencies), SoC, + PV curtailment (only where an export limit or negative prices make it useful, informational + until the export-limit chain in Phase 6), controllable device energy (band per device). +- Constraints: energy balance, SoC bounds incl. reserve + BMS-drift margin, battery power limits, + grid import limit, static export limit (13.5 kW case), target SoC by time on **P10** PV, + grid charging only in allowed windows (night window, fixed tariff off by default), discharge + lock where it pays; binaries only where needed (no simultaneous charge/discharge only if the + LP relaxation exploits it with negative prices; device min run later). +- Objective: import cost − export revenue (with negative prices and the region rule) + degradation + + switching penalty + soft target penalties; P50 in the objective. +- Warm start, time budget, MIP gap; on timeout use the incumbent if feasible, else the heuristic; + infeasible → relax soft targets, then heuristic, reason `planner.fallback.*`. +- `explain.py`: per interval the binding constraints and price comparisons become `Reason`s, e.g. + `planner.battery.grid_charge` {from, to, price, later_price, later_time, pv_p10_kwh}. +- Outputs `Plan` v2 with `BatteryBand` (min/max power, SoC corridor, `charge_from_surplus`) and + `DeviceBand` preferences; the controller's surplus rule "battery first / consumers first if the + forecast still reaches the target" now follows the plan. +- Property tests (hypothesis): balance holds, SoC within bounds, never worse than the heuristic, + no grid charging outside allowed windows, no export at negative prices when avoidable. +- Benchmark script `scripts/bench_planner.py` with the maintainer's reference scenario + (17 kWp east-west, 29.6 kWh, Tibber 15-min prices, export limit 13.5 kW, tiered EEG). + +### WP8 – Runtime and scheduler · `backend-engineer` + +Planner run every 15 min and on events (new prices, forecast update, deviation above threshold, +config change, override), in a worker thread with the time budget, at most one run at a time; +the controller gets the new plan atomically; plan snapshots stored; `/health` unaffected by +planner failures (status in the API). API: `GET /api/v1/plan` (current plan with reasons and +statistics), `POST /api/v1/plan/replan`, `GET /api/v1/forecast/accuracy`, `GET /api/v1/prices`; +live feed `plan` message. OpenAPI and frontend types regenerated. Demo: simulator prices and +forecasts so the demo shows a real plan. + +### WP9 – Battery control · `control-safety` (Fable), adapters by `device-adapters` + +- Controller: translates the plan's `BatteryBand` into `BatteryAction` intents (on change only, + hysteresis on power levels); the existing surplus share follows `charge_from_surplus`. +- Safety: new invariants (property-tested): never discharge below reserve + drift margin by + SunCore; grid charging only from a plan interval that allows it, within the import limit and + the night window; charge/discharge limits within battery and inverter limits; write rate and + write-on-change; battery release = restore the adapter's baseline (inverter self-control); + watchdog: stale SoC/battery power while the grid meter moves pauses battery control; the + hung-inverter case (BRIEF §9) releases and stops writing. "No SunCore-caused import" keeps + holding for devices; the only allowed import is planned grid charging of the battery. +- Adapters `suncore/devices/battery/`: `generic` (template kind `battery`, roles: min-SoC number, + max charge/discharge numbers, mode select, force-charge switch/action), `kostal_plenticore` + (core integration: min SoC → discharge lock/reserve, charging mode select; no charge set-point + over REST), `kostal_modbus` §3.5 limits with fallback timer (question 3; shadow/assisted only + until the timeout is measured on the maintainer's inverter). +- Simulator: battery with BMS drift, inverter self-consumption logic as the fallback behaviour, + Kostal-like fallback timer, scenarios `negative_prices`, `price_spike`, `winter_fog_target`, + `grid_charge_night`, `inverter_hang_battery`, `reference_house` (maintainer's system). + +### WP10 – UI · `frontend-engineer` + +- Plan view `#/plan` (minimal for Phase 4, the full chart is Phase 5): ECharts loaded on demand + (own chunk), 24/48 h, price band, PV P10–P90 band + P50, load forecast, planned SoC and battery + power, import/export; tap/hover on an interval shows its reasons; solver/heuristic badge. +- Now view: "next planned actions" with "Why?"; battery card (mode switch, proposals for + assisted, release/emergency stop like the devices). +- Wizard: step 4 several forecast sources (auto-detected), step 5 battery adapter or "observe + only" with model parameters, step 6 price formula and HT/NT, step 8 battery strategy options, + step 10 first plan as preview. Settings pages follow. +- Vitest, Playwright E2E against the demo, axe; screenshots `plan-*` (light/dark, phone/tablet, + en/de) via `demo-screenshots`. Bundle budget 250 kB gzip for first paint holds (ECharts lazy). + +### WP11 – Docs, translations, STATUS · `docs-writer`, `i18n-sync` (Haiku) + +User guide `docs/user//planning-and-battery.md` (en/de: forecasts and blending, tariffs and +price formula, how the planner decides, battery strategies, battery adapters and how to enable +external battery control, shadow → assisted walk-through, known limitations), quick start and +app `DOCS.md` updated, architecture §planner, compatibility list regenerated, all new keys en/de +(`sync-translations`), STATUS, roadmap, `UNVERIFIED` table. + +## 3. Pull requests and order + +Default (question 1): all work on the session branch `claude/zealous-mayer-j5oda9`, one commit +series per WP and **one PR**; if the maintainer allows extra branches, the same commits are split +into stacked PRs: + +| PR | Content | Depends on | +|---|---|---| +| 4-A `feat(core,storage,forecast,tariff)` | WP1–WP6 | – | +| 4-B `feat(planner,app)` | WP7, WP8 | 4-A | +| 4-C `feat(safety,controller,devices)` battery | WP9 | 4-B | +| 4-D `feat(frontend),docs` | WP10, WP11 | 4-C | + +```text +WP1 ─► WP2 ─┬► WP3 ─► WP4 ─┐ + ├► WP5 ────────┼─► WP7 ─► WP8 ─► WP9 ─► WP10 ─► WP11 + └► WP6 ────────┘ +``` + +I integrate, run `./scripts/check.sh` and the frontend checks after every WP and commit per WP +(Conventional Commits; CHANGELOG via release-please, no hand edits). Before the PR the `reviewer` +agent (Fable) reviews the whole change; its findings are fixed in separate `fix(...)` commits. +`control-safety` signs off WP9 with the invariant table in the PR description. + +## 4. Tests and verification + +| Layer | Tool | What | +|---|---|---| +| Series/contracts | pytest, hypothesis | resampling preserves energy, quantiles monotone, price grid 15/60 mixed | +| Adapters | pytest with fixtures from verified source shapes | Solcast, Forecast.Solar, Open-Meteo SF, generic, Tibber, Nord Pool, ENTSO-E, EPEX, fixed/HT-NT; missing/partial data; DST days (23/25 h) | +| Blending/calibration/load | pytest, hypothesis | weights sum to 1, bias bounded, clipping excluded, controlled loads subtracted | +| Planner | hypothesis + reference solver | balance, SoC bounds, objective ≤ heuristic, grid charging windows, negative prices, infeasible → fallback, time budget respected | +| Benchmark | `scripts/bench_planner.py`, CI job on `ubuntu-24.04-arm` | reference scenario 48 h; budget see R3 | +| Safety | hypothesis | battery invariants, gate never writes the battery in shadow/assisted without confirm, release paths | +| Simulator | `-m simulator` | scenarios of WP9, 7-day run with planner + controller, fail-safe of the battery | +| Storage | pytest | migrations 0004 + timeseries.db, rebuild from statistics, retention | +| Boundary | AST test | planner imports `core` only; battery adapters' `apply()` only via the gate; actions only through the rate limiter | +| API | httpx ASGI | plan, replan, accuracy, prices | +| Frontend | Vitest, Playwright, axe | plan view, reasons, battery card, wizard steps | +| Coverage | pytest-cov | ≥ 80 % core/safety/planner/controller (aim ≥ 90 % planner and safety) | + +No hardware and no Docker daemon in the session: image and smoke test run in CI; the maintainer's +shadow and assisted run is the acceptance test. + +## 5. Risks + +| # | Risk | Impact | Mitigation | +|---|---|---|---| +| R1 | A battery write empties the battery or charges from the grid at the wrong time | money, backup reserve | battery starts in shadow, assisted needs confirmation, invariants property-tested, reserve + BMS drift margin, `control-safety` sign-off, inverter fallback timer (§3.5) as hardware watchdog | +| R2 | Kostal Modbus timeout and enabling unverified | adapter does nothing or does not fall back | §3.5 limits with documented fallback first; §3.4 set-points out of scope; measurement by the maintainer before automatic | +| R3 | Solve time on a Pi 4 > 10 s | stale plans | LP first, few binaries, warm start, time budget 8 s + heuristic; CI arm runner is faster than a Pi 4 → CI budget 3 s as a proxy; the maintainer runs amd64 (KVM), a Pi 4 measurement needs a volunteer | +| R4 | `highspy` (+ NumPy) enlarges the image and memory | RSS budget 300 MB | musllinux wheels verified (research); smoke-test memory gate stays; solver in a thread, model freed after the run | +| R5 | Forecast adapters rely on attributes that change between integration versions | missing plan | tolerant parsers, source version in `tested_with`, fallback to other sources or flat profile with UI hint | +| R6 | Built-in fallbacks need internet and unverified terms | licence/terms breach, cloud dependency | opt-in, off by default, attribution shown, only after WP1 verified the terms (question 5) | +| R7 | Load forecast poor at start (little history) | bad battery decisions | rebuild from HA statistics, wide P10/P90, conservative target on P10, UI hint | +| R8 | Phase too large for one review | slow review, missed bugs | WP commits, stacked PRs if allowed (question 1), reviewer per PR | +| R9 | DB schema 4 blocks a downgrade | user cannot roll back | release note, export before update | +| R10 | Phase 3 acceptance finds controller issues while Phase 4 builds on it | rework | Phase 4 changes the controller only in the battery path and the plan input; fixes from the live test are merged first | +| R11 | ECharts breaks the bundle budget | slow first paint | lazy chunk for the plan view only | + +## 6. Questions to the maintainer + +Bold = default if there is no answer. + +1. **PR structure:** the session may only push `claude/zealous-mayer-j5oda9`. **One PR with one + commit series per WP** – or may I push stacked branches `phase-4/a…d` (four PRs as in + Phases 1–3)? +2. **Phase 3 not yet accepted:** **build Phase 4 in parallel; merge only after your Phase 3 + acceptance**, fixes from your live test go first. +3. **Kostal battery control path** (you have SunSpec Modbus read-only and the core + `kostal_plenticore` integration): + (a) **Phase 4 ships the generic adapter and `kostal_plenticore` (min SoC = discharge + lock/reserve, no grid charging over REST); the Modbus §3.5 limits are written through Home + Assistant's `modbus.write_register` action (you add a `modbus` hub in HA), shadow/assisted + only until the fallback timer is measured on your inverter**; + (b) SunCore talks Modbus TCP itself (`pymodbus`, opt-in) – the brief's non-goal "no own + Modbus implementation" speaks against it; + (c) no Modbus in Phase 4. Is "external battery management via Modbus" enabled in your + inverter (installer menu)? +4. **Model budget:** in Phase 3 the Fable credits ran out and Opus did the Fable work. **Same + rule again if Fable is unavailable**? +5. **Built-in fallbacks** (Open-Meteo PV, Energy-Charts/aWATTar prices) need internet from the + app and their APIs/terms could not be verified from the session: **only ship them if WP1 can + verify them, then opt-in and off by default; otherwise move them to Phase 6**. +6. **Battery economics defaults** (all expert parameters, shown with their source/explanation): + **charge and discharge efficiency 95 % each, degradation 0.04 €/kWh throughput, grid-charge + safety margin 0.03 €/kWh, BMS-drift margin 5 % above the reserve** (your BMS sags from 10 % + to 6–7 %). Do you know BYD HVM figures you want instead? +7. **Night window for "grid charging only at night":** **22:00–06:00 local**, configurable. +8. **Your EEG tiers** (10 kWp + rest): the region `de` template has no official rates (not + invented). **You enter the two rates in the tariff step**; or do you give me the BNetzA table + to cite? +9. **Tibber `price`:** **treated as the all-in end price** (no formula on top). Correct for your + contract? +10. **Plan view scope:** **minimal plan chart in Phase 4 (price, PV band, load, SoC, battery, + reasons), the full chart with Gantt/zoom in Phase 5**. +11. **Calibration charge:** **off by default; if on, once every 30 days to 100 %, scheduled into + a sunny day by the planner**. + +## 7. Out of scope for Phase 4 + +Full plan chart with device Gantt, zoom/pan, history, savings vs baselines, what-if and +backtesting (Phase 5); export limiting through the inverter, peak shaving with load shedding, +§14a control signal, backup-power awareness, wallbox/EV, heat pump, shiftable programs, MQTT +entities and notifications (Phase 6); Kostal §3.4 set-points and MDC; V2H/V2G; ML load models. From 3e9cf11cc792e241149eb2159cca0d4c812ca24b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 20:46:20 +0000 Subject: [PATCH 02/12] docs(plans): record the maintainer's answers for Phase 4 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/plans/phase-4.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/plans/phase-4.md b/docs/plans/phase-4.md index 8ea464e..9964ce0 100644 --- a/docs/plans/phase-4.md +++ b/docs/plans/phase-4.md @@ -1,7 +1,9 @@ # Phase 4 – Forecasts, tariffs, planner, battery -Status: **draft, waiting for the maintainer's approval** (questions in §6; bold defaults apply -where he does not answer) +Status: **approved** by the maintainer on 2026-10-16 (answers in §6: commit and push after every +WP, Opus takes the Fable roles, battery "as full as possible", grid charging by a price level and +a schedule instead of a fixed night window, calibration charge optional and off; defaults +otherwise) Source of truth: [`docs/BRIEF.md`](../BRIEF.md) §3 (two control layers), §4.1 (battery, weather), §4.2, §4.3, §4.4, §4.5 (battery rules of the surplus distribution), §4.8, §4.9 (plan view, only the part Phase 4 needs), §4.10 steps 4–6 and 10, §5, §8, §9, §10; @@ -331,6 +333,24 @@ Bold = default if there is no answer. 11. **Calibration charge:** **off by default; if on, once every 30 days to 100 %, scheduled into a sunny day by the planner**. +### Answers (2026-10-16) + +- Every WP ends with a commit **and a push**, so nothing is lost when the session's usage quota + runs out. +- **Opus takes the Fable roles** (`optimizer-engineer`, `control-safety`, `reviewer`, advisor). +- Question 1 (PR structure), 2, 3, 5, 6, 8, 9, 10: defaults. +- Question 11: the calibration charge is built for other users, **off by default**. +- Battery strategy of the maintainer: **"keep the battery as full as possible"** – a strategy + option `prefer_full` (target 100 % as early as the P10 forecast allows, PV surplus to the battery + before consumers until full, discharge only for the house); self-consumption stays the default + for new installations. +- Question 7: instead of a fixed night window, grid charging is allowed by **a price level** + (charge only when the import price is at or below a user-set level, currency/kWh) **and a + schedule** (weekly time windows, default: always allowed inside the price level). The spread rule + (efficiency + degradation + margin) still applies on dynamic tariffs; the existing + `goals.grid_charge_night_only` is migrated to a schedule 22:00–06:00. Grid charging stays off + until the user sets a level or a schedule. + ## 7. Out of scope for Phase 4 Full plan chart with device Gantt, zoom/pan, history, savings vs baselines, what-if and From bdce20ca5639b4c3d45dcc8c656b7b1da9286fc6 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 21:20:12 +0000 Subject: [PATCH 03/12] docs(research): verify the Phase 4 forecast, price, statistics and battery interfaces Read from primary sources (GitHub raw files at pinned SHAs and tags, PyPI wheels): recorder statistics, energy/solar_forecast, weather.get_forecasts, workday, modbus.write_register, Nord Pool, Tibber, Kostal Plenticore entities, Solcast, Open-Meteo Solar Forecast, ENTSO-E, EPEX Spot, Octopus Energy and the Open-Meteo API parameters and terms. Energy-Charts and aWATTar stay UNVERIFIED (unreachable). Adds a summary table with the consequence for each adapter. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/research/device-integrations.md | 2 +- docs/research/ha-websocket-api.md | 293 +++++++++++++- docs/research/integrations.md | 443 +++++++++++++++++++++- docs/research/kostal-plenticore-modbus.md | 17 + 4 files changed, 733 insertions(+), 22 deletions(-) diff --git a/docs/research/device-integrations.md b/docs/research/device-integrations.md index b91d177..eb48f01 100644 --- a/docs/research/device-integrations.md +++ b/docs/research/device-integrations.md @@ -336,7 +336,7 @@ The service device is named **"Solcast PV Forecast"**, manufacturer `BJReplay`, |---|---|---|---|---|---| | `total_kwh_forecast_today` | Forecast Today | kWh | energy | total | day total, attributes below | | `total_kwh_forecast_tomorrow` | Forecast Tomorrow | kWh | energy | total | attributes below | -| `total_kwh_forecast_d3` … `_d7` | Forecast Day 3 … | kWh | energy | total | number controlled by the "advanced day entities" option; `UNVERIFIED` default | +| `total_kwh_forecast_d3` … `_d7` | Forecast Day 3 … | kWh | energy | total | number controlled by the advanced option `forecast_day_entities` (min 8, max 14, default 8, i.e. `d3`..`d7` by default; verified 2026-10-16 at tag `v4.6.1`, see `integrations.md`) | | `power_now` | Power Now | W | power | measurement | | | `power_now_30m`, `power_now_1hr` | Power in 30 Minutes / 1 Hour | W | power | measurement | | | `peak_w_today`, `peak_w_tomorrow` | Peak Forecast Today / Tomorrow | W | power | measurement | | diff --git a/docs/research/ha-websocket-api.md b/docs/research/ha-websocket-api.md index 01a70e5..c98d41c 100644 --- a/docs/research/ha-websocket-api.md +++ b/docs/research/ha-websocket-api.md @@ -53,8 +53,8 @@ the connector takes the URL and token as parameters. | `config/entity_registry/list_for_display` (preferred), `config/entity_registry/list`, `config/device_registry/list`, `config/area_registry/list` | – | setup wizard auto-detection (platform, device, area, category, translation key); verified, shapes in [Registries](#registries) | | `render_template` | `template` | last resort for integrations without actions | | `energy/get_prefs` | – | pre-fill from the Energy dashboard | -| `energy/info`, `energy/validate`, `energy/solar_forecast` | – | diagnostics; `energy/solar_forecast` returns forecasts of integrations that implement the energy platform (Forecast.Solar, Solcast) | -| `recorder/statistics_during_period` | `start_time`, `end_time?`, `statistic_ids[]`, `period` ∈ `5minute\|hour\|day\|week\|month\|year`, `types[]` ⊂ `change,last_reset,max,mean,min,state,sum`, `units` | load history for the load forecast, backtesting | +| `energy/info`, `energy/validate`, `energy/solar_forecast` | – | diagnostics; `energy/solar_forecast` returns forecasts of integrations that implement the energy platform (core: Forecast.Solar, Victron Remote Monitoring; HACS: Solcast, Open-Meteo Solar Forecast); only for config entries listed in the Energy dashboard's solar sources, see section `energy/solar_forecast` below | +| `recorder/statistics_during_period` | `start_time`, `end_time?`, `statistic_ids[]`, `period` ∈ `5minute\|hour\|day\|week\|month\|year`, `types[]` ⊂ `change,last_reset,max,mean,min,state,sum`, `units` | load history for the load forecast, backtesting; request/response verified, see section Long-term statistics below | | `recorder/list_statistic_ids` | `statistic_type?` | which sensors have statistics | | `recorder/get_statistics_metadata` | `statistic_ids?` | units | | `recorder/import_statistics` | `metadata{has_sum,name,source,statistic_id,unit_of_measurement,has_mean?,mean_type?,unit_class?}`, `stats[{start,mean?,min?,max?,state?,sum?,last_reset?}]` | optional: write savings / baseline series as external statistics (`source` must be a domain-like prefix, `statistic_id` `suncore:…`) – not admin-gated | @@ -269,11 +269,260 @@ target `entity_id`. Response per entity: `{"forecast": [...]}` with items `wind_bearing`, `wind_speed`, `wind_gust_speed`, `uv_index`, `is_daytime`. Values are converted to the user's unit system. +### Phase 4 verification (2026-10-16) + +Read from `home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1` +(2026.11 dev; tag `2026.10.2` = `8d7fd8b68677c2585be36ec86043308f403875b8`), +files `homeassistant/components/weather/{services.py,services.yaml,const.py,entity.py}`, +`homeassistant/helpers/service.py`. + +- **Name and registration** (`weather/services.py:60-74`): the only registered + action is **`weather.get_forecasts`** (`SERVICE_GET_FORECASTS = "get_forecasts"`, + `weather/const.py:122`). `services.yaml` still lists a stale singular + `get_forecast` entry; no code registers it. Schema: + `{Required("type"): In(("daily", "hourly", "twice_daily"))}`, target is the + entity (`required_features` = any of `FORECAST_DAILY`, `FORECAST_HOURLY`, + `FORECAST_TWICE_DAILY`), `supports_response=SupportsResponse.ONLY`, so + `return_response: true` is mandatory. +- **Per-type support** (`_async_get_forecasts_service`): asking for a type the + entity lacks raises `ServiceValidationError` (`unsupported_forecast_type`, + placeholders `entity_id`, `forecast_type`) -> WebSocket `service_validation_error`. + A `None` result from the entity becomes an empty list. +- **Response shape**: `entity_service_call` collects `{entity_id: result}` + (`helpers/service.py:807-811, 926`), so over WebSocket `call_service` the + result is `{"context": {...}, "response": {"weather.home": {"forecast": [ {...}, ... ]}}}`. + Each list item is the integration's forecast dict after `_convert_forecast` + (`weather/entity.py:608-`): native keys are folded into `temperature`, + `templow`, `apparent_temperature`, `dew_point`, `pressure`, `wind_speed`, + `wind_gust_speed`, `precipitation`, converted to the **user's** unit system and + rounded; `datetime` is passed through as the integration delivers it (ISO + string in practice, `ATTR_FORECAST_TIME = "datetime"`). `temperature` is set + to `null` when the entity delivered none. Keys the integration does not + provide are simply absent: the adapter must treat every key but `datetime` as + optional. The temperature unit is **not** in the response; read + `unit_of_measurement`-like info from the entity's `temperature_unit` state + attribute (not read in this pass, `UNVERIFIED`). +- Whether `hourly` is offered depends on the integration (`supported_features` + bit `FORECAST_HOURLY = 2`); not every weather entity has it. Fall back to + `daily` (a coarser temperature profile) or no temperature term. + ## Workday The `workday` binary sensor indicates whether the current day is a workday (holidays per country/region). The load forecast uses it to classify days. +### Phase 4 verification (2026-10-16) + +Same core commit; `homeassistant/components/workday/{binary_sensor.py,entity.py,services.py,services.yaml,const.py}`. + +- State: `IsWorkdaySensor.update_data` sets `_attr_is_on = date_is_workday(now)`; + **`on` = today is a workday, `off` = not** (weekend, public holiday or a + configured exclusion). The state is recomputed at local midnight + (`get_next_interval` = start of the next local day), not at HA start only. +- Attributes: `workdays` (list of weekday keys `mon`..`sun`, `holiday`), + `excludes` (same keys), `days_offset` (int). The sensor's offset shifts the + evaluated day (`days_offset`), so a sensor with a non-zero offset describes + another day than "today"; the adapter should require offset 0 or warn. +- **Action `workday.check_date`** (`services.py`): entity service on the + `binary_sensor` entities of the integration, field `check_date` (required, + `cv.date`, `YYYY-MM-DD`), `supports_response=ONLY`; response + **`{"": {"workday": true|false}}`** (`entity.py` + `check_date` returns `{"workday": bool}`, wrapped per entity id by the entity + service machinery). It applies the sensor's own `days_offset`, workdays and + excludes to the given date. This is enough to classify any past or future + date (also holidays) for the load profile without a separate holiday library. +- Weekend vs holiday cannot be told apart from the boolean; the load profile + has the three classes workday / weekend / holiday (plan WP6). To split + "off" into weekend and holiday, combine with `date.weekday()` locally: + `off` on Mon-Fri means holiday. `UNVERIFIED`: that a user's `workdays`/`excludes` + configuration matches the Mon-Fri assumption (the attributes `workdays` and + `excludes` allow checking it). + +## Long-term statistics (`recorder/...`) (Phase 4 verification 2026-10-16) + +Read from `home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1`: +`homeassistant/components/recorder/{websocket_api.py,statistics.py,models/statistics.py,__init__.py,purge.py}`, +`homeassistant/util/{unit_conversion.py,dt.py}`. + +**`recorder/statistics_during_period`** (`websocket_api.py:288-311`), not admin-gated: + +| Field | Type | Notes | +|---|---|---| +| `type` | `"recorder/statistics_during_period"` | | +| `start_time` | string, required | `dt_util.parse_datetime`; a value **without UTC offset is read in the HA time zone** (`dt_util.as_utc` assumes `DEFAULT_TIME_ZONE`); send ISO 8601 with offset. Invalid -> error `invalid_start_time` | +| `end_time` | string, optional | exclusive end; omitted = everything from `start_time` on; invalid -> `invalid_end_time` | +| `statistic_ids` | list of string, required, non-empty | for sensors equal to the entity id | +| `period` | `5minute` \| `hour` \| `day` \| `week` \| `month` \| `year`, required | `5minute` reads the short-term table, the others the long-term table; `day`/`week`/`month`/`year` are re-aggregated in **local** time, start/end are aligned to local midnight / Monday / 1st / 1 Jan | +| `types` | list ⊂ `change, last_reset, max, mean, min, state, sum`, optional | omitted = all seven. `change` is derived from `sum` (difference to the previous row) | +| `units` | object, optional | keys = unit class (e.g. `"power"`, `"energy"`, `"temperature"`), value = target unit from the class's `VALID_UNITS` (`PowerConverter`: `mW, W, kW, MW, GW, TW, BTU/h`; `EnergyConverter` includes `Wh, kWh, MWh`) | + +Response `result` (`_ws_get_statistics_during_period`): +`{ "": [ {"start": , "end": , "mean": f, "min": f, "max": f, "state": f, "sum": f, "change": f, "last_reset": |null}, ... ] }`. + +- `start` and `end` are **integer milliseconds since the Unix epoch** (UTC) of + the period (`end = start + 5 min / 1 h`; for day and longer the reduced + period). `last_reset` is also ms (or `null`). Only the requested `types` + appear as keys. Statistic ids without data in the range are **absent** from + the result object (not an empty list), and `{}` is returned if none has data. + A value can be `null` (e.g. `mean` of a period without samples). +- **Units.** The response carries **no unit field**. Without `units`, values + are converted to the entity's *current state unit* (`_get_statistic_to_display_unit_converter`: + requested unit if given, else the unit in the live state attributes, else the + unit stored with the statistic). A power sensor that the user switches from W + to kW therefore changes the numbers. SunCore must either pass + `units: {"power": "W", "energy": "Wh"}` explicitly (verified valid unit + strings) or read `statistics_unit_of_measurement`/`display_unit_of_measurement` + from `recorder/list_statistic_ids` first. Statistics whose `unit_class` is + `null` (unitless or unknown units) are never converted. +- **Meaning per sensor kind.** `mean`/`min`/`max` exist for `state_class: + measurement` sensors (power in W/kW, temperature); `state`, `sum` (and `change`, + `last_reset`) for `total`/`total_increasing` sensors (energy counters). The + `mean` of a power sensor is the arithmetic mean of the samples in the period + (`mean_type` `1` = arithmetic, `2` = circular, `0` = none). Which of the + columns are filled is given by `list_statistic_ids` (`mean_type`, `has_sum`). +- **Retention.** The `5minute` table is purged with the recorder's + `purge_keep_days` (default **10**, `recorder/__init__.py:109`); the long-term (`hour` and + coarser) rows are not touched by the purge code, which removes only + `statistics_runs` and short-term rows (`recorder/purge.py:93-110`). A 14-day history for the load profile must therefore use + `period: "hour"`, or `5minute` only for the last 10 days. `hour` is the finest + period beyond 10 days; sub-hourly profile slots (15 min) can come from + `5minute` for the recent days only. + +**`recorder/list_statistic_ids`** (`websocket_api.py:344-353`): optional +`statistic_type` ∈ `sum|mean`. Result: list of +`{"statistic_id", "display_unit_of_measurement", "has_mean", "mean_type", "has_sum", "name", "source", "statistics_unit_of_measurement", "unit_class"}` +(`statistics.py:1136-1154`; `has_mean` is marked for removal with 2026.4 in the +source comment, use `mean_type`). Includes statistics that integrations will add +in the next period. **`recorder/get_statistics_metadata`** (`:456`) with optional +`statistic_ids` returns the same metadata. `recorder/statistic_during_period` +(singular, one `statistic_id`, `types` ⊂ `max, mean, min, change`) exists as +well; not needed. + +Request example (verified shape): + +```json +{"id": 31, "type": "recorder/statistics_during_period", + "start_time": "2026-10-02T00:00:00+02:00", "end_time": "2026-10-16T00:00:00+02:00", + "statistic_ids": ["sensor.house_power"], "period": "hour", + "types": ["mean", "min", "max"], "units": {"power": "W"}} +``` + +`UNVERIFIED`: the power/energy sign and unit of the user's own house-consumption +sensor (user data); whether the Supervisor proxy forwards `recorder/*` commands +(first run on HA OS; `call_service` is known to be forwarded, other commands +are expected to be). + +## `energy/solar_forecast` (Phase 4 verification 2026-10-16) + +Read from `home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1`: +`homeassistant/components/energy/{websocket_api.py,types.py}`, +`homeassistant/components/forecast_solar/{energy.py,coordinator.py,manifest.json}`, +`homeassistant/components/victron_remote_monitoring/energy.py`; +`BJReplay/ha-solcast-solar` tag `v4.6.1` `custom_components/solcast_solar/{energy.py,forecast.py}`; +`rany2/ha-open-meteo-solar-forecast` tag `0.1.33` `custom_components/open_meteo_solar_forecast/energy.py`; +PyPI wheels `forecast-solar 5.0.1`, `open-meteo-solar-forecast 0.1.32`. + +- **Request**: `{"id": n, "type": "energy/solar_forecast"}`, no fields, no admin + requirement (`websocket_api.py:187-190`). +- **Which entries**: the command reads the Energy preferences + (`manager.data`); for every `energy_sources[]` entry of `type == "solar"` + it collects the ids in **`config_entry_solar_forecast`** and asks the + integration of each such config entry (`config_entry.domain` must have an + `energy` platform with `async_get_solar_forecast`). Consequences: (a) empty + result `{}` when the Energy dashboard was never saved, has no solar source or + the solar source has no forecast linked; (b) a forecast integration that is + installed but **not linked** in the Energy dashboard is invisible here; + (c) `energy/info` lists the available platform domains + (`solar_forecast_domains`, `:165`) - use it for auto-detection of what *could* + be linked. +- **Response**: `{ "": {"wh_hours": {"": , ...}}, ... }` + (`types.py:9-12`: `SolarForecastType = {wh_hours: dict[str, float | int]}`); + entries returning `None` are omitted. The key is the **config entry id** + (not the domain, not an entity id); map it to the domain via the config entry + or `energy/get_prefs`. +- **Semantics of `wh_hours` per provider** (the type is only "Wh per timestamp", + the period length differs): + - `forecast_solar` (`energy.py`): `wh_period` of the `forecast-solar` library, + Wh, ISO strings with the offset of the **API time zone** (the plant's + location), one entry per hour in the free tier; the library sums values + with `begin <= timestamp < end` for "current hour" + (`forecast-solar 5.0.1` wheel, `forecast_solar/models.py` `_interval_value_sum`), i.e. it treats the key as the *start* of + the period. Entries with value 0 are dropped except non-midnight ones + (`val != 0 or time != 00:00:00`). `UNVERIFIED`: whether the vendor API itself + labels `watt_hours_period` keys as period start or end (api.forecast.solar + unreachable); the HA library's own usage implies start. + - `open_meteo_solar_forecast` (HACS, `energy.py`): `wh_period`, hourly, key = + start of the hour (`open_meteo_solar_forecast.py:610-617` groups by + `time.replace(minute=0)`), value = mean power of the hour in W (numerically + Wh for that hour), no zero filter, ISO strings in the location's fixed + offset. + - `solcast_solar` (HACS, `energy.py` -> `forecast.py:945-`): `make_energy_dict` + emits **half-hourly** keys = `period_start`, value = + `pv_estimate{,10,90} (kW) * 500` = Wh per 30 min, rounded to integers, for + **one** confidence level selected by the integration's option + (`use_forecast_confidence`: estimate, 10 or 90), not all three. So no P10/P90 + band here; use the Solcast attributes for the band. + - `victron_remote_monitoring` (core): `data.get_dict_isoformat`, not analysed. +- **Resampling rule for SunCore**: sum Wh per key into 15-minute slots by + splitting each period's energy uniformly over its length; infer the length + from the key spacing (30 or 60 min), never assume it. Only the energy sum is + trustworthy; there is no unit field (`UNVERIFIED` only in the sense that the + type has none; all four providers above deliver Wh). +- **No quantiles** over this command in any provider. P10/P90 only from Solcast + attributes/`query_forecast_data`. + +## Modbus write action (Phase 4 verification 2026-10-16) + +Read from `home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1`: +`homeassistant/components/modbus/{services.py,services.yaml,modbus.py,const.py,strings.json,websocket_api.py,__init__.py}`, +`homeassistant/helpers/config_validation.py`. + +- **Actions** registered by `async_setup_services`: `modbus.reload`, + `modbus.write_coil`, `modbus.write_register`, `modbus.stop`. There is **no + read action**: reading is only possible through `modbus:` YAML sensors / + binary sensors (polled by the hub) which become normal HA entities. The WebSocket + command `modbus/connections/list` (admin only) lists connections and unit + ids, nothing else. +- **`modbus.write_register` fields** (`services.py:_write_service_schema`, + `services.yaml`): + + | Field | Type | Notes | + |---|---|---| + | `address` | int >= 0, required | register address; the schema has **no upper bound or 16-bit check** (`cv.positive_int`) | + | `value` | required | either a single non-negative int, or a list of non-negative ints; **each is one 16-bit register word as an unsigned int** (negative numbers are rejected by `NonNegative`, so signed values (S16) must be sent as two's complement `v & 0xFFFF`); a value above 65535 is not range-checked by HA and fails inside pymodbus | + | `slave` **or** `unit` | int >= 0, optional, mutually exclusive (`Exclusive(…, "unit")`) | Modbus device/unit id; `slave` is the documented field (`services.yaml` min 1 max 255), `unit` is an accepted alias (`ATTR_UNIT = "unit"`); default **1** when both are absent (`service.data.get(ATTR_SLAVE, service.data.get(ATTR_UNIT, 1))`). The hub's own `slave` config option is **not** used as default for the service | + | `hub` | string, default `modbus_hub` (`DEFAULT_HUB`) | the hub `name` from the YAML `modbus:` entry | + +- **Single vs multiple registers**: `value` as int -> function code 0x06 (write + single register, `write_register`); `value` as list -> 0x10 (write multiple + registers, `write_registers`) (`services.py:_async_write_register`). A list of + length 1 uses 0x10 too. +- **No float32 / typed values.** The action only writes raw register words. + SunCore must encode IEEE 754 float32 itself into two words and know the + device's word order. For the Kostal inverter (Float = 2 registers, byte order + register 5, see `kostal-plenticore-modbus.md`) that is the adapter's job; + e.g. `1038` (max charge power) gets `[hi, lo]` words as `value`. +- **Errors are not raised.** `ModbusHub.async_pb_call` -> `low_level_pb_call` + (`modbus.py:325-361`) catches `ModbusException`, a `None`/error response and + `result.isError()`, **logs the error and returns `None`**; the service returns + normally. An unknown hub raises `KeyError` inside the service (HA reports + `unknown_error`); Modbus not set up raises `ServiceValidationError` + `not_loaded`. A failed write is therefore **silent** to the caller: read-back + of the register (via a modbus sensor) or of the battery power is the only + confirmation. A client that is not connected makes `async_pb_call` return + `None` immediately (`if not self._client: return None`). +- **Concurrency**: all calls of a hub are serialised by a lock and an optional + `message_wait_milliseconds` pause (`modbus.py:363-378`). A heartbeat writer + (Kostal §3.5, every 20 s) shares the lock with all polled entities. +- **Admin**: `write_register` is registered with a plain `hass.services.async_register` + (no `async_register_admin_service`), so the Supervisor's user may call it; + `reload` is admin-only. +- `UNVERIFIED`: the exact exception HA returns for an unknown `hub` name (derived + from `hass.data[...][name]` lookup, not run); `modbus` hub setup over the UI + (the source shows YAML plus a new `connection.py` with `async_get_unit` for + config-entry based integrations; whether users can add a plain hub in the UI is + not established, the docs site is unreachable). + ## Write services and contexts (Phase 3) Verified on 2026-10-14 against `home-assistant/core` `dev` at commit @@ -422,6 +671,11 @@ Success result (**the result carries the context**): ``` With `return_response: true` (only for services that support it) `result.response` is added. +For entity services the response is keyed by entity id (`helpers/service.py` +`entity_service_call`, `response_data[entity.entity_id] = result`), e.g. +`{"weather.home": {"forecast": [...]}}`; for plain services it is the handler's +dict (`{"prices": {...}}`, `{"data": [...]}`). Phase 4 verification 2026-10-16, +core `5cfe9aad`. `Context.as_dict()` is `{"id", "parent_id", "user_id"}` (`core.py:1322-1346`; JSON encoder `helpers/json.py` uses `as_dict`). @@ -629,3 +883,38 @@ legacy `flow_from`/`flow_to` grid lists, which the parser should tolerate); `unknown_command` as the error code when the `energy` integration is not loaded; behaviour of the Supervisor Core proxy for these commands (first run on HA OS, same open item as `supported_features`). + +### Phase 4 verification (2026-10-16, WP1) + +Read on 2026-10-16 from `home-assistant/core` `dev` +`5cfe9aad7cb428c2c2ebb449719ae81976780de1` (raw.githubusercontent.com; the +doc sites and `api.github.com` are unreachable): `components/recorder/{websocket_api,statistics,purge,__init__,util}.py`, +`components/recorder/models/statistics.py`, `components/energy/{websocket_api,types}.py`, +`components/forecast_solar/{energy,coordinator,manifest.json}`, `components/victron_remote_monitoring/energy.py`, +`components/weather/{services.py,services.yaml,const.py,entity.py}`, +`components/workday/{binary_sensor,entity,services,const}.py`, +`components/modbus/{services.py,services.yaml,modbus.py,const.py,websocket_api.py,strings.json,__init__.py}`, +`components/{number,select,switch,event}/{services,entity}.py`, +`helpers/{service,config_validation}.py`, `util/{dt,unit_conversion}.py`. +HACS sources at tags: `BJReplay/ha-solcast-solar` `v4.6.1`, +`rany2/ha-open-meteo-solar-forecast` `0.1.33`. PyPI wheels `forecast-solar 5.0.1` +and `open-meteo-solar-forecast 0.1.32`. + +Resolved: the `recorder/statistics_during_period` request and response shape and +unit behaviour; `energy/solar_forecast` request, response and the `wh_hours` +key format and period length per provider; `weather.get_forecasts` name, +`supports_response` and response shape; the `workday` state meaning and +`workday.check_date`; `modbus.write_register` fields, multi-register form and +the absence of float32 and of a read action; `number.set_value` and +`select.select_option` payloads (unchanged from Phase 3, now with the dev +`probatio` schemas: `value` is `Coerce(float)`, `option` is a string; `number` +still range-checks against the entity's `min_value`/`max_value` and clamps to the +native range, `select` still validates against `options`); `switch.turn_on/off/toggle` +take no fields. + +Still `UNVERIFIED`: the Supervisor proxy forwarding of `recorder/*` and +`energy/*` commands; the temperature unit of a `weather.get_forecasts` response +(taken from the entity's attributes, not read); the vendor meaning of the +Forecast.Solar `watt_hours_period` keys (start vs end of the period, API +unreachable); the HA error for an unknown Modbus hub name; whether a plain Modbus +hub can be added in the UI. diff --git a/docs/research/integrations.md b/docs/research/integrations.md index 6315c61..fdfadd8 100644 --- a/docs/research/integrations.md +++ b/docs/research/integrations.md @@ -2,6 +2,11 @@ ## PV forecasts +Phase 4 verification (2026-10-16): every section below carries the facts read +from source in WP1 with the repository, tag or commit and path. Core is +`home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1` +(2026.11 dev). Quantities: power in kW/W as stated, energy in Wh, time zones as stated. + ### Solcast (HACS, `BJReplay/ha-solcast-solar`, Apache-2.0) Domain `solcast_solar`. Sensors: forecast today/tomorrow/day 3–7 (kWh), @@ -23,31 +28,184 @@ call per site per update. **SunCore never calls Solcast directly when the integration is present**; it reads attributes or calls the query action, which does not consume API calls. +**Phase 4 verification (2026-10-16)** – `BJReplay/ha-solcast-solar` tag `v4.6.1` +(commit `34e1d007e9d4a9b7a783520d723fcd05adb2ff9a`), `custom_components/solcast_solar/` +`{manifest.json,const.py,forecast.py,coordinator.py,sensor.py,actions.py,services.yaml, +energy.py,solcastapi.py,__init__.py,strings.json}`: + +- **Attributes** (`forecast.py` `get_forecast_day`, attached by + `coordinator.py` `get_sensor_extra_attributes`): on the day sensors + `total_kwh_forecast_today`, `total_kwh_forecast_tomorrow`, + `total_kwh_forecast_d3` … (names "Forecast Today", "Forecast Tomorrow", + "Forecast Day N", `strings.json`; `unique_id` = key). Keys (constants + `const.py:115,150-151,189-191,324`): `detailedForecast`, `detailedHourly`, + `dataCorrect`, `dayname`, `analysis`; entries + `{"period_start": , "pv_estimate": f, "pv_estimate10": f, "pv_estimate90": f}` + (`ESTIMATE`, `ESTIMATE10`, `ESTIMATE90`, `PERIOD_START`), with an extra + `dampening_factor` only when the advanced option asks for it + (`forecast.py:152-167`). The adapter must ignore unknown keys. +- **Unit and interval**: values are **average power in kW** over the 30-minute + slot (`energy_kwh = 0.5 * sum(kW)`, `forecast.py:290-291`); `detailedHourly` + is the mean of two slots per hour. `pv_estimate` = P50, `…10` = P10, + `…90` = P90. A sensor holds the **whole local day** (up to 48 slots; `dataCorrect` + is `false` when fewer than 48 slots exist, including DST days). +- **Time zone**: `period_start` is converted with `astimezone(self.api.tz)`; + `tz` is the **Home Assistant configured time zone** + (`__init__.py:120`: `dt_util.async_get_time_zone(hass.config.time_zone)`, UTC + fallback). Serialised by HA as ISO 8601 with the local offset. +- **Which attributes exist**: `detailedForecast` and `detailedHourly` are both + default **on** (`__init__.py:157-158`, options `attr_brk_halfhourly`, + `attr_brk_hourly`); the user can switch them off, then the adapter falls back to + the action. Per-site keys `detailedForecast_` only with the + "site breakdown" option. The attributes are excluded from the recorder + (`UNRECORDED_ATTRIBUTES`). +- **Number of day sensors**: advanced option `forecast_day_entities`, minimum 8, + maximum 14, default `DEFAULT_FORECAST_DAY_SENSORS = 8` (`const.py:138,537-542`); + the day sensors are `today`, `tomorrow` and `total_kwh_forecast_d3` … + `d{N-1}` (`coordinator.py:135-137`: `range(3, advanced_day_entities)`), so by + default **d3–d7** (resolves the earlier `UNVERIFIED` default). +- **Action `solcast_solar.query_forecast_data`** (`actions.py:370-373,397-424`, + `services.yaml:22-37`): `SupportsResponse.ONLY`; fields `start_date_time`, + `end_date_time` (both default to *now*, i.e. an empty/invalid range unless + given), `undampened` (bool, default false), `site` (resource id, `_` and `-` + accepted, default `all`). Response **`{"data": [ {"period_start", "pv_estimate", + "pv_estimate10", "pv_estimate90"}, ... ]}`**, `period_start` in the HA time + zone, 30-minute slots, kW. An invalid range raises `ServiceValidationError` + (`invalid_query_range`), an unknown site `not_a_site`. +- **Energy platform**: Solcast implements `energy.py` (`async_get_solar_forecast`) + with `wh_hours` at half-hour keys = `period_start`, value = `kW * 500` (Wh per + 30 min) for **one** confidence level chosen by the integration option, rounded to + integers (`forecast.py:945-`). No P10/P90 through that path. +- **Licence**: Apache-2.0 (`LICENSE` at the tag). + ### Forecast.Solar (Core) Sensors: energy today/remaining/tomorrow/this hour/next hour (kWh), power now (W), peak times; disabled by default power in 1/12/24 h. Updates hourly (free) or every 30 min (API key). Free tier: one plane, lower resolution. The page does not document series attributes; the integration implements the -`energy` platform, so `energy/solar_forecast` returns its hourly `wh_hours` -(UNVERIFIED field name – check `forecast_solar/energy.py` in Phase 4). +`energy` platform, so `energy/solar_forecast` returns its hourly `wh_hours`. + +**Phase 4 verification (2026-10-16)** – `home-assistant/core` `dev` `5cfe9aad…`, +`homeassistant/components/forecast_solar/{energy.py,coordinator.py,manifest.json}` +(requirement `forecast-solar==5.0.1`, domain `forecast_solar`, `iot_class cloud_polling`); +PyPI wheel `forecast-solar 5.0.1` `forecast_solar/models.py`: + +- `energy.py` (24 lines): `async_get_solar_forecast(hass, config_entry_id)` returns + `{"wh_hours": {timestamp.isoformat(): val for timestamp, val in + entry.runtime_data.data.wh_period.items() if val != 0 or + (timestamp.hour, timestamp.minute, timestamp.second) != (0, 0, 0)}}`; `None` + when the entry is unknown or not loaded. **Field name `wh_hours` verified.** +- `wh_period` = the API's `watt_hours_period` (`models.py` `Estimate.from_dict`): + an int Wh per period, keys are `datetime.fromisoformat` of the API response, + i.e. ISO strings with the offset of the plant's time zone. The library treats + a key as the **start** of the period (`_interval_value_sum` sums + `begin <= key < end`). Period length: hourly for the free tier; finer for paid + plans (`UNVERIFIED`, API docs unreachable). The free tier also needs a + rate-limit-aware refresh: the coordinator polls hourly, 30 min with an API + key (`coordinator.py:79-82`). +- The attribute route does not exist; no quantiles. `energy/solar_forecast` + only lists the entry when the user linked it in the Energy dashboard + (`config_entry_solar_forecast`). ### Open-Meteo Solar Forecast (HACS, `rany2/ha-open-meteo-solar-forecast`, Apache-2.0) -Domain presumably `open_meteo_solar_forecast` (UNVERIFIED). Per-array -declination, azimuth (0 = north), module power, DC efficiency, inverter -limit, damping, horizon shading. Attribute structure not documented in the -README → read the source in Phase 4. +Per-array declination, azimuth, module power, DC efficiency, inverter +limit, damping, horizon shading. + +**Phase 4 verification (2026-10-16)** – tag `0.1.33` (commit +`7af5d681c3ffeb7d8f678fd4eadd7f5aebd2a1ac`), `custom_components/open_meteo_solar_forecast/` +`{manifest.json,const.py,sensor.py,coordinator.py,energy.py,strings.json,services.yaml}`, +README; PyPI wheel `open-meteo-solar-forecast 0.1.32`: + +- **Domain `open_meteo_solar_forecast`** (`manifest.json`, `const.py`), config + flow, `iot_class cloud_polling`, update interval 30 min (`coordinator.py:298`), + requirement `open_meteo_solar_forecast==0.1.32`. Service device "Solar + production forecast", manufacturer "Open-Meteo". The only action is + `update_array_location`. +- **Sensors** (`sensor.py`): `energy_production_today`, `…_today_remaining`, + `…_tomorrow`, `…_d2` … `…_d7` (Wh, `suggested_unit` kWh), `energy_current_hour`, + `energy_next_hour`, `power_production_now` (W), `power_production_next_15minutes`, + `…_30minutes`, `…_hour`, `…_12hours`, `…_24hours` (W, **disabled by default**), + `power_highest_peak_time_today/tomorrow` (timestamp). `unique_id` = + `_`; the entity id is forced to + `sensor._` (`sensor.py` `self.entity_id = …`). +- **Series attributes** (`sensor.py` `extra_state_attributes`), on + `energy_production_today`, `energy_production_tomorrow` and + `energy_production_d2` … `d7` only, restricted to that local date: + - `watts`: `{iso_timestamp: W}` (int; instantaneous power), + - `wh_period`: `{iso_hour_start: Wh}` (hourly; mean 15-min power over the hour, + numerically Wh of that hour), + - `wh_period_15m`: `{iso_interval_start: Wh}` (float, `W × 0.25` per half-open + interval `[t, t+15 min)`, inverter-clipped). + Constants `ATTR_WATTS`, `ATTR_WH_PERIOD`, `ATTR_WH_PERIOD_15M` (`const.py`). Keys + are ISO 8601 strings with a fixed UTC offset of the location. **Best series for + SunCore: `wh_period_15m`** (native 15-minute grid, energy per interval). + A single deterministic estimate: **no P10/P90**. +- **Energy platform**: `energy.py` implements `async_get_solar_forecast`, returning + `{"wh_hours": {ts.isoformat(): wh_period value}}` (hourly), so the forecast is + also available through `energy/solar_forecast` once linked in the Energy dashboard. +- **Azimuth in the integration**: UI 0–360 with 0 = North, 90 = East, 180 = South, + 270 = West (README, `strings.json`); the coordinator subtracts 180 before calling + the library (`coordinator.py:230-233`), which uses the Open-Meteo API convention + (see the built-in fallback section). ### Built-in fallback: Open-Meteo API + plant model Open-Meteo: free for non-commercial and open-source use, fair use, contact above 10 000 requests/day; data CC BY 4.0 (attribution "Weather data by Open-Meteo.com" must be shown in the UI); code AGPL-3.0 (irrelevant, we only -call the API). Variable names for tilted irradiance -(`global_tilted_irradiance`, `tilt`, `azimuth`) are UNVERIFIED in this pass -(docs site unreachable; the repo's `openapi/` folder exists). Verify in -Phase 4 before implementing. +call the API). + +**Phase 4 verification (2026-10-16)** – `open-meteo/open-meteo` `main` +`01ceed5b028fb6ac19794c3310aed5a4f7aa6d4a` (`version.txt` `1.7.0`): +`openapi/forecast.yml`, `Sources/App/Helper/Solar/GlobalTilitedIrradiance.swift`, +`Sources/App/Controllers/VariableHourly.swift:1754,1766`, +`Sources/App/Helper/Reader/GenericReaderDerived.swift:54`, `README.md`, `LICENSE`. +`api.open-meteo.com` and `open-meteo.com` are **not reachable** from the session +(proxy `CONNECT 403`), so nothing was tested against the live API. + +- **Parameters exist**: query parameters `tilt` (float, default 0) and `azimuth` + (float, default 0) in `openapi/forecast.yml:574-587`; variables + `global_tilted_irradiance` and `global_tilted_irradiance_instant` are listed for + `hourly` (`:188,194`) and for `minutely_15`; `forecast_days` 0–16, default 7 + (`:543-551`). +- **Azimuth convention – the repo contradicts itself.** The OpenAPI text says + "North=0, East=90, South=180, West=270" (`forecast.yml:585`), but the + implementation documents and uses **0° = south, −90° = east, +90° = west** + (`GlobalTilitedIrradiance.swift:12`: "Azimuth 0° south, -90° = east, +90° = west. + Tilt should be from 0° horizontal to 90° vertical"), the value is passed + through unchanged (`VariableHourly.swift:1754: azimuth: options.azimuth`; + `GenericReaderDerived.swift:54`: `azimuth ?? 0`), and the maintained client + `open-meteo-solar-forecast 0.1.32` states the same and rejects values outside + [-180, 180] ("Open-Meteo convention: 0 = South, -90 = East, 90 = West, +-180 = + North. To convert a compass bearing … subtract 180"). **SunCore uses the source + convention: `api_azimuth = compass_bearing - 180`** (compass: N 0, E 90, S 180, + W 270), tilt 0 = flat, 90 = vertical. `NaN` = tracking axis (client sends + `"nan"`). The OpenAPI description is considered wrong; a live check with a + south-facing and a west-facing array on a clear day stays a Phase 4 acceptance + item (`UNVERIFIED` against the live API). +- **Time semantics**: radiation variables are **backward averages** + ("Direct and diffuse radiation must be backwards averaged", Swift doc comment); + a `minutely_15` timestamp is the *end* of the interval, and the client moves it + back by 15 minutes (`time_start = time - 15 min`, + `open_meteo_solar_forecast.py:557`). `*_instant` variants are instantaneous. + Request used by the client: `minutely_15=temperature_2m,global_tilted_irradiance, + global_tilted_irradiance_instant,diffuse_radiation,…,snow_depth`, + `timezone=auto`, `timeformat=unixtime`, `forecast_days`, `past_days`. +- **Terms and licence (README, `main` at the SHA above)**: "Open-Meteo APIs are + free for open-source developers and non-commercial use. We do not restrict + access, but ask for fair use. If your application exceeds 10'000 requests per + day, please contact us. We reserve the right to block applications and IP + addresses that misuse our service. For commercial use … please contact us." + Data: "Attribution 4.0 International license (CC BY 4.0)"; attribution link next + to every place the data is displayed, e.g. ` + Weather data by Open-Meteo.com`. Source code AGPL-3.0 (`LICENSE`). The + separate terms page (`open-meteo.com/en/terms`) is unreachable and stays + `UNVERIFIED`; the README text is the verified part. +- **Consequence**: the parameters and licence are verified from the source; the + live response was not seen. Per plan question 5 the fallback may ship only as an + explicit opt-in (off by default), with the attribution link in the UI. ## Prices @@ -62,6 +220,18 @@ No `price_level` in the response. Whether `price` includes taxes is not stated on the page → the wizard asks (BRIEF 4.3 "includes fees?") and shows a live comparison with the total-price sensor. +**Phase 4 verification (2026-10-16)** – re-read `home-assistant/core` `dev` +`5cfe9aad…` `components/tibber/services.py` (152 lines): unchanged from +[`device-integrations.md`](device-integrations.md). Action `tibber.get_prices`, +`SupportsResponse.ONLY`, fields `start`/`end` (optional strings, `parse_datetime`, +local time), response `{"prices": {"": [{"start_time": "", +"price": }]}}`, items filtered `start <= t < end`, `price` = `home.price_total` +(**energy plus tax**, per the library's `total`), default window = local today, +`start >= end` -> `end = start + 1 day`, refetch when the cache lacks the next day +after 13:00 local. All matches the earlier note; the price includes taxes +(`price_total`), which answers plan question 9 for the action (a supplier +contract with additional fees stays the user's call). + ### Nord Pool (Core) Sensors current/previous/next price, daily average/min/max, peak and @@ -72,12 +242,67 @@ Automatically switches to 15-minute market time units; mixed 60/15-minute series are possible during the transition. `nordpool.get_price_indices_for_date` exists but is undocumented on the page. +**Phase 4 verification (2026-10-16)** – `home-assistant/core` `dev` `5cfe9aad…` +`components/nordpool/{services.py,services.yaml,sensor.py,coordinator.py,strings.json,manifest.json}` +(`pynordpool==0.4.0`, PyPI wheel read: `__init__.py`, `util.py`, `model.py`): + +- **Action `nordpool.get_prices_for_date`** (`services.py:62-70,168-175`): + `SupportsResponse.ONLY`; fields `config_entry` (required, config entry id), + `date` (required, `cv.date` `YYYY-MM-DD`; "only dates from two months in the + past to one day in the future", `strings.json`), `areas` (optional list, + upper-cased and checked against the library's `AREAS`; **empty = the entry's + configured areas**), `currency` (optional, `Currency` enum value, upper-cased; + default = the entry's currency). `get_price_indices_for_date` is the same plus + `resolution` ∈ 15/30/60 (default 60). +- **Response**: `{ "": [ {"start": "", "end": "", "price": }, ... ] }` + for each requested area (`services.py:140-150`). `start`/`end` are + `datetime.isoformat()` of datetimes parsed with `fromisoformat` from the API's + `…Z` strings, i.e. **UTC with `+00:00`**. `price` is the raw `entryPerArea[area]` + value of the Nord Pool day-ahead API. An empty response returns `{area: []}`; + authentication/connection errors raise `ServiceValidationError` + (`authentication_error` / `connection_error`). +- **Unit**: **currency per MWh** – the integration's own sensors divide every + such value by 1000 and label the result `"/kWh"` (`sensor.py:38,205-243,347,413`). + The action returns the undivided MWh value; the adapter divides by 1000. + The values are the day-ahead market price per area; VAT, grid fees and levies are not part of them (no such field in the source). +- **Resolution**: slot length = `end - start` per entry (the sensors compute + `resolution = entry.end - entry.start`, `sensor.py:58`), 15 or 60 minutes; a + coordinator tick every 15 minutes (`coordinator.py:57-73`). The adapter must not + assume a fixed length; a series may mix lengths across the transition date. +- The action only returns what the Nord Pool day-ahead call has for that delivery + date (today; tomorrow after publication, `UNVERIFIED` exact time, afternoon CET + expected). + ### ENTSO-E (HACS, `JaccoR/hass-entso-e`, Apache-2.0) Day-ahead prices for European zones; sensors current/next/avg/min/max with a price list attribute; optional template for fees/VAT; needs a free Transparency Platform API key (requested by e-mail). +**Phase 4 verification (2026-10-16)** – tag `v0.7.5` (commit +`cbdf67ac0e73fd685a02e27ad7b6bef2c5af7ca6`), `custom_components/entsoe/{manifest.json, +sensor.py,coordinator.py,const.py,api_client.py}`: + +- Domain `entsoe`, `iot_class cloud_polling`; sensors (`sensor.py:50-111`) keys + `current_price`, `next_hour_price`, `min_price`, `max_price`, `avg_price`, + `percentage_of_max`, `percentage_of_range`, `highest_price_time_today`, + `lowest_price_time_today`; `unique_id` + `entsoe.` (or `entsoe._`). +- **The price list sits only on the `avg_price` sensor** ("Average electricity + price", `sensor.py:269-279`), attributes **`prices_today`**, **`prices_tomorrow`**, + **`prices`** (today+tomorrow, or yesterday+today when only two days exist). + Entry shape (`coordinator.py:250-256`): `{"time": "", "price": }`; + `time` is `str()` of a timezone-aware **local** datetime, i.e. + `"2026-10-16 00:00:00+02:00"` (space separator, not `T`; `fromisoformat` + accepts it), slot **start**. Slot length is the configured `period` + (`PT60M` default or `PT15M`, `const.py:22,33`) and is not in the entry; infer + it from the spacing. +- **Unit**: `"/"` with `energy_scale` ∈ `kWh` (default) or + `MWh` (`const.py:21,32`, `sensor.py:57`). **The price already includes the + user's modifier template and VAT** (`coordinator.py:calc_price`: template result + `* (1 + vat)`), so it can be market price or end price depending on the + user's template; the wizard must ask. + ### EPEX Spot (HACS, `mampfes/ha_epex_spot`, MIT) Sources: aWATTar, Energyforecast.de, SMARD.de, smartENERGY.at, Tibber, @@ -85,10 +310,64 @@ ENTSO-E, Energy-Charts.info, Hofer Grünstrom. Total-price sensor `data` attribute: `[{start_time, end_time, price_per_kwh}]`; also market price, average, median, min, max, quantile, rank sensors. +**Phase 4 verification (2026-10-16)** – tag `4.1.0` (commit +`2aad1eb796f9ae8b1607f5c3d33198fcc454e980`), `custom_components/epex_spot/{manifest.json, +sensor.py,const.py,localization.py,SourceShell.py,common.py,EPEXSpot/…}`, README: + +- Domain `epex_spot`. Sensors "Total Price" (`key="Total Price"`) and "Market + Price" (`key="MarketPrice"`) both expose the attribute **`data`**: list of + `{"start_time": "", "end_time": "", + "price_per_kwh": }` (`const.py:6-8,14`; `sensor.py:69-82,107-119`; + `dt_util.as_local(...).isoformat()`). The price key is `price_per_kwh` for EUR + **and** GBP (`localization.py`). +- **Unit**: `"€/kWh"` (the literal euro sign, `const.py:57`) or `"£/kWh"`, not + `EUR/kWh`. Slot length = `end_time - start_time`; configurable per entry + (`duration`, default 60, Energy-Charts accepts 15 or 60). +- **Total vs market**: Total = market + percentage surcharge, then + absolute + surcharge, then × (1 + VAT) (`SourceShell.to_total_price`), except for the Tibber + source, which is returned as is. Market price excludes all of that. The + `data` of the total sensor is a **user-configured all-in price**; of the market + sensor the raw market price (kWh, converted from MWh by the source clients). +- `unique_id` = ` `. Licence MIT (`LICENSE`). + ### Octopus Energy (HACS, `BottlecapDave/HomeAssistant-OctopusEnergy`, MIT) Electricity/gas rate sensors with rate information and intelligent-tariff -controls. Attribute names (`rates` list) UNVERIFIED – Phase 4. +controls. + +**Phase 4 verification (2026-10-16)** – tag `v19.2.1` (commit +`ccfbccc3a6e6b75d48a955501afd232d9a9f23f7`), `custom_components/octopus_energy/` +`{manifest.json,const.py,electricity/{current_rate,rates_current_day,rates_next_day}.py, +coordinators/{__init__,electricity_rates}.py,utils/{__init__,conversions}.py}`; +core `components/event/entity.py`: + +- Domain `octopus_energy`. **Current rate sensor** `electricity/current_rate.py`: + state `value_inc_vat` in **GBP/kWh** (`native_unit_of_measurement "GBP/kWh"`, + pounds, rounded 6 decimals from pence), attributes `mpan`, `serial_number`, + `is_export`, `is_smart_meter`, `tariff`, `start`, `end`, `is_capped`, + `is_intelligent_adjusted`, `current_day_min_rate`, `current_day_max_rate`, + `current_day_average_rate`; `unique_id` + `octopus_energy_electricity__[_export]_current_rate`. No rate list + on that sensor. +- **The rate lists are on event entities** `rates_current_day.py` / + `rates_next_day.py` / `rates_previous_day.py` (`unique_id` + `…_current_day_rates`, `…_next_day_rates`, `…_previous_day_rates`). Core's + `EventEntity.state_attributes` returns `{"event_type": …} | last_event_attributes` + (`event/entity.py:145-152`), so the last event's data appear as attributes: + **`rates`** = list of `{"start": , "end": , + "value_inc_vat": [, "is_capped": bool][, "is_intelligent_adjusted": bool]}` + (`utils/__init__.py:149-170` `private_rates_to_public_rates`), plus + `min_rate`, `max_rate`, `average_rate`, `mpan`, `serial_number`, `tariff_code` + (`coordinators/__init__.py:77`). The same payload is also fired on the bus as + `octopus_energy_electricity_current_day_rates` / `…_next_day_rates` / + `…_previous_day_rates` (`const.py:134-136`). The event only fires **when the + rates changed** (`has_rates_changed`), and the entity restores the last event + after a restart. +- Slot length is the rate's `end - start` (30 minutes for the usual Octopus + tariffs; not read from a tariff list). Export rates have their own + `…_export_…` entities (`_export_id_addition`). +- Mapping consequence: SunCore reads the `rates` attribute of the current-day and + next-day event entities (merge by `start`), unit GBP/kWh incl. VAT. ### Built-in fallback: Energy-Charts / aWATTar @@ -96,8 +375,28 @@ Both APIs were unreachable from this session. `api.energy-charts.info` `/price` (bidding zone, start, end → `unix_seconds[]`, `price[]`, `unit`, `license_info`) and `api.awattar.de/v1/marketdata` (`data[{start_timestamp, end_timestamp, marketprice, unit}]`) are UNVERIFIED including their terms of -use. Phase 4 verifies before shipping; the EPEX Spot integration above is a -working alternative in the meantime. +use. + +**Phase 4 verification (2026-10-16)** – direct access: `curl` through the agent +proxy to `api.energy-charts.info`, `www.energy-charts.info`, `api.awattar.de`, +`api.awattar.at` returned `000` (CONNECT refused); **no API response and no terms +page could be read**. Secondary evidence from the clients in `mampfes/ha_epex_spot` +tag `4.1.0` (commit `2aad1eb796f9ae8b1607f5c3d33198fcc454e980`), which parse the +real responses: + +- Energy-Charts (`EPEXSpot/EnergyCharts/__init__.py`): `GET https://api.energy-charts.info/price` + with query `bzn` (e.g. `DE-LU`), `start`, `end` (ISO dates); JSON arrays + `unix_seconds[]` (slot **start**, UTC seconds), `price[]` (**EUR/MWh**, divided by + 1000 by the client), `unit` (default `"EUR/MWh"` in the client). Slot length from + the spacing of `unix_seconds` (15 or 60 minutes). `license_info` is **not** + read by the client; presence in the response `UNVERIFIED`. +- aWATTar (`EPEXSpot/Awattar/__init__.py`): `GET https://api.awattar.{de|at}/v1/marketdata` + with `start`, `end` as epoch **milliseconds**; JSON `data[]` with + `start_timestamp`, `end_timestamp` (ms), `marketprice` (**EUR/MWh**), `unit` + (the client asserts `"Eur/MWh"` case-insensitively). Hourly slots. +- **Terms of use, licence, rate limits, attribution of both APIs: `UNVERIFIED`.** + Per plan question 5 the fallbacks are therefore **not shipped in Phase 4**; the + HACS EPEX Spot integration (which uses these sources) is the supported route. ## Consumers @@ -123,8 +422,8 @@ Entities: numbers `Battery:MinSoc` (5–100 %, step 5), usage mode" (`None` | `Battery:SmartBatteryControl:Enable` | `Battery:TimeControl:Enable`); switches battery strategy, shadow management, manual charge (installer only). Process data (disabled by default): grid -power (+ import), home power by source, battery power (+ discharge, − -charge), SoC, PV/DC per string, energy totals. **No charge/discharge power +power (+ import), home power by source, battery power (+ discharge, − charge), +SoC, PV/DC per string, energy totals. **No charge/discharge power set-point over REST** – active control needs Modbus (see `kostal-plenticore-modbus.md`). Useful for SunCore: min-SoC and min-home- consumption numbers as a "soft" battery adapter without installer access, @@ -132,6 +431,58 @@ and `ActivePowerLimitation` for export limiting (sign/meaning UNVERIFIED – the HA number is 0–10000 W, the maintainer's inverter is 17 kW; check the inverter's own limit semantics in Phase 4). +**Phase 4 verification (2026-10-16)** – `home-assistant/core` `dev` `5cfe9aad…` +`components/kostal_plenticore/{manifest.json,number.py,select.py,switch.py,sensor.py,coordinator.py}` +(`pykoplenti==1.5.0`, `iot_class local_polling`, domain `kostal_plenticore`): + +- **Number entities** (`number.py:35-85`, class `PlenticoreDataNumber`): described + by `key`, `module_id`, `data_id`; **no `translation_key`**; `entity_category + config`; **all three `entity_registry_enabled_default=False`** (the user must + enable them); `mode box`; `unique_id = "__"`, + `name = " "`. + + | `key` | Name | `module_id` / `data_id` | Unit | Default min / max / step | + |---|---|---|---|---| + | `battery_min_soc` | Battery min SoC | `devices:local` / `Battery:MinSoc` | % | 5 / 100 / 5 | + | `battery_min_home_consumption` | Battery min Home Consumption | `devices:local` / `Battery:MinHomeComsumption` | W (power) | 50 / 38000 / 1 | + | `active_power_limitation` | Active Power Limitation | `devices:local` / `Inverter:ActivePowerLimitation` | W (power) | 0 / 10000 / 1 | + + `min`/`max` are then **overwritten with the values the inverter reports for the + setting** (`SettingsData.min/max`, `number.py:150-153`), so read the `min`/`max` + attributes of the live entity. The entity is created only when the inverter + lists the setting. The state is polled every **30 s** (`SettingDataUpdateCoordinator`). + `step` is informational (core does not enforce it, see Phase 3 notes). +- **Writes go to the inverter immediately**: `async_set_native_value` -> + `coordinator.async_write_data(module_id, {data_id: value})` -> + `client.set_setting_values` (REST), then `async_refresh()`. **A failed write is + silent**: `async_write_data` catches `ApiException` and returns `False` + (`coordinator.py:152-166`), the caller ignores the result, so + `number.set_value` succeeds from HA's point of view. Read back the entity (after + the forced refresh) before believing a write. +- **Select "Battery Charging / Usage mode"** (`select.py:27-37`): + `key="battery_charge"`, `module_id devices:local`, options exactly + `["None", "Battery:SmartBatteryControl:Enable", "Battery:TimeControl:Enable"]`, + `entity_category config`, enabled by default, `unique_id "_devices:local"`; + created only if both Enable settings exist on the inverter. `select_option` + writes `"0"` to **both** Enable settings first, then `"1"` to the chosen one + (`select.py:123-131`; not atomic) and then only calls `async_write_ha_state()` + **without a refresh**: the state shown for up to 30 s (poll interval) is the + *old* coordinator value. Read-back needs a wait of at least one poll interval + (or `homeassistant.update_entity`). +- **Switches** (`switch.py:35-58`): `Battery:Strategy` (`1` = "Automatic", `2` = + "Automatic economical"), `Battery:ManualCharge` (installer only; `1`/`0`). +- **Process data for SoC and power** (sensors poll every **10 s**, + `sensor.py:813`; `unique_id "__"`): `devices:local:battery` + / `SoC` ("Battery SoC", %, device_class battery), `devices:local:battery` / `P` + ("Battery Power", W, power, measurement), `devices:local:battery` / `Cycles`, + `devices:local` / `HomeBat_P` ("Home Power from Battery"), `PV2Bat_P` ("PV to + Battery Power"), `Grid_P` ("Grid Power"), `HomeGrid_P`. The HA source gives **no + sign comment** for `Battery Power` / `Grid Power`; the earlier note + "+ discharge, − charge" is carried over and stays `UNVERIFIED` (confirm on the + live entity before relying on it). +- `Inverter:ActivePowerLimitation` semantics (limit in W of what? absolute AC + limit or percent-like) are not in the HA source: still `UNVERIFIED`. + ### SunSpec (HACS, `CJNE/ha-sunspec`, MIT) Reads SunSpec models over Modbus TCP, sensors only, no writes. This is what @@ -148,7 +499,61 @@ kostal_plenticore/{number,select}.py}`; `rany2/ha-open-meteo-solar-forecast`, `BottlecapDave/HomeAssistant-OctopusEnergy`, `CJNE/ha-sunspec` READMEs; `open-meteo/open-meteo` README. -`UNVERIFIED`: Forecast.Solar series field names, Open-Meteo Solar Forecast -attributes and domain, Open-Meteo tilted-irradiance parameters, Octopus -attribute names, Energy-Charts and aWATTar APIs and terms, semantics of -`Inverter:ActivePowerLimitation`. +**Phase 4 verification (2026-10-16, WP1).** All reads from GitHub raw files and +git clones (`raw.githubusercontent.com`, `git clone` of github.com) plus PyPI +wheels; `api.github.com`, the documentation sites, `open-meteo.com`, +`api.energy-charts.info` and `api.awattar.de` were unreachable (proxy `CONNECT 403`). +Pinned sources: + +| Repository | Ref (SHA) | +|---|---| +| `home-assistant/core` | `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1` (tag `2026.10.2` = `8d7fd8b68677c2585be36ec86043308f403875b8`) | +| `BJReplay/ha-solcast-solar` | tag `v4.6.1` `34e1d007e9d4a9b7a783520d723fcd05adb2ff9a` | +| `rany2/ha-open-meteo-solar-forecast` | tag `0.1.33` `7af5d681c3ffeb7d8f678fd4eadd7f5aebd2a1ac` | +| `JaccoR/hass-entso-e` | tag `v0.7.5` `cbdf67ac0e73fd685a02e27ad7b6bef2c5af7ca6` | +| `mampfes/ha_epex_spot` | tag `4.1.0` (commit `2aad1eb796f9ae8b1607f5c3d33198fcc454e980`) | +| `BottlecapDave/HomeAssistant-OctopusEnergy` | tag `v19.2.1` `ccfbccc3a6e6b75d48a955501afd232d9a9f23f7` | +| `open-meteo/open-meteo` | `main` `01ceed5b028fb6ac19794c3310aed5a4f7aa6d4a` (`version.txt` 1.7.0) | +| PyPI | `forecast-solar 5.0.1`, `open-meteo-solar-forecast 0.1.32`, `pynordpool 0.4.0` | + +Resolved in this pass (earlier `UNVERIFIED` removed): Forecast.Solar series field +name (`wh_hours`); Open-Meteo Solar Forecast domain and attributes; Open-Meteo +`tilt`/`azimuth`/`global_tilted_irradiance` parameters and licence/terms +text (README); Octopus `rates` attribute names; ENTSO-E and EPEX Spot attribute +names; Nord Pool action shape and unit; Solcast default number of day sensors; +Kostal number/select/switch definitions and the immediate-write behaviour. + +Still `UNVERIFIED`: the live Open-Meteo API response and its `azimuth` convention +against the contradictory OpenAPI text (source says 0 = south); Open-Meteo's +separate terms page; Energy-Charts and aWATTar API behaviour, `license_info`, terms +and rate limits (only the field names used by `ha_epex_spot` are known); the vendor +meaning (start vs end) of Forecast.Solar `watt_hours_period` keys and its +sub-hourly periods on paid plans; the sign of Kostal `Battery Power` / +`Grid Power` sensors; the semantics of `Inverter:ActivePowerLimitation`; the +exact publication time of tomorrow's day-ahead prices (Tibber, Nord Pool); live +entity ids of all integrations (derived from code only). + +## Summary: Phase 4 interfaces and consequences for the adapters + +| Interface | Verified? | Source (repo@ref path) | Consequence for the adapter | +|---|---|---|---| +| `recorder/statistics_during_period` | yes | `core@5cfe9aad` `components/recorder/websocket_api.py:288-311`, `statistics.py` | Send ISO 8601 with offset, `period: "hour"` for >10 days (`5minute` only 10 days), pass `units: {"power":"W","energy":"Wh"}` (response has no unit; default = entity state unit), `start`/`end` are epoch ms, ids without data are absent | +| `recorder/list_statistic_ids` | yes | `core@5cfe9aad` `recorder/websocket_api.py:344`, `statistics.py:1136` | Use `display_unit_of_measurement`, `mean_type`, `has_sum` to choose `types` | +| `energy/solar_forecast` | yes | `core@5cfe9aad` `components/energy/{websocket_api,types}.py`; `forecast_solar/energy.py` | Result keyed by config entry id, `{wh_hours: {iso: Wh}}`; only linked entries; period length per provider (30 or 60 min) from key spacing; no P10/P90 | +| `weather.get_forecasts` | yes | `core@5cfe9aad` `components/weather/{services.py,const.py,entity.py}` | `type` required, response `{entity: {forecast: [...]}}`, only `datetime` guaranteed, fall back hourly -> daily, check `supported_features` | +| `workday` binary sensor + `workday.check_date` | yes | `core@5cfe9aad` `components/workday/{binary_sensor,entity,services}.py` | `on` = workday; `check_date` returns `{entity: {"workday": bool}}`; offset attribute must be 0 | +| `number.set_value`, `select.select_option`, `switch.turn_on/off` | yes | `core@5cfe9aad` `components/{number,select,switch}/services.py` | Unchanged semantics (range check, option check); no step enforcement | +| `modbus.write_register` | yes | `core@5cfe9aad` `components/modbus/{services.py,services.yaml,modbus.py}` | Fields `hub`, `slave`|`unit` (default 1), `address`, `value` int or list of unsigned 16-bit words; no float32 (encode two words), no read action, **errors only logged -> read back every write** | +| `nordpool.get_prices_for_date` | yes | `core@5cfe9aad` `components/nordpool/services.py`; `pynordpool 0.4.0` | Response `{AREA: [{start,end,price}]}`, UTC `+00:00`, price in currency/MWh (divide by 1000), excl. VAT/fees, 15 or 60 min per entry | +| `tibber.get_prices` | yes (re-read) | `core@5cfe9aad` `components/tibber/services.py` | `{"prices": {home: [{start_time, price}]}}`, price incl. taxes, call with `return_response` | +| `kostal_plenticore` numbers/select | yes | `core@5cfe9aad` `components/kostal_plenticore/{number,select,coordinator}.py` | Numbers disabled by default, no translation key, ids by `unique_id` suffix `devices:local_Battery:MinSoc`; limits from live `min`/`max`; failed REST write is silent; select read-back lags 30 s; no charge set-point | +| Kostal SoC / power sensors | partly | `core@5cfe9aad` `components/kostal_plenticore/sensor.py` | `devices:local:battery` `SoC` and `P`; sign of `P` UNVERIFIED -> confirm on the live entity | +| Solcast attributes + `query_forecast_data` | yes | `BJReplay/ha-solcast-solar@v4.6.1` `forecast.py`, `actions.py`, `const.py` | `detailedForecast`/`detailedHourly` entries `{period_start, pv_estimate, pv_estimate10, pv_estimate90}`, kW, 30 min, HA time zone; action `{"data": [...]}` as fallback; day sensors today, tomorrow, d3-d7 | +| Open-Meteo Solar Forecast | yes | `rany2/ha-open-meteo-solar-forecast@0.1.33` `sensor.py`, `const.py`, `energy.py` | Domain `open_meteo_solar_forecast`; attributes `watts`, `wh_period`, `wh_period_15m` on `energy_production_{today,tomorrow,d2..d7}`; use `wh_period_15m`; single estimate (no quantiles) | +| ENTSO-E | yes | `JaccoR/hass-entso-e@v0.7.5` `sensor.py:269`, `coordinator.py:250` | Attribute `prices`/`prices_today`/`prices_tomorrow` on the `avg_price` sensor, entries `{time, price}`, local `str(datetime)`, currency/kWh or MWh, may include the user's template and VAT | +| EPEX Spot | yes | `mampfes/ha_epex_spot@4.1.0` `sensor.py`, `const.py`, `SourceShell.py` | Attribute `data`: `[{start_time, end_time, price_per_kwh}]` on "Total Price" (all-in, user surcharges) and "Market Price"; unit literal `€/kWh`; slot length from `end - start` | +| Octopus Energy rates | yes | `BottlecapDave/HomeAssistant-OctopusEnergy@v19.2.1` `electricity/rates_current_day.py`, `utils/__init__.py:149` | Event entities `…_current_day_rates`/`…_next_day_rates`; attribute `rates` `[{start, end, value_inc_vat, …}]`, GBP/kWh incl. VAT, merge current + next day | +| Open-Meteo API `tilt`/`azimuth`/`global_tilted_irradiance` | source only | `open-meteo/open-meteo@01ceed5b` `openapi/forecast.yml`, `GlobalTilitedIrradiance.swift` | Convention in code: azimuth 0 = south, -90 east, +90 west (`compass - 180`); OpenAPI text contradicts it; live response not seen; opt-in only | +| Open-Meteo terms | partly | `open-meteo/open-meteo@01ceed5b` `README.md` | Free for open-source/non-commercial, fair use, 10 000 requests/day, CC BY 4.0 attribution link in the UI; separate terms page unreachable | +| Energy-Charts `/price` | **no** (unreachable) | field names only via `mampfes/ha_epex_spot@4.1.0` `EPEXSpot/EnergyCharts/__init__.py` | Not shipped in Phase 4 (plan question 5); terms/licence/limits UNVERIFIED | +| aWATTar `/v1/marketdata` | **no** (unreachable) | field names only via `mampfes/ha_epex_spot@4.1.0` `EPEXSpot/Awattar/__init__.py` | Not shipped in Phase 4; terms UNVERIFIED | diff --git a/docs/research/kostal-plenticore-modbus.md b/docs/research/kostal-plenticore-modbus.md index 91d1e59..8765394 100644 --- a/docs/research/kostal-plenticore-modbus.md +++ b/docs/research/kostal-plenticore-modbus.md @@ -191,3 +191,20 @@ extracted with `pdftotext`). `UNVERIFIED`: timeout behaviour of §3.4 set-points; how external battery management is enabled; whether 533 limits AC output including battery discharge (to be tested in Phase 4). + +### Phase 4 verification (2026-10-16): writing through Home Assistant + +`home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1`, +`components/modbus/{services.py,services.yaml,modbus.py}` (details in +[`ha-websocket-api.md`](ha-websocket-api.md), section "Modbus write action"): +the `modbus.write_register` action writes raw unsigned 16-bit words (single value +-> function 0x06, list -> 0x10), has no float32 support and no read action, and +**does not raise on a failed write** (the hub logs the error). For the register +map above this means: the adapter encodes every `Float` (2 registers) into two +words itself (word order per the byte-order register 5), sends them as one list +`value` to the start address (e.g. 1280/1282 for the §3.5 limits), uses `slave` +(unit id of the inverter, the action defaults to 1, not the hub's configured id), +and **must read the value back** (modbus sensors or battery power) because a failed +write is invisible to the caller. The heartbeat (20 s) shares the hub's lock with +all polled entities. `UNVERIFIED` still: the fallback timer behaviour on the +maintainer's inverter (measurement pending). From dbc31aece47fb1236363e96aefa523508e1ac187 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 21:24:02 +0000 Subject: [PATCH 04/12] docs(plans): merge the forecast, buffer and battery concept into the Phase 4 plan Revision 2, approved by the maintainer on 2026-10-16. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/plans/phase-4.md | 565 +++++++++++++++++++++--------------------- 1 file changed, 286 insertions(+), 279 deletions(-) diff --git a/docs/plans/phase-4.md b/docs/plans/phase-4.md index 9964ce0..9ab9f38 100644 --- a/docs/plans/phase-4.md +++ b/docs/plans/phase-4.md @@ -1,9 +1,8 @@ # Phase 4 – Forecasts, tariffs, planner, battery -Status: **approved** by the maintainer on 2026-10-16 (answers in §6: commit and push after every -WP, Opus takes the Fable roles, battery "as full as possible", grid charging by a price level and -a schedule instead of a fixed night window, calibration charge optional and off; defaults -otherwise) +Status: **revision 2 approved** by the maintainer on 2026-10-16 (answers in §6). Revision 1 was +approved earlier the same day. Revision 2 merges the concept "Prognose, Puffer und Akkuplanung" +from a separate chat (§0) and splits the work into small stacked PRs (§3). Source of truth: [`docs/BRIEF.md`](../BRIEF.md) §3 (two control layers), §4.1 (battery, weather), §4.2, §4.3, §4.4, §4.5 (battery rules of the surplus distribution), §4.8, §4.9 (plan view, only the part Phase 4 needs), §4.10 steps 4–6 and 10, §5, §8, §9, §10; @@ -13,266 +12,287 @@ the part Phase 4 needs), §4.10 steps 4–6 and 10, §5, §8, §9, §10; [`solver-on-aarch64.md`](../research/solver-on-aarch64.md), [`device-integrations.md`](../research/device-integrations.md); the "Phase 4 follow-ups" and the `UNVERIFIED` table in [`docs/STATUS.md`](../STATUS.md). -Branch: `claude/zealous-mayer-j5oda9` (session branch, see §3 and question 1). +Branches: `claude/zealous-mayer-j5oda9` (PR 4-A), then stacked `phase-4/-` (§3). Phase 4 gives SunCore foresight. Adapters read PV forecasts and prices from the integrations the -user already runs, a load forecast is learnt from Home Assistant's statistics, and a planner -(HiGHS LP/MILP with a heuristic fallback) computes a 24–48 h schedule in 15-minute steps. The plan -fills the `Plan` contract the Phase 3 controller already consumes (`DeviceBand`, `BatteryBand`, -`grid_import_limit_w`), so the controller keeps the last word in real time and the safety gate -stays the only writer. The battery becomes a controllable asset through battery adapters, and -every plan interval carries structured reasons that the UI renders as "Why?". **Shadow mode stays -the default; the battery starts in shadow like every device.** +user already runs, a load forecast is learnt from Home Assistant's statistics, a risk model learns +how wrong those forecasts are, and a planner (HiGHS LP/MILP with a heuristic fallback) computes a +48 h schedule in 15-minute steps. The plan fills the `Plan` contract the Phase 3 controller +already consumes (`DeviceBand`, `BatteryBand`, `grid_import_limit_w`), so the controller keeps the +last word in real time and the safety gate stays the only writer. The battery becomes a +controllable asset through battery adapters, and every plan interval carries structured reasons +that the UI renders as "Why?". **Shadow mode stays the default; the battery starts in shadow like +every device.** + +**Goal (from the merged concept):** as much self-sufficiency as possible and as much +self-consumption as is sensible. The battery covers the night and, where possible, bridges a poor +next day. Buffers and limits can be set by every user and are also learnt from history. Phase 3 is implemented but its acceptance (shadow run and one device to automatic on the -maintainer's system) is still open. Phase 4 is built on top of `main` in parallel; nothing in -Phase 4 changes the Phase 3 write paths of devices (question 2). +maintainer's system) is still open. Phase 4 is built in parallel; nothing in Phase 4 changes the +Phase 3 write paths of devices (question 2). + +## 0. Merged concept – decisions + +Where the concept and revision 1 overlapped or disagreed, the better solution was kept: + +| Topic | Revision 1 | Concept | Kept, and why | +|---|---|---|---| +| Uncertainty | P10 PV in target constraints, flat "safety discount" | learnt error distribution per weather class, risk level cautious/balanced/bold → P10/P25/P50 | **Concept**, refined: the quantile is taken from the learnt distribution of the **net** energy error (PV error − load error) per weather class, not P10 PV and P90 load separately – taking both extremes at once double-counts caution and would keep the battery needlessly full. Solcast's own P10/P90 are used until 30 days of history exist. The flat discount is dropped. | +| Risk default | – | open question | **cautious** (maintainer, 2026-10-16): targets met in 90 % of cases. | +| "Battery first" | `prefer_full` strategy for the maintainer, controller rule from Phase 3 | morning buffer "battery first until X %", X rises with uncertainty; consumers before battery while the evening target is still reached; battery wins when the target gets tight | **Both combined:** the plan sends `BatteryBand.priority_until_soc_pct` = X. The controller compares it with the **live** SoC every tick (robust when the day deviates from the plan); above X, surplus goes to consumers while the plan says the evening target is reachable; when it is not, the plan sets the device bands to `allowed=False`/`max_power_w`. `prefer_full` = X is the max SoC (always battery first) – the maintainer's "always full when possible". | +| Next day | not planned | raise the evening target up to 100 % when the 48 h horizon shows a poor next day | **Concept.** Done as a computed lower SoC bound at the evening time (night load + the next day's deficit at the chosen risk quantile), plus a terminal value for energy left at the horizon end so the LP never empties the battery just because the horizon stops. | +| Objective | cost (+ degradation, switching) | cost + optional self-sufficiency weight | **Concept**; default weight **0.01 currency/kWh** of import: it only decides between options whose cost differs by less than 1 ct/kWh (so it never costs real money) and removes the LP's many equal-cost solutions on flat tariffs. 0 = pure cost; expert parameter. | +| Device value | `DeviceBand` preference only | consumers need a utility value, otherwise export is as good as consumption | **Concept.** `DeviceConfig.planner_value_per_kwh`; automatic = just above the export price (so consumption beats export) and never above the import price (SunCore never imports for a device – Phase 3 answer 6). Comfort bands and the water heater's tank are not modelled thermally in Phase 4 (Phase 6). | +| Grid charging | price-level + schedule (answer 2026-10-16) | – | kept unchanged. | +| Load forecast | profiles + exponential smoothing, no ML | ML (gradient boosting/KNN) as an optional layer, only if measurably better | **Concept, with the maintainer's answer:** profiles stay the default and work from day 1; an ML model (KNN on analogue days, implemented with NumPy only – no scikit-learn, whose musl/aarch64 wheels are `UNVERIFIED`) runs **in parallel in shadow** and is scored next to the profiles. It is used only in mode `auto` and only while it beats the profiles; selectable after 30 days of history. | +| Calibration start | not fixed | 14 or 60 days? | **30 days** (maintainer). Before that statistics are only collected; the load forecast can start earlier because HA's long-term statistics already hold history (rebuilt on first start). | +| Re-planning | every 15 min + events | rolling horizon every 15 min | same; kept. | +| Calibration charge | off by default (answer 11) | – | kept. | ## 1. Goals and acceptance criteria -Phase 4 is done when (roadmap Phase 4 milestones): - -1. **Forecast adapters:** Solcast (attributes or the read-only query action, never the Solcast - API), Forecast.Solar (core, via `energy/solar_forecast`), Open-Meteo Solar Forecast (HACS), - a generic attribute adapter (mapping per template), and the built-in fallback behind an - explicit opt-in (question 5). Several sources at once are blended by measured skill; a bias - calibration per hour of day and season is learnt; clipping (inverter limit, export limit) is - recognised; accuracy (MAE, bias per source and horizon) is stored. -2. **Load forecast:** base-load profile from HA long-term statistics (`recorder/statistics_during_period`), - weekday/holiday (optional `workday` binary sensor), optional temperature from a `weather` - entity (`weather.get_forecasts`); exponential smoothing; the draw of controlled devices is - subtracted (no double counting). -3. **Tariffs:** dynamic import from Tibber, Nord Pool, ENTSO-E, EPEX Spot (and Octopus if its +Phase 4 is done when (roadmap Phase 4 milestones plus the concept): + +1. **Forecast adapters:** Solcast (P10/P50/P90 from the attributes or the read-only query action, + never the Solcast API), Forecast.Solar (core, via `energy/solar_forecast`), Open-Meteo Solar + Forecast (HACS), a generic attribute adapter (mapping per template), and the built-in fallback + behind an explicit opt-in (question 5). Several sources are blended by measured skill; a bias + calibration per hour of day × season × weather class (cloudiness) is learnt; clipping + (inverter limit, export limit) is recognised and excluded from the statistics; accuracy (MAE, + bias per source and horizon) is stored and shown. +2. **Load forecast:** base load = house − controlled devices, profiles from HA long-term + statistics by weekday/holiday (optional `workday` sensor) and outside temperature (optional + `weather` entity), exponential smoothing; controlled devices modelled separately; optional + NumPy KNN model in shadow, used only when better (§0). +3. **Risk model and buffer:** learnt distribution of the net forecast error per weather class and + horizon; risk level cautious/balanced/bold (default cautious) → quantile 10/25/50 %; the morning + buffer and the evening target are computed from it so a target is met with the chosen + probability; hard limits (reserve, max SoC, target SoC + time, morning buffer floor) always + manual and always win. +4. **Tariffs:** dynamic import from Tibber, Nord Pool, ENTSO-E, EPEX Spot (and Octopus if its source can be verified), fixed and HT/NT (time windows, seasonal); price formula (spot → end price with grid fee incl. time-variable grid fees §14a module 3, levies, margin, VAT); 15- and 60-minute grids; export fixed, tiered by kWp share (blended rate) and none; negative-price region rule (`de`: no remuneration → avoid export). -4. **Planner:** HiGHS model (`highspy`), heuristic fallback (always available, also when the - solver times out or the model is infeasible, with a UI hint), warm start from the previous - plan, time budget (default 8 s, MIP gap 1 %), runs every 15 min and on events. Benchmark: - the maintainer's reference scenario solves in < 10 s on aarch64 (CI arm runner as proxy, - see R3). -5. **Battery strategies:** self-consumption (default), target SoC by time (P10 PV in the target - constraint), grid charging on a dynamic tariff only when the spread exceeds efficiency loss + - degradation + margin (option "night only", already in `goals`), grid charging on a fixed - tariff only with an HT/NT spread or in an emergency (off by default), discharge lock, reserve - separate from BMS drift, max-SoC limit, periodic calibration charge, cycle counter, - degradation cost in currency/kWh. -6. **Battery adapters:** generic HA entities (template kind `battery`: number/select/switch/ - actions), Kostal Plenticore through the core `kostal_plenticore` integration (min SoC as - discharge lock/reserve), Kostal Modbus §3.5 limits only with the verified register map and - behind question 3. Read-back after every write, write only on change, release = hand the - battery back to the inverter's own control. -7. **Explanations:** every plan interval carries `Reason(key, args)` (price, forecast, target, - limit, rule); the UI renders them in a plan view and in the Now view's "next actions". -8. **`control-safety` review** of every change to `safety/`, the controller's battery path and - the battery adapters' write paths, documented in the PR (invariant table). +5. **Planner:** HiGHS (`highspy`), 48 h, 15 min, rolling every 15 min and on events, warm start, + time budget (default 8 s, MIP gap 1 %), heuristic fallback with a UI hint. Objective cost + + degradation + switching + self-sufficiency weight + device values. Benchmark < 10 s on + aarch64 (R3). +6. **Battery strategies:** self-consumption (default), `prefer_full` (maintainer), target SoC by + time, next-day bridging, grid charging by price level + schedule (+ spread rule on dynamic + tariffs), discharge lock, reserve separate from BMS drift, max-SoC limit, calibration charge + (off by default), cycle counter, degradation cost. +7. **Battery adapters:** generic HA entities (template kind `battery`), Kostal Plenticore through + the core `kostal_plenticore` integration (min SoC as discharge lock/reserve), Kostal Modbus §3.5 + limits through HA's `modbus.write_register` (question 3, shadow/assisted until the fallback + timer is measured). Read-back, write only on change, release = inverter self-control. +8. **Explanations:** every plan interval carries `Reason(key, args)`, e.g. + `planner.battery.morning_buffer` {soc_pct: 40, cause: forecast_uncertain}; forecast accuracy + and bias per source in the UI. +9. **`control-safety` review** of every change to `safety/`, the controller's battery path and the + battery adapters' write paths, documented in the PR (invariant table). Exit (roadmap): plan with battery strategy and "Why?" on the maintainer's system, battery control in shadow mode, then assisted. ## 2. Work packages -Owner = subagent that implements it. Budget rule (CLAUDE.md, your instruction): **Fable only for -`optimizer-engineer` (planner, forecast mathematics, battery economics), `control-safety` -(controller battery path, safety, battery write paths) and the final `reviewer`**; Sonnet for -adapters, backend, simulator, frontend and docs; Haiku for catalogue sync. At most two subagents -in parallel. +Owner = subagent that implements it. Budget rule (answer 2026-10-16): **Opus takes the Fable +roles** – `optimizer-engineer` (planner, forecast mathematics, risk model, battery economics), +`control-safety` (controller battery path, safety, battery write paths), `reviewer` and advisor; +Sonnet for adapters, backend, simulator, frontend and docs; Haiku for catalogue sync. At most two +subagents in parallel. **Every WP ends with a commit and a push.** ### WP1 – Verify interfaces · `ha-integration` (skills `verify-ha-api`, `new-adapter`) -Read from the GitHub sources (the doc sites are unreachable, see `research/README.md`) and record -in `research/integrations.md` / `research/ha-websocket-api.md`, each fact with commit and path: - -- HA: `recorder/statistics_during_period` (request/response, `period`, `types`, units), - `energy/solar_forecast` (response shape, `wh_hours`), `weather.get_forecasts` (response, - `hourly`), `workday` binary sensor, `number.set_value`, `select.select_option`, - `modbus.write_register` (fields, multi-register values) for question 3. -- Integrations: Forecast.Solar `energy.py`; Open-Meteo Solar Forecast (domain, attributes); - Nord Pool `get_prices_for_date`; ENTSO-E and EPEX Spot attributes; Octopus `rates`; - `kostal_plenticore` `Battery:MinSoc`/charging mode entities (translation keys, ranges). -- Built-in fallbacks: Open-Meteo `global_tilted_irradiance`/`tilt`/`azimuth` and its terms; - Energy-Charts `/price` and aWATTar `marketdata` and their terms. If unreachable they stay - `UNVERIFIED` and are not shipped (question 5). - -### WP2 – Contracts · `optimizer-engineer` (Fable), reviewed by `architect` - -- `core/series.py`: `TimeSeries` (aware start/step, values, unit), `QuantileSeries` - (P10/P50/P90, monotone), `PriceSeries` (import/export per interval, resolution 15/60 min, - source, `price_includes`), resampling to the 15-minute planning grid (energy-preserving for - power/energy, step-hold for prices). -- `core/forecast.py`: `ForecastSource` protocol (`fetch(window) -> QuantileSeries`), accuracy - record; `core/tariff.py`: `PriceSource` protocol, `PriceFormula`, HT/NT windows; - `core/battery.py`: `BatteryModel` (capacity, efficiencies, limits, reserve, BMS-drift margin, - degradation cost), `BatteryAction` for the gate (charge/discharge limits, min SoC, grid charge - power) – **`BatteryAction` is reviewed by `control-safety`**. -- `Plan` v2 (additive to `core/control.py`): per interval the expected flows (PV P10/P50/P90, - load, import, export, battery power, SoC), prices, the battery band, reasons; `Plan.source` - (`solver`/`heuristic`/`default`) and solver statistics. The controller keeps reading - `DeviceBand`/`BatteryBand`/`grid_import_limit_w` unchanged. -- Config documents (additive, migrated): `ForecastConfig` v2 (several sources, weights auto/ - manual, safety discount), `TariffConfig` v3 (formula components, HT/NT windows, seasonal), - section `battery` (adapter template id, entities, model parameters, strategy options), - `goals` v2 (night window for grid charging, calibration interval, conservative planning on/off). -- Template schema v1 additions (additive, schema v1 changelog): kind `battery` (ADR-0012 already - names it), forecast `series` mapping for the generic attribute adapter, tariff HT/NT. +From the GitHub sources (doc sites unreachable), recorded with commit and path in +`research/integrations.md` / `research/ha-websocket-api.md`: `recorder/statistics_during_period`, +`energy/solar_forecast`, `weather.get_forecasts`, `workday`, `number.set_value`, +`select.select_option`, `modbus.write_register`, `nordpool.get_prices_for_date`, +`tibber.get_prices`, `kostal_plenticore` battery entities; Solcast, Open-Meteo Solar Forecast, +ENTSO-E, EPEX Spot, Octopus attributes; Open-Meteo API and Energy-Charts/aWATTar and their terms. +What cannot be read stays `UNVERIFIED`. + +### WP2 – Contracts · `optimizer-engineer` (Opus), reviewed by `architect` + +- `core/series.py`: `TimeSeries`, `QuantileSeries`, `PriceSeries`; resampling onto the 15-minute + grid (energy-preserving for power/energy, step-hold for prices, mixed 15/60 min, DST-safe). +- `core/forecast.py`: `ForecastSource`/`PriceSource` protocols, `AccuracyRecord`; + `core/tariff.py`: `PriceFormula`, weekly windows (HT/NT and the grid-charge schedule), blended + export rate; `core/battery.py`: `BatteryModel`, `BatteryStrategy`, `RiskLevel`, + `GridChargePolicy`; `BatteryAction` (**reviewed by `control-safety`**). +- `Plan` v2 (additive): per interval expected flows, prices, reasons; `Plan.source`, `Plan.stats`; + `BatteryBand.priority_until_soc_pct`; `DeviceConfig.planner_value_per_kwh`. +- Config (additive, migrated): `ForecastConfig` v2 (several sources, weights auto/manual, + learning: `min_history_days` 30, `load_ml_mode` off|shadow|auto = shadow), `TariffConfig` v3 + (formula, HT/NT), section `battery` (adapter, model, strategy, risk = cautious, + morning-buffer floor, `bridge_next_day`, self-sufficiency weight, grid-charge policy, + calibration), `goals` v2 (`grid_charge_night_only` → schedule 22:00–06:00, charging still off). +- Template schema v1 additions (schema v1 changelog): kind `battery`, generic forecast `series` + mapping, tariff HT/NT. ### WP3 – Storage and statistics · `backend-engineer` -- `/data/timeseries.db` (ADR-0004; `backup_exclude` in both `config.yaml`, via `ha-integration`): - own migrations; tables for measurements (controller resolution 48 h, 1-min 30 days, 15-min - unlimited), forecasts versioned by issue time, prices; nightly downsample + vacuum; rebuild of - 15-min history from `recorder/statistics_during_period` when the file is missing. -- `suncore.db` migration `0004`: `plan_snapshot` (compressed JSON, last N plans + retention), - learnt parameters (forecast bias tables, load profiles, blending weights). Schema 4: 0.4.x - refuses newer databases → export before update (release note). -- Rate limiter per (domain, service) with try-acquire/deadline for `call_action_response` - (follow-up A3 from Phase 1) before the first adapter calls an action; boundary rule 7 admits - the adapter wiring. +`/data/timeseries.db` (ADR-0004, `backup_exclude`): measurements (48 h raw, 1-min 30 days, +15-min unlimited), forecasts versioned by issue time, prices, nightly downsample + vacuum, +rebuild of 15-min history from `recorder/statistics_during_period`. `suncore.db` migration +`0004`: `plan_snapshot`, learnt parameters (bias tables, error distributions, load profiles, +blending weights, ML state). Rate limiter per (domain, service) with try-acquire/deadline for +`call_action_response` (follow-up A3). ### WP4 – Tariff adapters · `backend-engineer` (skill `new-adapter`), maths by `optimizer-engineer` -`suncore/tariff/`: adapters Tibber (`tibber.get_prices`), Nord Pool, ENTSO-E, EPEX Spot, -Octopus (only if WP1 verifies it), generic price-list attribute, fixed and HT/NT; price formula; -export fixed/tiered (blended rate by kWp share)/none; negative-price rule from the region -template; refresh on the integration's update, never more than the rate limiter allows. Fixtures -copied from the verified source shapes. New templates per adapter with `source` and a conformance -scenario (`new-device-template` skill). +`suncore/tariff/`: Tibber, Nord Pool, ENTSO-E, EPEX Spot, Octopus (only if verified), generic +price list, fixed and HT/NT; price formula; export fixed/tiered/none; negative-price rule from the +region; fixtures from verified shapes; templates with `source` and a conformance scenario. + +### WP5 – PV forecast · `device-adapters` (adapters), `optimizer-engineer` (Opus, blending/calibration) + +`suncore/forecast/pv/`: Solcast, Forecast.Solar, Open-Meteo Solar Forecast, generic attribute, +built-in fallback (question 5). Blending by inverse MAE over 14 days with a minimum weight and a +manual override; bias calibration per hour × season × weather class (clear-sky index buckets +from the forecast itself), bounded and shrinking to 1 with little data, active after +`min_history_days`; clipping detection; accuracy in `timeseries.db`. -### WP5 – PV forecast · `device-adapters` (adapters), `optimizer-engineer` (Fable, blending/calibration) +### WP6 – Load forecast · `optimizer-engineer` (Opus) -`suncore/forecast/pv/`: Solcast (`detailedForecast` attributes, `query_forecast_data` as -fallback), Forecast.Solar, Open-Meteo Solar Forecast, generic attribute, built-in fallback (only -after question 5); blending weighted by recent skill (inverse MAE over 14 days, minimum weight, -manual override); bias calibration per hour of day × season × clear-sky index bucket (bounded, -shrinks to 1 with little data); clipping detection (inverter/export limit plateaus excluded from -the error statistics); accuracy bookkeeping in `timeseries.db`. +`suncore/forecast/load.py`: base load = house − controlled devices; profile per day type +(workday/weekend/holiday) × 15-min slot with exponential smoothing; temperature term with a +mapped weather entity; residual quantiles; cold start from HA statistics, else a flat profile +with a wide band and a UI hint. Controlled devices are forecast separately (their Phase 3 power +models and the plan), never in the base load. -### WP6 – Load forecast · `optimizer-engineer` (Fable) +### WP7 – Risk model and buffer · `optimizer-engineer` (Opus) *(new, from the concept)* -`suncore/forecast/load.py`: house base load = house − controlled devices (their estimates from -the Phase 3 power models); profile per weekday type (workday/weekend/holiday) × 15-min slot with -exponential smoothing; temperature regression term when a weather entity is mapped; P10/P90 from -residuals; cold start from HA statistics (WP3), else a flat profile from the current mean with a -wide band and a UI hint. No ML model in Phase 4 (aarch64 footprint). +`suncore/forecast/risk.py`: empirical distribution of the net energy error (actual − forecast, +PV minus load) per weather class and horizon window (rest of today, night, next day), from the +stored forecasts and actuals; until `min_history_days` the source quantiles (Solcast P10/P90) and +the load residuals are used, labelled as such. Functions the planner calls: +`net_energy_quantile(window, risk)`, `morning_buffer_pct(...)` (higher when the PV forecast is +uncertain), `evening_target_pct(...)` (night load + next-day deficit at the risk quantile, capped +at max SoC, never below the user's target). Hypothesis tests: monotone in risk level and in +uncertainty, never below the manual limits. -### WP7 – Planner · `optimizer-engineer` (Fable) +### WP8 – Planner · `optimizer-engineer` (Opus) -`suncore/planner/`: `model.py` (sparse matrices for `highspy`, Protocol for the backend so tests -can use a reference solver), `heuristic.py`, `explain.py`, `run.py`. +`suncore/planner/`: `model.py` (sparse matrices for `highspy`, backend protocol), `heuristic.py`, +`explain.py`, `run.py`. - Variables per interval: import, export, battery charge/discharge (separate efficiencies), SoC, - PV curtailment (only where an export limit or negative prices make it useful, informational - until the export-limit chain in Phase 6), controllable device energy (band per device). -- Constraints: energy balance, SoC bounds incl. reserve + BMS-drift margin, battery power limits, - grid import limit, static export limit (13.5 kW case), target SoC by time on **P10** PV, - grid charging only in allowed windows (night window, fixed tariff off by default), discharge - lock where it pays; binaries only where needed (no simultaneous charge/discharge only if the - LP relaxation exploits it with negative prices; device min run later). -- Objective: import cost − export revenue (with negative prices and the region rule) + degradation - + switching penalty + soft target penalties; P50 in the objective. -- Warm start, time budget, MIP gap; on timeout use the incumbent if feasible, else the heuristic; - infeasible → relax soft targets, then heuristic, reason `planner.fallback.*`. -- `explain.py`: per interval the binding constraints and price comparisons become `Reason`s, e.g. - `planner.battery.grid_charge` {from, to, price, later_price, later_time, pv_p10_kwh}. -- Outputs `Plan` v2 with `BatteryBand` (min/max power, SoC corridor, `charge_from_surplus`) and - `DeviceBand` preferences; the controller's surplus rule "battery first / consumers first if the - forecast still reaches the target" now follows the plan. -- Property tests (hypothesis): balance holds, SoC within bounds, never worse than the heuristic, - no grid charging outside allowed windows, no export at negative prices when avoidable. -- Benchmark script `scripts/bench_planner.py` with the maintainer's reference scenario - (17 kWp east-west, 29.6 kWh, Tibber 15-min prices, export limit 13.5 kW, tiered EEG). - -### WP8 – Runtime and scheduler · `backend-engineer` - -Planner run every 15 min and on events (new prices, forecast update, deviation above threshold, -config change, override), in a worker thread with the time budget, at most one run at a time; -the controller gets the new plan atomically; plan snapshots stored; `/health` unaffected by -planner failures (status in the API). API: `GET /api/v1/plan` (current plan with reasons and -statistics), `POST /api/v1/plan/replan`, `GET /api/v1/forecast/accuracy`, `GET /api/v1/prices`; -live feed `plan` message. OpenAPI and frontend types regenerated. Demo: simulator prices and -forecasts so the demo shows a real plan. - -### WP9 – Battery control · `control-safety` (Fable), adapters by `device-adapters` - -- Controller: translates the plan's `BatteryBand` into `BatteryAction` intents (on change only, - hysteresis on power levels); the existing surplus share follows `charge_from_surplus`. -- Safety: new invariants (property-tested): never discharge below reserve + drift margin by - SunCore; grid charging only from a plan interval that allows it, within the import limit and - the night window; charge/discharge limits within battery and inverter limits; write rate and - write-on-change; battery release = restore the adapter's baseline (inverter self-control); - watchdog: stale SoC/battery power while the grid meter moves pauses battery control; the - hung-inverter case (BRIEF §9) releases and stops writing. "No SunCore-caused import" keeps - holding for devices; the only allowed import is planned grid charging of the battery. -- Adapters `suncore/devices/battery/`: `generic` (template kind `battery`, roles: min-SoC number, - max charge/discharge numbers, mode select, force-charge switch/action), `kostal_plenticore` - (core integration: min SoC → discharge lock/reserve, charging mode select; no charge set-point - over REST), `kostal_modbus` §3.5 limits with fallback timer (question 3; shadow/assisted only - until the timeout is measured on the maintainer's inverter). -- Simulator: battery with BMS drift, inverter self-consumption logic as the fallback behaviour, - Kostal-like fallback timer, scenarios `negative_prices`, `price_spike`, `winter_fog_target`, - `grid_charge_night`, `inverter_hang_battery`, `reference_house` (maintainer's system). - -### WP10 – UI · `frontend-engineer` - -- Plan view `#/plan` (minimal for Phase 4, the full chart is Phase 5): ECharts loaded on demand - (own chunk), 24/48 h, price band, PV P10–P90 band + P50, load forecast, planned SoC and battery - power, import/export; tap/hover on an interval shows its reasons; solver/heuristic badge. -- Now view: "next planned actions" with "Why?"; battery card (mode switch, proposals for - assisted, release/emergency stop like the devices). -- Wizard: step 4 several forecast sources (auto-detected), step 5 battery adapter or "observe - only" with model parameters, step 6 price formula and HT/NT, step 8 battery strategy options, - step 10 first plan as preview. Settings pages follow. -- Vitest, Playwright E2E against the demo, axe; screenshots `plan-*` (light/dark, phone/tablet, - en/de) via `demo-screenshots`. Bundle budget 250 kB gzip for first paint holds (ECharts lazy). - -### WP11 – Docs, translations, STATUS · `docs-writer`, `i18n-sync` (Haiku) - -User guide `docs/user//planning-and-battery.md` (en/de: forecasts and blending, tariffs and -price formula, how the planner decides, battery strategies, battery adapters and how to enable -external battery control, shadow → assisted walk-through, known limitations), quick start and -app `DOCS.md` updated, architecture §planner, compatibility list regenerated, all new keys en/de -(`sync-translations`), STATUS, roadmap, `UNVERIFIED` table. + device energy within its band, PV curtailment where an export limit or negative prices make it + useful (informational until the export-limit chain in Phase 6). +- Constraints: energy balance, SoC bounds with reserve + BMS-drift margin, battery limits, import + limit, static export limit, morning buffer and evening target from WP7 (soft with a high + penalty, so the plan stays feasible and explains the shortfall), grid charging only inside the + price level and schedule (and the spread rule on dynamic tariffs), discharge lock where it + pays; binaries only where needed. +- Objective: import cost − export revenue (negative prices and the region rule) + degradation + + switching + self-sufficiency weight − device values − terminal SoC value; PV at the risk + quantile in the target constraints, P50 in the objective. +- Warm start, time budget, MIP gap, fallback chain incumbent → relaxed targets → heuristic. +- `explain.py`: binding constraints and price comparisons → `Reason`s. +- Outputs `Plan` v2: `BatteryBand` (power limits, SoC corridor, `priority_until_soc_pct`, + `charge_from_surplus`) and `DeviceBand` (allowed, `max_power_w`, preference: throttle or switch + off consumers when the target gets tight). +- Property tests: balance, SoC bounds, never worse than the heuristic, no grid charging outside + the policy, no avoidable export at negative prices, targets met when the actual lies inside the + planned quantile. +- `scripts/bench_planner.py`: the maintainer's reference scenario (17 kWp east-west, 29.6 kWh, + Tibber 15-min prices, export limit 13.5 kW, tiered EEG). + +### WP9 – Runtime and scheduler · `backend-engineer` + +Planner every 15 min (rolling horizon with live SoC) and on events (new prices, forecast update, +deviation above threshold, config change, override), in a worker thread with the budget, one run +at a time; atomic plan hand-over to the controller; plan snapshots; `/health` unaffected. API: +`GET /api/v1/plan`, `POST /api/v1/plan/replan`, `GET /api/v1/forecast/accuracy` (incl. ML vs +profile), `GET /api/v1/prices`; live feed `plan`. Demo with simulator prices and forecasts. + +### WP10 – Battery control · `control-safety` (Opus), adapters by `device-adapters` + +- Controller: `BatteryBand` → `BatteryAction` intents (on change, hysteresis); evaluates + `priority_until_soc_pct` against the live SoC; device bands from the plan. +- Safety invariants (property-tested): SunCore never discharges below reserve + drift margin; + grid charging only from a plan interval that allows it, within the import limit, price level + and schedule; limits within battery/inverter limits; write rate, write-on-change; release = + restore the adapter's baseline; stale SoC/battery power while the grid meter moves pauses + battery control; hung inverter → release and stop writing. "No SunCore-caused import" keeps + holding for devices. +- Adapters `suncore/devices/battery/`: `generic`, `kostal_plenticore`, `kostal_modbus` (§3.5 via + `modbus.write_register`, question 3). +- Simulator: battery with BMS drift, inverter self-consumption as fallback, Kostal-like fallback + timer. Scenarios: `sunny_day`, `mixed_clouds`, `two_poor_days`, `forecast_too_optimistic`, + `negative_prices`, `price_spike`, `grid_charge_price_level`, `inverter_hang_battery`, + `reference_house`. + +### WP11 – UI · `frontend-engineer` + +Plan view `#/plan` (minimal, the full chart is Phase 5): ECharts on demand, 24/48 h, price, PV +P10–P90 + P50, load, planned SoC (with morning buffer and evening target lines), battery power, +import/export; reasons per interval; solver/heuristic badge. Forecast accuracy and bias per source +(incl. ML vs profile). Now view: next planned actions with "Why?", battery card. Wizard: steps 4 +(several sources), 5 (adapter, limits, strategy, risk level), 6 (formula, HT/NT), 8 (grid-charge +price level and schedule, self-sufficiency weight), 10 (first plan preview). Vitest, Playwright, +axe, screenshots `plan-*` from the demo. + +### WP12 – Optional ML load model · `optimizer-engineer` (Opus) + +NumPy KNN on analogue days (features: day type, slot, temperature, recent load); mode `shadow` +by default: scored next to the profiles, shown in the accuracy view, used only in `auto` while +better over the last 14 days. Benchmark on aarch64 (training + prediction < 1 s). + +### WP13 – Docs, translations, STATUS · `docs-writer`, `i18n-sync` (Haiku) + +User guide `docs/user//planning-and-battery.md` (forecasts, risk level and buffers, +tariffs, how the planner decides, strategies, adapters and enabling external battery control, +shadow → assisted walk-through, limitations), quick start, app `DOCS.md`, architecture §planner, +compatibility list, all keys en/de, STATUS, roadmap, `UNVERIFIED` table. ## 3. Pull requests and order -Default (question 1): all work on the session branch `claude/zealous-mayer-j5oda9`, one commit -series per WP and **one PR**; if the maintainer allows extra branches, the same commits are split -into stacked PRs: +Small stacked PRs (maintainer, 2026-10-16). 4-A uses the session branch; the others are pushed as +`phase-4/-`, each based on the previous one. Every PR gets the `reviewer` agent +(Opus) before it is opened, its findings fixed in `fix(...)` commits; PRs with safety-relevant +code additionally the `control-safety` sign-off. | PR | Content | Depends on | |---|---|---| -| 4-A `feat(core,storage,forecast,tariff)` | WP1–WP6 | – | -| 4-B `feat(planner,app)` | WP7, WP8 | 4-A | -| 4-C `feat(safety,controller,devices)` battery | WP9 | 4-B | -| 4-D `feat(frontend),docs` | WP10, WP11 | 4-C | +| 4-A `feat(core)` contracts | plan, WP1, WP2 | – | +| 4-B `feat(storage)` time series | WP3 | 4-A | +| 4-C `feat(tariff)` | WP4 | 4-B | +| 4-D `feat(forecast)` PV | WP5 | 4-B | +| 4-E `feat(forecast)` load + risk | WP6, WP7 | 4-D | +| 4-F `feat(planner,app)` | WP8, WP9 | 4-C, 4-E | +| 4-G `feat(safety,controller,devices)` battery | WP10 | 4-F | +| 4-H `feat(frontend),docs` | WP11, WP13 | 4-G | +| 4-I `feat(forecast)` ML load (optional) | WP12 | 4-E | ```text -WP1 ─► WP2 ─┬► WP3 ─► WP4 ─┐ - ├► WP5 ────────┼─► WP7 ─► WP8 ─► WP9 ─► WP10 ─► WP11 - └► WP6 ────────┘ +WP1 ─► WP2 ─► WP3 ─┬► WP4 ───────────────┐ + └► WP5 ─► WP6 ─► WP7 ─┴► WP8 ─► WP9 ─► WP10 ─► WP11 ─► WP13 + └► WP12 ``` -I integrate, run `./scripts/check.sh` and the frontend checks after every WP and commit per WP -(Conventional Commits; CHANGELOG via release-please, no hand edits). Before the PR the `reviewer` -agent (Fable) reviews the whole change; its findings are fixed in separate `fix(...)` commits. -`control-safety` signs off WP9 with the invariant table in the PR description. +I integrate, run `./scripts/check.sh` and the frontend checks after every WP, commit and push per +WP (Conventional Commits; CHANGELOG via release-please, no hand edits). Open questions go into +each PR under "Offene Fragen an den Maintainer". ## 4. Tests and verification | Layer | Tool | What | |---|---|---| -| Series/contracts | pytest, hypothesis | resampling preserves energy, quantiles monotone, price grid 15/60 mixed | -| Adapters | pytest with fixtures from verified source shapes | Solcast, Forecast.Solar, Open-Meteo SF, generic, Tibber, Nord Pool, ENTSO-E, EPEX, fixed/HT-NT; missing/partial data; DST days (23/25 h) | +| Series/contracts | pytest, hypothesis | resampling preserves energy, quantiles monotone, mixed 15/60 grids, DST days | +| Adapters | pytest with fixtures from verified shapes | every forecast and price adapter; missing/partial data | | Blending/calibration/load | pytest, hypothesis | weights sum to 1, bias bounded, clipping excluded, controlled loads subtracted | -| Planner | hypothesis + reference solver | balance, SoC bounds, objective ≤ heuristic, grid charging windows, negative prices, infeasible → fallback, time budget respected | -| Benchmark | `scripts/bench_planner.py`, CI job on `ubuntu-24.04-arm` | reference scenario 48 h; budget see R3 | -| Safety | hypothesis | battery invariants, gate never writes the battery in shadow/assisted without confirm, release paths | -| Simulator | `-m simulator` | scenarios of WP9, 7-day run with planner + controller, fail-safe of the battery | -| Storage | pytest | migrations 0004 + timeseries.db, rebuild from statistics, retention | -| Boundary | AST test | planner imports `core` only; battery adapters' `apply()` only via the gate; actions only through the rate limiter | -| API | httpx ASGI | plan, replan, accuracy, prices | -| Frontend | Vitest, Playwright, axe | plan view, reasons, battery card, wizard steps | +| Risk model | hypothesis | buffer/target monotone in risk and uncertainty, never below manual limits | +| Planner | hypothesis + reference solver | balance, SoC bounds, objective ≤ heuristic, grid-charge policy, negative prices, fallback, budget; targets met when actuals lie inside the planned quantile | +| Benchmark | `scripts/bench_planner.py`, CI job on `ubuntu-24.04-arm` | reference scenario 48 h (R3); ML training/prediction | +| Safety | hypothesis | the battery never falls below the reserve through SunCore; no write in shadow/assisted without confirm; release paths | +| Simulator | `-m simulator` | scenarios of WP10 incl. sunny day, mixed clouds, two poor days, too optimistic forecast; 7-day run planner + controller | +| Storage | pytest | migration 0004 + timeseries.db, rebuild, retention | +| Boundary | AST test | planner imports `core` only; battery `apply()` only via the gate; actions only through the rate limiter | +| API, Frontend | httpx ASGI; Vitest, Playwright, axe | plan, replan, accuracy, prices; plan view, battery card, wizard | | Coverage | pytest-cov | ≥ 80 % core/safety/planner/controller (aim ≥ 90 % planner and safety) | No hardware and no Docker daemon in the session: image and smoke test run in CI; the maintainer's @@ -282,78 +302,65 @@ shadow and assisted run is the acceptance test. | # | Risk | Impact | Mitigation | |---|---|---|---| -| R1 | A battery write empties the battery or charges from the grid at the wrong time | money, backup reserve | battery starts in shadow, assisted needs confirmation, invariants property-tested, reserve + BMS drift margin, `control-safety` sign-off, inverter fallback timer (§3.5) as hardware watchdog | -| R2 | Kostal Modbus timeout and enabling unverified | adapter does nothing or does not fall back | §3.5 limits with documented fallback first; §3.4 set-points out of scope; measurement by the maintainer before automatic | -| R3 | Solve time on a Pi 4 > 10 s | stale plans | LP first, few binaries, warm start, time budget 8 s + heuristic; CI arm runner is faster than a Pi 4 → CI budget 3 s as a proxy; the maintainer runs amd64 (KVM), a Pi 4 measurement needs a volunteer | -| R4 | `highspy` (+ NumPy) enlarges the image and memory | RSS budget 300 MB | musllinux wheels verified (research); smoke-test memory gate stays; solver in a thread, model freed after the run | -| R5 | Forecast adapters rely on attributes that change between integration versions | missing plan | tolerant parsers, source version in `tested_with`, fallback to other sources or flat profile with UI hint | -| R6 | Built-in fallbacks need internet and unverified terms | licence/terms breach, cloud dependency | opt-in, off by default, attribution shown, only after WP1 verified the terms (question 5) | -| R7 | Load forecast poor at start (little history) | bad battery decisions | rebuild from HA statistics, wide P10/P90, conservative target on P10, UI hint | -| R8 | Phase too large for one review | slow review, missed bugs | WP commits, stacked PRs if allowed (question 1), reviewer per PR | -| R9 | DB schema 4 blocks a downgrade | user cannot roll back | release note, export before update | -| R10 | Phase 3 acceptance finds controller issues while Phase 4 builds on it | rework | Phase 4 changes the controller only in the battery path and the plan input; fixes from the live test are merged first | -| R11 | ECharts breaks the bundle budget | slow first paint | lazy chunk for the plan view only | - -## 6. Questions to the maintainer - -Bold = default if there is no answer. - -1. **PR structure:** the session may only push `claude/zealous-mayer-j5oda9`. **One PR with one - commit series per WP** – or may I push stacked branches `phase-4/a…d` (four PRs as in - Phases 1–3)? -2. **Phase 3 not yet accepted:** **build Phase 4 in parallel; merge only after your Phase 3 - acceptance**, fixes from your live test go first. -3. **Kostal battery control path** (you have SunSpec Modbus read-only and the core - `kostal_plenticore` integration): - (a) **Phase 4 ships the generic adapter and `kostal_plenticore` (min SoC = discharge - lock/reserve, no grid charging over REST); the Modbus §3.5 limits are written through Home - Assistant's `modbus.write_register` action (you add a `modbus` hub in HA), shadow/assisted - only until the fallback timer is measured on your inverter**; - (b) SunCore talks Modbus TCP itself (`pymodbus`, opt-in) – the brief's non-goal "no own - Modbus implementation" speaks against it; - (c) no Modbus in Phase 4. Is "external battery management via Modbus" enabled in your - inverter (installer menu)? -4. **Model budget:** in Phase 3 the Fable credits ran out and Opus did the Fable work. **Same - rule again if Fable is unavailable**? -5. **Built-in fallbacks** (Open-Meteo PV, Energy-Charts/aWATTar prices) need internet from the - app and their APIs/terms could not be verified from the session: **only ship them if WP1 can - verify them, then opt-in and off by default; otherwise move them to Phase 6**. -6. **Battery economics defaults** (all expert parameters, shown with their source/explanation): - **charge and discharge efficiency 95 % each, degradation 0.04 €/kWh throughput, grid-charge - safety margin 0.03 €/kWh, BMS-drift margin 5 % above the reserve** (your BMS sags from 10 % - to 6–7 %). Do you know BYD HVM figures you want instead? -7. **Night window for "grid charging only at night":** **22:00–06:00 local**, configurable. -8. **Your EEG tiers** (10 kWp + rest): the region `de` template has no official rates (not - invented). **You enter the two rates in the tariff step**; or do you give me the BNetzA table - to cite? -9. **Tibber `price`:** **treated as the all-in end price** (no formula on top). Correct for your - contract? -10. **Plan view scope:** **minimal plan chart in Phase 4 (price, PV band, load, SoC, battery, - reasons), the full chart with Gantt/zoom in Phase 5**. -11. **Calibration charge:** **off by default; if on, once every 30 days to 100 %, scheduled into - a sunny day by the planner**. +| R1 | A battery write empties the battery or charges from the grid at the wrong time | money, backup reserve | battery starts in shadow, assisted needs confirmation, invariants property-tested, reserve + drift margin, `control-safety` sign-off, §3.5 fallback timer as hardware watchdog | +| R2 | Kostal Modbus timeout and enabling unverified | adapter does nothing or does not fall back | §3.5 limits first; §3.4 out of scope; measurement before automatic | +| R3 | Solve time on a Pi 4 > 10 s | stale plans | LP first, few binaries, warm start, budget 8 s + heuristic; CI arm runner as proxy (budget 3 s); the maintainer runs amd64, a Pi 4 needs a volunteer | +| R4 | `highspy` + NumPy enlarge image and memory | RSS budget 300 MB | musllinux wheels verified; smoke-test memory gate; solver in a thread | +| R5 | Integration attributes change between versions | missing plan | tolerant parsers, `tested_with`, fallback to other sources or a flat profile | +| R6 | Built-in fallbacks need internet and unverified terms | terms breach, cloud dependency | opt-in, off, attribution, only after WP1 verified the terms (question 5) | +| R7 | Little history at the start | poor buffers | HA statistics rebuild for load; source quantiles for PV until 30 days; cautious default | +| R8 | Learnt buffer too high | battery always full, less self-consumption | bounded by max SoC, shown with its reason, risk level and manual floor adjustable | +| R9 | DB schema 4 blocks a downgrade | no rollback | release note, export before update | +| R10 | Phase 3 live test finds controller issues | rework | Phase 4 touches the controller only in the battery path and the plan input; fixes first | +| R11 | ECharts breaks the bundle budget | slow first paint | lazy chunk | +| R12 | Nine stacked PRs drift when an early one changes | rebase work | merge in order, small PRs, merge commits (no history rewrite) | + +## 6. Questions and answers + +1. PR structure: ~~one PR~~ **small stacked PRs** (maintainer, 2026-10-16, §3). May I push the + extra branches `phase-4/b…i`? (Needed for the stacked PRs.) +2. Phase 3 not yet accepted: **build in parallel; merge after the Phase 3 acceptance**. +3. Kostal battery control path: **(a) generic + `kostal_plenticore`; Modbus §3.5 via HA's + `modbus.write_register`, shadow/assisted until the fallback timer is measured** – still open: + is "external battery management via Modbus" enabled in your inverter? +4. Model budget: **Opus takes the Fable roles** (answered). +5. Built-in fallbacks: **only if verifiable, then opt-in and off; else Phase 6**. +6. Battery economics defaults: **95 %/95 %, 0.04 €/kWh degradation, 0.03 €/kWh margin, drift + margin 5 %**; BYD HVM figures welcome. +7. Grid charging: **price level + schedule** (answered); off until set. +8. EEG tiers: **you enter the two rates**. +9. Tibber `price`: **all-in end price**. +10. Plan view: **minimal in Phase 4**. +11. Calibration charge: **built, off by default** (answered). +12. Concept: history needed for calibration and ML: **30 days** (answered); ML **runs in + parallel in shadow for comparison** (answered). +13. Concept: risk level default: **cautious** (answered). +14. Concept: self-sufficiency weight: **on with 0.01 currency/kWh** (my choice, §0; maintainer + left it open). ### Answers (2026-10-16) - Every WP ends with a commit **and a push**, so nothing is lost when the session's usage quota runs out. - **Opus takes the Fable roles** (`optimizer-engineer`, `control-safety`, `reviewer`, advisor). -- Question 1 (PR structure), 2, 3, 5, 6, 8, 9, 10: defaults. -- Question 11: the calibration charge is built for other users, **off by default**. -- Battery strategy of the maintainer: **"keep the battery as full as possible"** – a strategy - option `prefer_full` (target 100 % as early as the P10 forecast allows, PV surplus to the battery - before consumers until full, discharge only for the house); self-consumption stays the default - for new installations. -- Question 7: instead of a fixed night window, grid charging is allowed by **a price level** - (charge only when the import price is at or below a user-set level, currency/kWh) **and a - schedule** (weekly time windows, default: always allowed inside the price level). The spread rule - (efficiency + degradation + margin) still applies on dynamic tariffs; the existing - `goals.grid_charge_night_only` is migrated to a schedule 22:00–06:00. Grid charging stays off - until the user sets a level or a schedule. +- Calibration charge: built for other users, **off by default**. +- Battery of the maintainer: **"as full as possible"** → strategy `prefer_full` (§0). +- Grid charging by **a price level and a schedule** instead of a fixed night window; the spread + rule still applies on dynamic tariffs; `goals.grid_charge_night_only` is migrated to a schedule + 22:00–06:00; grid charging stays off until the user sets a level or a schedule. +- Concept: 30 days of history, ML in parallel for comparison, risk level cautious. +- Stacked branches `phase-4/b…i` may be pushed. +- External battery management via Modbus is **not enabled** on the maintainer's inverter yet; he + enables it when he wants grid charging. Consequence: the Modbus adapter is built and tested in + the simulator but stays `UNVERIFIED` on hardware; without it SunCore controls the battery only + through `kostal_plenticore` (min SoC: reserve and discharge lock). Charging from PV needs no + write at all (the inverter's own self-consumption logic does it). +- Self-sufficiency weight: explained to the maintainer, default 0.01 currency/kWh kept. ## 7. Out of scope for Phase 4 Full plan chart with device Gantt, zoom/pan, history, savings vs baselines, what-if and -backtesting (Phase 5); export limiting through the inverter, peak shaving with load shedding, -§14a control signal, backup-power awareness, wallbox/EV, heat pump, shiftable programs, MQTT -entities and notifications (Phase 6); Kostal §3.4 set-points and MDC; V2H/V2G; ML load models. +backtesting (Phase 5); thermal models for climate and water heater in the planner, export limiting +through the inverter, peak shaving with load shedding, §14a control signal, backup-power +awareness, wallbox/EV, heat pump, shiftable programs, MQTT entities and notifications (Phase 6); +Kostal §3.4 set-points and MDC; V2H/V2G; ML models beyond the NumPy KNN. From d11006874bfb261cd37fe7250647dff264bae480 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 21:26:56 +0000 Subject: [PATCH 05/12] feat(core): add the Phase 4 contracts for series, forecasts, tariffs, battery and plan v2 Time series with energy-preserving resampling onto the 15-minute grid, forecast and price source protocols, price formula and time-of-use windows, battery model with strategies, risk level, grid-charge policy (price level + schedule) and calibration charge, BatteryAction (rejected by the gate until the battery path exists), Plan v2 with expected flows, prices and reasons, config sections forecast v2, tariff v3, goals v2 and the new battery section with migrations, template kind battery. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- frontend/e2e/settings.spec.ts | 4 +- frontend/e2e/wizard-review.spec.ts | 10 +- frontend/e2e/wizard.spec.ts | 2 +- frontend/src/lib/api/openapi.json | 1061 ++++++++++++++++++++- frontend/src/lib/api/schema.d.ts | 615 +++++++++++- frontend/src/lib/api/types.ts | 13 +- frontend/src/lib/control/actions.ts | 2 + frontend/src/lib/control/present.test.ts | 1 + frontend/src/lib/i18n/locales/de.json | 21 +- frontend/src/lib/i18n/locales/en.json | 21 +- frontend/src/lib/views/Settings.svelte | 3 +- frontend/src/lib/wizard/forecastForm.ts | 54 +- frontend/src/lib/wizard/forms.test.ts | 84 +- frontend/src/lib/wizard/goalsForm.ts | 16 +- frontend/src/lib/wizard/tariffForm.ts | 31 +- scripts/gen_compat.py | 3 +- suncore/api/validation.py | 17 + suncore/core/battery.py | 284 ++++++ suncore/core/capabilities.py | 7 + suncore/core/config.py | 180 +++- suncore/core/control.py | 113 ++- suncore/core/decisions.py | 46 +- suncore/core/devices.py | 5 + suncore/core/forecast.py | 97 ++ suncore/core/series.py | 505 ++++++++++ suncore/core/tariff.py | 262 +++++ suncore/safety/gate.py | 10 +- suncore/safety/invariants.py | 7 +- suncore/sim/demo_config.py | 18 +- suncore/storage/config_store.py | 19 +- suncore/templates/__init__.py | 2 + suncore/templates/loader.py | 5 + suncore/templates/models.py | 206 +++- templates/README.md | 21 +- templates/schema/battery.schema.json | 513 ++++++++++ templates/schema/device.schema.json | 8 +- templates/schema/forecast.schema.json | 24 +- templates/schema/region.schema.json | 8 +- templates/schema/tariff.schema.json | 24 +- templates/schema/template.schema.json | 297 +++++- tests/api/test_config_api.py | 9 +- tests/api/test_templates_api.py | 2 +- tests/app/test_demo.py | 2 +- tests/core/test_battery.py | 170 ++++ tests/core/test_config_phase4.py | 130 +++ tests/core/test_config_sections.py | 33 +- tests/core/test_decisions.py | 1 + tests/core/test_forecast_contracts.py | 78 ++ tests/core/test_plan_v2.py | 131 +++ tests/core/test_series.py | 315 ++++++ tests/core/test_tariff.py | 186 ++++ tests/safety/test_gate.py | 28 + tests/storage/fixtures/export_phase3.json | 35 + tests/storage/test_config_sections.py | 55 +- tests/storage/test_phase4_migrations.py | 75 ++ tests/templates/test_battery_template.py | 150 +++ tests/templates/test_loader.py | 8 +- tests/templates/test_models.py | 2 +- tests/templates/test_schemas.py | 1 + 59 files changed, 5842 insertions(+), 188 deletions(-) create mode 100644 suncore/core/battery.py create mode 100644 suncore/core/forecast.py create mode 100644 suncore/core/series.py create mode 100644 suncore/core/tariff.py create mode 100644 templates/schema/battery.schema.json create mode 100644 tests/core/test_battery.py create mode 100644 tests/core/test_config_phase4.py create mode 100644 tests/core/test_forecast_contracts.py create mode 100644 tests/core/test_plan_v2.py create mode 100644 tests/core/test_series.py create mode 100644 tests/core/test_tariff.py create mode 100644 tests/storage/fixtures/export_phase3.json create mode 100644 tests/storage/test_phase4_migrations.py create mode 100644 tests/templates/test_battery_template.py diff --git a/frontend/e2e/settings.spec.ts b/frontend/e2e/settings.spec.ts index bc112e1..34c7f9b 100644 --- a/frontend/e2e/settings.spec.ts +++ b/frontend/e2e/settings.spec.ts @@ -44,7 +44,7 @@ test.describe('Settings overview', () => { test('history: restore the previous version of a section', async ({ page, request }) => { // the demo has `climate_before_export: true`; the replaced version is the one to get back await request.put('/api/v1/config/goals', { - data: { schema_version: 1, climate_before_export: false, grid_charge_night_only: true }, + data: { schema_version: 2, climate_before_export: false }, }); await page.reload(); await page.getByLabel('Section').selectOption({ label: 'Goals' }); @@ -67,7 +67,7 @@ test.describe('Settings overview', () => { expect(href).toMatch(/api\/v1\/config\/export$/); const exported = await (await request.get('/api/v1/config/export')).text(); await request.put('/api/v1/config/goals', { - data: { schema_version: 1, climate_before_export: false, grid_charge_night_only: false }, + data: { schema_version: 2, climate_before_export: false }, }); await page.getByTestId('import-file').setInputFiles({ name: 'suncore-config.json', diff --git a/frontend/e2e/wizard-review.spec.ts b/frontend/e2e/wizard-review.spec.ts index 2987ef3..8b24401 100644 --- a/frontend/e2e/wizard-review.spec.ts +++ b/frontend/e2e/wizard-review.spec.ts @@ -48,12 +48,12 @@ test.describe('dynamic tariff with only a price sensor', () => { price_entity: { entity_id: 'sensor.sim_electricity_price' }, }); expect(tariff.import.fixed_price).toBeNull(); - expect(tariff.schema_version).toBe(2); + expect(tariff.schema_version).toBe(3); }); test('the forecast choice is stored without a complete tariff', async ({ page, request }) => { await request.put('/api/v1/config/forecast', { - data: { schema_version: 1, forecast_template_ids: [], forecast_entities: {} }, + data: { schema_version: 2, sources: [] }, }); await openStep(page, 'pv'); await page.getByLabel('Solcast PV forecast').check(); @@ -63,8 +63,10 @@ test.describe('dynamic tariff with only a price sensor', () => { await next(page).click(); await expect(page).toHaveURL(/#\/setup\/battery$/); const forecast = await (await request.get('/api/v1/config/forecast')).json(); - expect(forecast.forecast_template_ids).toEqual(['solcast.pv']); - expect(forecast.forecast_entities.forecast_today.entity_id).toBe( + expect(forecast.sources.map((s: { template_id: string }) => s.template_id)).toEqual([ + 'solcast.pv', + ]); + expect(forecast.sources[0].entities.forecast_today.entity_id).toBe( 'sensor.sim_solcast_forecast_today', ); }); diff --git a/frontend/e2e/wizard.spec.ts b/frontend/e2e/wizard.spec.ts index 9aac233..92e650c 100644 --- a/frontend/e2e/wizard.spec.ts +++ b/frontend/e2e/wizard.spec.ts @@ -136,7 +136,7 @@ test.describe('setup wizard', () => { import: { kind: 'fixed', fixed_price: 0.32 }, export: { kind: 'fixed', fixed_price: 0.08 }, }); - expect(await get('forecast')).toMatchObject({ forecast_template_ids: ['solcast.pv'] }); + expect(await get('forecast')).toMatchObject({ sources: [{ template_id: 'solcast.pv' }] }); const devices = (await get('devices')).devices; expect(devices).toHaveLength(1); expect(devices[0]).toMatchObject({ template_id: 'generic.switch', priority: 0 }); diff --git a/frontend/src/lib/api/openapi.json b/frontend/src/lib/api/openapi.json index 22d568e..98704f4 100644 --- a/frontend/src/lib/api/openapi.json +++ b/frontend/src/lib/api/openapi.json @@ -77,7 +77,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -88,6 +88,19 @@ "fields": { "$ref": "#/components/schemas/SeriesFields" }, + "list_path": { + "anyOf": [ + { + "maxLength": 200, + "pattern": "^[A-Za-z0-9_]+(\\.[A-Za-z0-9_]+)*$", + "type": "string" + }, + { + "type": "null" + } + ], + "title": "List Path" + }, "roles": { "items": { "pattern": "^[a-z][a-z0-9_]{0,31}$", @@ -113,6 +126,72 @@ "title": "AttributeSeries", "type": "object" }, + "BatteryAction": { + "additionalProperties": false, + "description": "Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP9).\n\nEvery field is optional, at least one must be set; a missing field leaves that limit\nas it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a\ncharging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the\ninverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at\nor above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the\ngrid. Until the battery path exists the gate rejects this action\n(``safety.battery_not_supported``); nothing is written.", + "properties": { + "grid_charge_w": { + "anyOf": [ + { + "maximum": 1000000.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Grid Charge W" + }, + "kind": { + "const": "battery", + "default": "battery", + "title": "Kind", + "type": "string" + }, + "max_charge_w": { + "anyOf": [ + { + "maximum": 1000000.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Max Charge W" + }, + "max_discharge_w": { + "anyOf": [ + { + "maximum": 1000000.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Max Discharge W" + }, + "min_soc_pct": { + "anyOf": [ + { + "maximum": 100.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Min Soc Pct" + } + }, + "title": "BatteryAction", + "type": "object" + }, "BatteryConfig": { "additionalProperties": false, "description": "Battery: power (+ = charging), state of charge in percent, limits.\n\n``control`` is ``observe`` only: SunCore reads the battery; adapters that write\nset-points come in a later phase.", @@ -202,6 +281,220 @@ "title": "BatteryConfig", "type": "object" }, + "BatteryModeMeaning": { + "description": "What selecting a battery mode option does.", + "enum": [ + "auto", + "force_charge", + "no_discharge", + "no_charge", + "other" + ], + "title": "BatteryModeMeaning", + "type": "string" + }, + "BatteryModeOption": { + "additionalProperties": false, + "properties": { + "meaning": { + "$ref": "#/components/schemas/BatteryModeMeaning" + }, + "value": { + "maxLength": 200, + "minLength": 1, + "title": "Value", + "type": "string" + } + }, + "required": [ + "value", + "meaning" + ], + "title": "BatteryModeOption", + "type": "object" + }, + "BatteryModel": { + "additionalProperties": false, + "description": "What the planner knows about the battery.\n\nThe planning floor is ``reserve_pct + bms_drift_margin_pct``: the reserve is the user's\nbackup energy, the drift margin keeps SunCore off the BMS's own cut-off when the SoC\nreading sags (the maintainer's BMS drops from 10 % to 6-7 %). SunCore never plans to\ndischarge below it; ``max_soc_pct`` caps charging (battery care).", + "properties": { + "bms_drift_margin_pct": { + "default": 5.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Bms Drift Margin Pct", + "type": "number" + }, + "capacity_wh": { + "exclusiveMinimum": 0.0, + "maximum": 100000000.0, + "title": "Capacity Wh", + "type": "number" + }, + "charge_efficiency": { + "default": 0.95, + "exclusiveMinimum": 0.0, + "maximum": 1.0, + "title": "Charge Efficiency", + "type": "number" + }, + "degradation_cost_per_kwh": { + "default": 0.04, + "maximum": 10.0, + "minimum": 0.0, + "title": "Degradation Cost Per Kwh", + "type": "number" + }, + "discharge_efficiency": { + "default": 0.95, + "exclusiveMinimum": 0.0, + "maximum": 1.0, + "title": "Discharge Efficiency", + "type": "number" + }, + "grid_charge_margin_per_kwh": { + "default": 0.03, + "maximum": 10.0, + "minimum": 0.0, + "title": "Grid Charge Margin Per Kwh", + "type": "number" + }, + "max_charge_w": { + "exclusiveMinimum": 0.0, + "maximum": 1000000.0, + "title": "Max Charge W", + "type": "number" + }, + "max_discharge_w": { + "exclusiveMinimum": 0.0, + "maximum": 1000000.0, + "title": "Max Discharge W", + "type": "number" + }, + "max_soc_pct": { + "default": 100.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Max Soc Pct", + "type": "number" + }, + "reserve_pct": { + "default": 10.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Reserve Pct", + "type": "number" + } + }, + "required": [ + "capacity_wh", + "max_charge_w", + "max_discharge_w" + ], + "title": "BatteryModel", + "type": "object" + }, + "BatterySettings": { + "additionalProperties": false, + "description": "Configuration section ``battery`` (schema v1).\n\n``adapter_template_id = None`` means *observe only*: SunCore reads the battery through\nthe ``site`` section and plans with it, but never writes. With an adapter, ``entities``\nmaps the template's roles; the battery starts in shadow like every device (the mode\nlives in the ``operation`` section). ``target_by_time`` uses ``goals.battery_target``\n(target SoC and time); reserve and max SoC are part of ``model``.", + "properties": { + "adapter_template_id": { + "anyOf": [ + { + "maxLength": 64, + "pattern": "^[a-z][a-z0-9_]*\\.[a-z][a-z0-9_]*$", + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Adapter Template Id" + }, + "autarky_weight_per_kwh": { + "default": 0.01, + "maximum": 1.0, + "minimum": 0.0, + "title": "Autarky Weight Per Kwh", + "type": "number" + }, + "bridge_next_day": { + "default": true, + "title": "Bridge Next Day", + "type": "boolean" + }, + "calibration": { + "$ref": "#/components/schemas/CalibrationCharge", + "default": { + "enabled": false, + "interval_days": 30, + "target_soc_pct": 100.0 + } + }, + "entities": { + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "$ref": "#/components/schemas/EntityRef" + } + }, + "title": "Entities", + "type": "object" + }, + "grid_charge": { + "$ref": "#/components/schemas/GridChargePolicy", + "default": { + "enabled": false + } + }, + "model": { + "anyOf": [ + { + "$ref": "#/components/schemas/BatteryModel" + }, + { + "type": "null" + } + ] + }, + "morning_buffer_pct": { + "anyOf": [ + { + "maximum": 100.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Morning Buffer Pct" + }, + "risk": { + "$ref": "#/components/schemas/RiskLevel", + "default": "cautious" + }, + "schema_version": { + "default": 1, + "title": "Schema Version", + "type": "integer" + }, + "strategy": { + "$ref": "#/components/schemas/BatteryStrategy", + "default": "self_consumption" + } + }, + "title": "BatterySettings", + "type": "object" + }, + "BatteryStrategy": { + "description": "How the planner uses the battery (plan §1 item 5, answers 2026-10-16).", + "enum": [ + "self_consumption", + "prefer_full", + "target_by_time" + ], + "title": "BatteryStrategy", + "type": "string" + }, "BatteryTarget": { "additionalProperties": false, "description": "Be at ``soc_pct`` by local time ``time`` (``HH:MM``).", @@ -225,6 +518,170 @@ "title": "BatteryTarget", "type": "object" }, + "BatteryTemplate": { + "additionalProperties": false, + "description": "Battery adapter template (Phase 4, ADR-0006/ADR-0012 kind ``battery``).\n\n``roles`` are the readable and writable entities (e.g. ``soc``, ``power``,\n``min_soc_number``, ``max_charge_number``, ``max_discharge_number``, ``mode_select``,\n``force_charge_switch``); ``controls`` says what each *writable* role expresses. A\nrole without a control is read only. Writes go through the safety gate only.", + "properties": { + "capabilities": { + "items": { + "$ref": "#/components/schemas/Capability" + }, + "maxItems": 32, + "title": "Capabilities", + "type": "array" + }, + "controls": { + "maxProperties": 8, + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "discriminator": { + "mapping": { + "force_charge": "#/components/schemas/ForceChargeSwitchControl", + "grid_charge_power": "#/components/schemas/NumberLimitControl", + "max_charge": "#/components/schemas/NumberLimitControl", + "max_discharge": "#/components/schemas/NumberLimitControl", + "min_soc": "#/components/schemas/NumberLimitControl", + "mode": "#/components/schemas/ModeSelectControl" + }, + "propertyName": "kind" + }, + "oneOf": [ + { + "$ref": "#/components/schemas/NumberLimitControl" + }, + { + "$ref": "#/components/schemas/ModeSelectControl" + }, + { + "$ref": "#/components/schemas/ForceChargeSwitchControl" + } + ] + } + }, + "title": "Controls", + "type": "object" + }, + "id": { + "maxLength": 64, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)?$", + "title": "Id", + "type": "string" + }, + "kind": { + "const": "battery", + "title": "Kind", + "type": "string" + }, + "min_app_version": { + "anyOf": [ + { + "pattern": "^\\d+(\\.\\d+)*((a|b|rc)\\d+)?(\\.post\\d+)?(\\.dev\\d+)?$", + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Min App Version" + }, + "name": { + "maxLength": 120, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$", + "title": "Name", + "type": "string" + }, + "roles": { + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "$ref": "#/components/schemas/RoleSpec" + } + }, + "title": "Roles", + "type": "object" + }, + "schema_version": { + "const": 1, + "title": "Schema Version", + "type": "integer" + }, + "settle_s": { + "anyOf": [ + { + "maximum": 600.0, + "minimum": 0.0, + "type": "integer" + }, + { + "type": "null" + } + ], + "title": "Settle S" + }, + "source": { + "items": { + "$ref": "#/components/schemas/SourceRef" + }, + "minItems": 1, + "title": "Source", + "type": "array" + }, + "tested_with": { + "items": { + "$ref": "#/components/schemas/TestedWith" + }, + "title": "Tested With", + "type": "array" + }, + "unverified": { + "items": { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + "title": "Unverified", + "type": "array" + } + }, + "required": [ + "id", + "schema_version", + "name", + "source", + "tested_with", + "capabilities", + "kind", + "roles" + ], + "title": "BatteryTemplate", + "type": "object" + }, + "CalibrationCharge": { + "additionalProperties": false, + "description": "Periodic full charge for the BMS's SoC calibration (off by default, question 11).\n\nWhen enabled the planner schedules a charge to ``target_soc_pct`` once every\n``interval_days`` into a sunny day (from PV; grid only if the policy allows it).", + "properties": { + "enabled": { + "default": false, + "title": "Enabled", + "type": "boolean" + }, + "interval_days": { + "default": 30, + "maximum": 365.0, + "minimum": 1.0, + "title": "Interval Days", + "type": "integer" + }, + "target_soc_pct": { + "default": 100.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Target Soc Pct", + "type": "number" + } + }, + "title": "CalibrationCharge", + "type": "object" + }, "Capability": { "description": "What a template provides.", "enum": [ @@ -241,7 +698,13 @@ "pv_curve", "pv_quantiles", "feed_in_tiers", - "negative_price_rule" + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" ], "title": "Capability", "type": "string" @@ -584,6 +1047,7 @@ { "discriminator": { "mapping": { + "battery": "#/components/schemas/BatteryAction", "mode": "#/components/schemas/ModeAction", "power_setpoint": "#/components/schemas/PowerSetpointAction", "release": "#/components/schemas/ReleaseAction", @@ -611,6 +1075,9 @@ }, { "$ref": "#/components/schemas/ReleaseAction" + }, + { + "$ref": "#/components/schemas/BatteryAction" } ] }, @@ -822,6 +1289,19 @@ "resume_time": "04:00:00" } }, + "planner_value_per_kwh": { + "anyOf": [ + { + "maximum": 100.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Planner Value Per Kwh" + }, "power_model": { "$ref": "#/components/schemas/PowerModel" }, @@ -2075,35 +2555,96 @@ "title": "FlowsOut", "type": "object" }, + "ForceChargeSwitchControl": { + "additionalProperties": false, + "description": "A ``switch`` role: on = force charging (from the grid if PV is short), off = auto.", + "properties": { + "kind": { + "const": "force_charge", + "title": "Kind", + "type": "string" + } + }, + "required": [ + "kind" + ], + "title": "ForceChargeSwitchControl", + "type": "object" + }, "ForecastConfig": { "additionalProperties": false, - "description": "PV forecast sources: templates and the entities of their roles.\n\nIndependent of the tariff so that the wizard can store the choice without prices.", + "description": "PV forecast sources (schema v2: several sources, blending, learning settings).\n\nIndependent of the tariff so that the wizard can store the choice without prices.\nForecast uncertainty is handled by the learned risk model and the battery section's\n``risk`` level, not by a flat discount.", "properties": { - "forecast_entities": { + "learning": { + "$ref": "#/components/schemas/LearningSettings", + "default": { + "load_ml_mode": "shadow", + "min_history_days": 30 + } + }, + "schema_version": { + "default": 2, + "title": "Schema Version", + "type": "integer" + }, + "sources": { + "items": { + "$ref": "#/components/schemas/ForecastSourceConfig" + }, + "maxItems": 8, + "title": "Sources", + "type": "array" + } + }, + "title": "ForecastConfig", + "type": "object" + }, + "ForecastSourceConfig": { + "additionalProperties": false, + "description": "One PV forecast source: its template, the entities of its roles and its weight.\n\n``weight_mode = auto`` blends by measured skill (inverse MAE, WP5); ``manual`` uses\n``weight`` (relative, normalised over all sources by the blender).", + "properties": { + "entities": { "patternProperties": { "^[a-z][a-z0-9_]{0,31}$": { "$ref": "#/components/schemas/EntityRef" } }, - "title": "Forecast Entities", + "title": "Entities", "type": "object" }, - "forecast_template_ids": { - "items": { - "maxLength": 64, - "pattern": "^[a-z][a-z0-9_]*\\.[a-z][a-z0-9_]*$", - "type": "string" - }, - "title": "Forecast Template Ids", - "type": "array" + "template_id": { + "maxLength": 64, + "pattern": "^[a-z][a-z0-9_]*\\.[a-z][a-z0-9_]*$", + "title": "Template Id", + "type": "string" }, - "schema_version": { - "default": 1, - "title": "Schema Version", - "type": "integer" + "weight": { + "anyOf": [ + { + "exclusiveMinimum": 0.0, + "maximum": 1.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Weight" + }, + "weight_mode": { + "default": "auto", + "enum": [ + "auto", + "manual" + ], + "title": "Weight Mode", + "type": "string" } }, - "title": "ForecastConfig", + "required": [ + "template_id" + ], + "title": "ForecastSourceConfig", "type": "object" }, "ForecastTemplate": { @@ -2234,6 +2775,7 @@ }, "GoalsConfig": { "additionalProperties": false, + "description": "User goals (schema v2).\n\nv2 drops ``grid_charge_night_only`` (answers 2026-10-16, question 7): grid charging is\nallowed by a price level and/or a schedule in the ``battery`` section's\n:class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the\nwindow a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not\nlost; the planner uses it as the policy's schedule when the policy has none. **It never\nenables grid charging**: that stays off until the user enables the policy with a level\nor a schedule. ``conservative_planning`` = targets are planned on P10 PV (default on).", "properties": { "battery_target": { "anyOf": [ @@ -2250,13 +2792,23 @@ "title": "Climate Before Export", "type": "boolean" }, - "grid_charge_night_only": { + "conservative_planning": { "default": true, - "title": "Grid Charge Night Only", + "title": "Conservative Planning", "type": "boolean" }, + "grid_charge_schedule": { + "anyOf": [ + { + "$ref": "#/components/schemas/WeeklySchedule" + }, + { + "type": "null" + } + ] + }, "schema_version": { - "default": 1, + "default": 2, "title": "Schema Version", "type": "integer" } @@ -2264,6 +2816,42 @@ "title": "GoalsConfig", "type": "object" }, + "GridChargePolicy": { + "additionalProperties": false, + "description": "When the battery may be charged from the grid (answers 2026-10-16, question 7).\n\nOff by default. When enabled, a ``max_price`` (charge only at or below this import\nprice) and/or a ``schedule`` (only inside these weekly windows) must be set; both given\nmeans both must hold. On a dynamic tariff the planner's spread rule\n(:meth:`BatteryModel.grid_charge_pays`) applies in addition.", + "properties": { + "enabled": { + "default": false, + "title": "Enabled", + "type": "boolean" + }, + "max_price": { + "anyOf": [ + { + "maximum": 100.0, + "minimum": -10.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Max Price" + }, + "schedule": { + "anyOf": [ + { + "$ref": "#/components/schemas/WeeklySchedule" + }, + { + "type": "null" + } + ] + } + }, + "title": "GridChargePolicy", + "type": "object" + }, "GridConfig": { "additionalProperties": false, "description": "Grid connection: one signed sensor (+ = import) OR an import/export pair, plus limits.", @@ -2376,6 +2964,27 @@ "title": "GridConfig", "type": "object" }, + "GridFeeWindow": { + "additionalProperties": false, + "description": "A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3).", + "properties": { + "fee": { + "maximum": 100.0, + "minimum": 0.0, + "title": "Fee", + "type": "number" + }, + "window": { + "$ref": "#/components/schemas/TouWindow" + } + }, + "required": [ + "window", + "fee" + ], + "title": "GridFeeWindow", + "type": "object" + }, "HealthMessage": { "additionalProperties": false, "properties": { @@ -2475,7 +3084,7 @@ }, "ImportTariff": { "additionalProperties": false, - "description": "How the price of imported energy is obtained.", + "description": "How the price of imported energy is obtained.\n\n* ``fixed``: one ``fixed_price``.\n* ``dynamic``: a price source (``template_id`` and/or ``price_entity``); ``formula``\n turns a spot-only price into the end price (tariff v3; ``None`` = the source's price\n is used as it is, e.g. Tibber's all-in price).\n* ``tou``: fixed time-of-use prices (HT/NT, seasonal) in ``tou`` (tariff v3).", "properties": { "fixed_price": { "anyOf": [ @@ -2490,10 +3099,21 @@ ], "title": "Fixed Price" }, + "formula": { + "anyOf": [ + { + "$ref": "#/components/schemas/PriceFormula" + }, + { + "type": "null" + } + ] + }, "kind": { "enum": [ "fixed", - "dynamic" + "dynamic", + "tou" ], "title": "Kind", "type": "string" @@ -2520,6 +3140,16 @@ } ], "title": "Template Id" + }, + "tou": { + "anyOf": [ + { + "$ref": "#/components/schemas/TouTariff" + }, + { + "type": "null" + } + ] } }, "required": [ @@ -2564,6 +3194,7 @@ "action": { "discriminator": { "mapping": { + "battery": "#/components/schemas/BatteryAction", "mode": "#/components/schemas/ModeAction", "power_setpoint": "#/components/schemas/PowerSetpointAction", "release": "#/components/schemas/ReleaseAction", @@ -2591,6 +3222,9 @@ }, { "$ref": "#/components/schemas/ReleaseAction" + }, + { + "$ref": "#/components/schemas/BatteryAction" } ], "title": "Action" @@ -2632,6 +3266,31 @@ "title": "Intent", "type": "object" }, + "LearningSettings": { + "additionalProperties": false, + "description": "What SunCore may learn from history (forecast calibration, load model).\n\nKept in the ``forecast`` section because it governs how forecasts are produced. Before\n``min_history_days`` of history exist, calibration and the ML load model only collect\nstatistics and change nothing. ``load_ml_mode``: ``off``; ``shadow`` (default) = the\noptional ML load model runs in parallel and is only scored and shown for comparison;\n``auto`` = it is used when it beats the profile forecast in the accuracy statistics -\nit never replaces the profiles otherwise.", + "properties": { + "load_ml_mode": { + "default": "shadow", + "enum": [ + "off", + "shadow", + "auto" + ], + "title": "Load Ml Mode", + "type": "string" + }, + "min_history_days": { + "default": 30, + "maximum": 365.0, + "minimum": 1.0, + "title": "Min History Days", + "type": "integer" + } + }, + "title": "LearningSettings", + "type": "object" + }, "LiveMessage": { "discriminator": { "mapping": { @@ -2749,9 +3408,35 @@ } }, "required": [ - "mode" + "mode" + ], + "title": "ModeRequest", + "type": "object" + }, + "ModeSelectControl": { + "additionalProperties": false, + "description": "A ``select`` role choosing the battery's operating mode.", + "properties": { + "kind": { + "const": "mode", + "title": "Kind", + "type": "string" + }, + "options": { + "items": { + "$ref": "#/components/schemas/BatteryModeOption" + }, + "maxItems": 16, + "minItems": 2, + "title": "Options", + "type": "array" + } + }, + "required": [ + "kind", + "options" ], - "title": "ModeRequest", + "title": "ModeSelectControl", "type": "object" }, "NegativePriceRule": { @@ -2764,6 +3449,61 @@ "title": "NegativePriceRule", "type": "string" }, + "NumberLimitControl": { + "additionalProperties": false, + "description": "A ``number`` role that sets a battery limit.\n\n``min_soc``: discharge floor in ``%`` (discharge lock / reserve); ``max_charge`` and\n``max_discharge``: power limits; ``grid_charge_power``: forced charging power from the\ngrid. ``min``/``max``/``step`` are the entity's range in ``unit`` (``%`` of the\nbattery's maximum power for the power kinds).", + "properties": { + "kind": { + "enum": [ + "min_soc", + "max_charge", + "max_discharge", + "grid_charge_power" + ], + "title": "Kind", + "type": "string" + }, + "max": { + "exclusiveMinimum": 0.0, + "title": "Max", + "type": "number" + }, + "min": { + "minimum": 0.0, + "title": "Min", + "type": "number" + }, + "step": { + "anyOf": [ + { + "exclusiveMinimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Step" + }, + "unit": { + "enum": [ + "W", + "kW", + "%" + ], + "title": "Unit", + "type": "string" + } + }, + "required": [ + "kind", + "unit", + "min", + "max" + ], + "title": "NumberLimitControl", + "type": "object" + }, "OnOffControl": { "additionalProperties": false, "description": "Switch on/off through ``role``.", @@ -3251,6 +3991,58 @@ "title": "PreviewOut", "type": "object" }, + "PriceFormula": { + "additionalProperties": false, + "description": "Spot price -> end price (BRIEF §4.4).\n\n``end = (spot * spot_multiplier + grid_fee(at) + levies + margin) * (1 + vat_pct/100)``\n\n``grid_fee(at)`` is the fee of the first ``grid_fee_windows`` entry containing ``at``,\nelse ``grid_fee``. VAT is applied to the whole sum, as on a German bill; a negative\nspot price can make the end price negative. Only for ``price_includes = spot_only``\nsources - an all-in price never gets the formula on top.", + "properties": { + "grid_fee": { + "default": 0.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Grid Fee", + "type": "number" + }, + "grid_fee_windows": { + "default": [], + "items": { + "$ref": "#/components/schemas/GridFeeWindow" + }, + "maxItems": 12, + "title": "Grid Fee Windows", + "type": "array" + }, + "levies": { + "default": 0.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Levies", + "type": "number" + }, + "margin": { + "default": 0.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Margin", + "type": "number" + }, + "spot_multiplier": { + "default": 1.0, + "exclusiveMinimum": 0.0, + "maximum": 10.0, + "title": "Spot Multiplier", + "type": "number" + }, + "vat_pct": { + "default": 0.0, + "maximum": 100.0, + "minimum": 0.0, + "title": "Vat Pct", + "type": "number" + } + }, + "title": "PriceFormula", + "type": "object" + }, "PvArray": { "additionalProperties": false, "description": "One PV array (roof plane / inverter input). Everything but ``name`` is optional.", @@ -3621,6 +4413,16 @@ "title": "ReleaseAction", "type": "object" }, + "RiskLevel": { + "description": "How sure the plan must be that a battery target is met (forecast/buffer concept).\n\nThe planner plans the target on a quantile of the net energy forecast (PV minus load):\n``cautious`` meets it with probability 0.9 (10 % quantile), ``balanced`` 0.75 (25 %),\n``bold`` 0.5 (median).", + "enum": [ + "cautious", + "balanced", + "bold" + ], + "title": "RiskLevel", + "type": "string" + }, "RoleSpec": { "additionalProperties": false, "description": "One entity role of a template.", @@ -4133,7 +4935,7 @@ }, "TariffConfig": { "additionalProperties": false, - "description": "Import and export prices (the forecast sources live in :class:`ForecastConfig`).", + "description": "Import and export prices (the forecast sources live in :class:`ForecastConfig`).\n\nv3 (Phase 4): ``import.formula`` and kind ``tou``; v2 documents are valid v3 bodies.", "properties": { "export": { "$ref": "#/components/schemas/ExportTariff" @@ -4142,7 +4944,7 @@ "$ref": "#/components/schemas/ImportTariff" }, "schema_version": { - "default": 2, + "default": 3, "title": "Schema Version", "type": "integer" } @@ -4338,6 +5140,7 @@ "template": { "discriminator": { "mapping": { + "battery": "#/components/schemas/BatteryTemplate", "device": "#/components/schemas/DeviceTemplate", "forecast": "#/components/schemas/ForecastTemplate", "region": "#/components/schemas/RegionTemplate", @@ -4357,6 +5160,9 @@ }, { "$ref": "#/components/schemas/RegionTemplate" + }, + { + "$ref": "#/components/schemas/BatteryTemplate" } ], "title": "Template" @@ -4595,6 +5401,112 @@ "title": "TimeWindow", "type": "object" }, + "TouPeriod": { + "additionalProperties": false, + "description": "A price that applies inside ``window``, optionally only in some ``months``.", + "properties": { + "months": { + "anyOf": [ + { + "items": { + "maximum": 12.0, + "minimum": 1.0, + "type": "integer" + }, + "type": "array", + "uniqueItems": true + }, + { + "type": "null" + } + ], + "title": "Months" + }, + "price": { + "maximum": 100.0, + "minimum": 0.0, + "title": "Price", + "type": "number" + }, + "window": { + "$ref": "#/components/schemas/TouWindow" + } + }, + "required": [ + "window", + "price" + ], + "title": "TouPeriod", + "type": "object" + }, + "TouTariff": { + "additionalProperties": false, + "description": "Fixed time-of-use import tariff (HT/NT, seasonal).\n\nThe first period in list order that applies wins; outside every period ``base_price``\napplies. Overlaps are therefore allowed and resolved by order (put the special case\nfirst).", + "properties": { + "base_price": { + "maximum": 100.0, + "minimum": 0.0, + "title": "Base Price", + "type": "number" + }, + "periods": { + "items": { + "$ref": "#/components/schemas/TouPeriod" + }, + "maxItems": 24, + "minItems": 1, + "title": "Periods", + "type": "array" + } + }, + "required": [ + "base_price", + "periods" + ], + "title": "TouTariff", + "type": "object" + }, + "TouWindow": { + "additionalProperties": false, + "description": "A local time window on a set of weekdays, e.g. NT Mon-Fri 22:00-06:00.\n\n* ``end`` before ``start`` crosses midnight; the part after midnight belongs to the\n weekday the window *started* on (Fri 22:00-06:00 covers Saturday 00:00-06:00, not\n Friday 00:00-06:00).\n* ``start == end`` is the whole day (00:00-00:00 = 24 h).\n* ``start`` is included, ``end`` is excluded.", + "properties": { + "days": { + "default": [ + 0, + 1, + 2, + 3, + 4, + 5, + 6 + ], + "items": { + "maximum": 6.0, + "minimum": 0.0, + "type": "integer" + }, + "title": "Days", + "type": "array", + "uniqueItems": true + }, + "end": { + "pattern": "^([01]\\d|2[0-3]):[0-5]\\d$", + "title": "End", + "type": "string" + }, + "start": { + "pattern": "^([01]\\d|2[0-3]):[0-5]\\d$", + "title": "Start", + "type": "string" + } + }, + "required": [ + "start", + "end" + ], + "title": "TouWindow", + "type": "object" + }, "ValidationErrorOut": { "additionalProperties": false, "description": "422 body of a request that failed validation.", @@ -4669,6 +5581,26 @@ "title": "ValidationIssueOut", "type": "object" }, + "WeeklySchedule": { + "additionalProperties": false, + "description": "Union of weekly windows (grid-charge schedule, HT/NT windows, §14a module 3).", + "properties": { + "windows": { + "items": { + "$ref": "#/components/schemas/TouWindow" + }, + "maxItems": 28, + "minItems": 1, + "title": "Windows", + "type": "array" + } + }, + "required": [ + "windows" + ], + "title": "WeeklySchedule", + "type": "object" + }, "WizardState": { "additionalProperties": false, "properties": { @@ -4709,6 +5641,78 @@ }, "openapi": "3.1.0", "paths": { + "/api/v1/config/battery": { + "get": { + "operationId": "get_config_battery", + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/BatterySettings" + }, + { + "type": "null" + } + ], + "title": "Response Get Config Battery" + } + } + }, + "description": "Successful Response" + } + }, + "summary": "Get the battery section" + }, + "put": { + "operationId": "put_config_battery", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BatterySettings" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BatterySettings" + } + } + }, + "description": "Successful Response" + }, + "413": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorOut" + } + } + }, + "description": "Body too large" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ValidationErrorOut" + } + } + }, + "description": "Validation error" + } + }, + "summary": "Replace the battery section" + } + }, "/api/v1/config/devices": { "get": { "operationId": "get_config_devices", @@ -6128,7 +7132,8 @@ "device", "tariff", "forecast", - "region" + "region", + "battery" ], "type": "string" }, diff --git a/frontend/src/lib/api/schema.d.ts b/frontend/src/lib/api/schema.d.ts index a1aee51..a2cf786 100644 --- a/frontend/src/lib/api/schema.d.ts +++ b/frontend/src/lib/api/schema.d.ts @@ -4,6 +4,24 @@ */ export interface paths { + "/api/v1/config/battery": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** Get the battery section */ + get: operations["get_config_battery"]; + /** Replace the battery section */ + put: operations["put_config_battery"]; + post?: never; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/api/v1/config/devices": { parameters: { query?: never; @@ -613,11 +631,18 @@ export interface components { /** * AttributeSeries * @description Series held in an attribute of the entities of ``roles`` (concatenated in order). + * + * ``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's + * value to the list of points when the attribute is a mapping (``None`` = the attribute + * itself is the list). The period length of a point is the template's + * ``resolution_min``. */ AttributeSeries: { /** Attribute */ attribute: string; fields: components["schemas"]["SeriesFields"]; + /** List Path */ + list_path?: string | null; /** Roles */ roles: string[]; /** @@ -626,6 +651,33 @@ export interface components { */ source: "attribute"; }; + /** + * BatteryAction + * @description Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP9). + * + * Every field is optional, at least one must be set; a missing field leaves that limit + * as it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a + * charging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the + * inverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at + * or above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the + * grid. Until the battery path exists the gate rejects this action + * (``safety.battery_not_supported``); nothing is written. + */ + BatteryAction: { + /** Grid Charge W */ + grid_charge_w?: number | null; + /** + * @description discriminator enum property added by openapi-typescript + * @enum {string} + */ + kind: "battery"; + /** Max Charge W */ + max_charge_w?: number | null; + /** Max Discharge W */ + max_discharge_w?: number | null; + /** Min Soc Pct */ + min_soc_pct?: number | null; + }; /** * BatteryConfig * @description Battery: power (+ = charging), state of charge in percent, limits. @@ -653,6 +705,130 @@ export interface components { power: components["schemas"]["EntityRef"]; soc: components["schemas"]["EntityRef"]; }; + /** + * BatteryModeMeaning + * @description What selecting a battery mode option does. + * @enum {string} + */ + BatteryModeMeaning: "auto" | "force_charge" | "no_discharge" | "no_charge" | "other"; + /** BatteryModeOption */ + BatteryModeOption: { + meaning: components["schemas"]["BatteryModeMeaning"]; + /** Value */ + value: string; + }; + /** + * BatteryModel + * @description What the planner knows about the battery. + * + * The planning floor is ``reserve_pct + bms_drift_margin_pct``: the reserve is the user's + * backup energy, the drift margin keeps SunCore off the BMS's own cut-off when the SoC + * reading sags (the maintainer's BMS drops from 10 % to 6-7 %). SunCore never plans to + * discharge below it; ``max_soc_pct`` caps charging (battery care). + */ + BatteryModel: { + /** + * Bms Drift Margin Pct + * @default 5 + */ + bms_drift_margin_pct: number; + /** Capacity Wh */ + capacity_wh: number; + /** + * Charge Efficiency + * @default 0.95 + */ + charge_efficiency: number; + /** + * Degradation Cost Per Kwh + * @default 0.04 + */ + degradation_cost_per_kwh: number; + /** + * Discharge Efficiency + * @default 0.95 + */ + discharge_efficiency: number; + /** + * Grid Charge Margin Per Kwh + * @default 0.03 + */ + grid_charge_margin_per_kwh: number; + /** Max Charge W */ + max_charge_w: number; + /** Max Discharge W */ + max_discharge_w: number; + /** + * Max Soc Pct + * @default 100 + */ + max_soc_pct: number; + /** + * Reserve Pct + * @default 10 + */ + reserve_pct: number; + }; + /** + * BatterySettings + * @description Configuration section ``battery`` (schema v1). + * + * ``adapter_template_id = None`` means *observe only*: SunCore reads the battery through + * the ``site`` section and plans with it, but never writes. With an adapter, ``entities`` + * maps the template's roles; the battery starts in shadow like every device (the mode + * lives in the ``operation`` section). ``target_by_time`` uses ``goals.battery_target`` + * (target SoC and time); reserve and max SoC are part of ``model``. + */ + BatterySettings: { + /** Adapter Template Id */ + adapter_template_id?: string | null; + /** + * Autarky Weight Per Kwh + * @default 0.01 + */ + autarky_weight_per_kwh: number; + /** + * Bridge Next Day + * @default true + */ + bridge_next_day: boolean; + /** + * @default { + * "enabled": false, + * "interval_days": 30, + * "target_soc_pct": 100 + * } + */ + calibration: components["schemas"]["CalibrationCharge"]; + /** Entities */ + entities?: { + [key: string]: components["schemas"]["EntityRef"]; + }; + /** + * @default { + * "enabled": false + * } + */ + grid_charge: components["schemas"]["GridChargePolicy"]; + model?: components["schemas"]["BatteryModel"] | null; + /** Morning Buffer Pct */ + morning_buffer_pct?: number | null; + /** @default cautious */ + risk: components["schemas"]["RiskLevel"]; + /** + * Schema Version + * @default 1 + */ + schema_version: number; + /** @default self_consumption */ + strategy: components["schemas"]["BatteryStrategy"]; + }; + /** + * BatteryStrategy + * @description How the planner uses the battery (plan §1 item 5, answers 2026-10-16). + * @enum {string} + */ + BatteryStrategy: "self_consumption" | "prefer_full" | "target_by_time"; /** * BatteryTarget * @description Be at ``soc_pct`` by local time ``time`` (``HH:MM``). @@ -663,12 +839,81 @@ export interface components { /** Time */ time: string; }; + /** + * BatteryTemplate + * @description Battery adapter template (Phase 4, ADR-0006/ADR-0012 kind ``battery``). + * + * ``roles`` are the readable and writable entities (e.g. ``soc``, ``power``, + * ``min_soc_number``, ``max_charge_number``, ``max_discharge_number``, ``mode_select``, + * ``force_charge_switch``); ``controls`` says what each *writable* role expresses. A + * role without a control is read only. Writes go through the safety gate only. + */ + BatteryTemplate: { + /** Capabilities */ + capabilities: components["schemas"]["Capability"][]; + /** Controls */ + controls?: { + [key: string]: components["schemas"]["NumberLimitControl"] | components["schemas"]["ModeSelectControl"] | components["schemas"]["ForceChargeSwitchControl"]; + }; + /** Id */ + id: string; + /** + * @description discriminator enum property added by openapi-typescript + * @enum {string} + */ + kind: "battery"; + /** Min App Version */ + min_app_version?: string | null; + /** Name */ + name: string; + /** Roles */ + roles: { + [key: string]: components["schemas"]["RoleSpec"]; + }; + /** + * Schema Version + * @constant + */ + schema_version: 1; + /** Settle S */ + settle_s?: number | null; + /** Source */ + source: components["schemas"]["SourceRef"][]; + /** Tested With */ + tested_with: components["schemas"]["TestedWith"][]; + /** Unverified */ + unverified?: string[]; + }; + /** + * CalibrationCharge + * @description Periodic full charge for the BMS's SoC calibration (off by default, question 11). + * + * When enabled the planner schedules a charge to ``target_soc_pct`` once every + * ``interval_days`` into a sunny day (from PV; grid only if the policy allows it). + */ + CalibrationCharge: { + /** + * Enabled + * @default false + */ + enabled: boolean; + /** + * Interval Days + * @default 30 + */ + interval_days: number; + /** + * Target Soc Pct + * @default 100 + */ + target_soc_pct: number; + }; /** * Capability * @description What a template provides. * @enum {string} */ - Capability: "power_measured" | "energy_counter" | "on_off" | "power_setpoint" | "temperature_setpoint" | "hvac_modes" | "preset_modes" | "fan_modes" | "price_now" | "price_curve" | "pv_curve" | "pv_quantiles" | "feed_in_tiers" | "negative_price_rule"; + Capability: "power_measured" | "energy_counter" | "on_off" | "power_setpoint" | "temperature_setpoint" | "hvac_modes" | "preset_modes" | "fan_modes" | "price_now" | "price_curve" | "pv_curve" | "pv_quantiles" | "feed_in_tiers" | "negative_price_rule" | "battery_soc" | "min_soc_limit" | "charge_limit" | "discharge_limit" | "grid_charge" | "battery_modes"; /** CheckItemOut */ CheckItemOut: { reason: components["schemas"]["Reason"]; @@ -803,7 +1048,7 @@ export interface components { kind: components["schemas"]["DecisionKind"]; mode: components["schemas"]["OperatingMode"]; /** Observed */ - observed?: (components["schemas"]["SwitchAction"] | components["schemas"]["PowerSetpointAction"] | components["schemas"]["SetpointAction"] | components["schemas"]["StepAction"] | components["schemas"]["ModeAction"] | components["schemas"]["ReleaseAction"]) | null; + observed?: (components["schemas"]["SwitchAction"] | components["schemas"]["PowerSetpointAction"] | components["schemas"]["SetpointAction"] | components["schemas"]["StepAction"] | components["schemas"]["ModeAction"] | components["schemas"]["ReleaseAction"] | components["schemas"]["BatteryAction"]) | null; /** * Reasons * @default [] @@ -889,6 +1134,8 @@ export interface components { * } */ override_policy: components["schemas"]["OverridePolicy"]; + /** Planner Value Per Kwh */ + planner_value_per_kwh?: number | null; power_model: components["schemas"]["PowerModel"]; /** * Priority @@ -1274,24 +1521,63 @@ export interface components { /** Ts */ ts: string | null; }; + /** + * ForceChargeSwitchControl + * @description A ``switch`` role: on = force charging (from the grid if PV is short), off = auto. + */ + ForceChargeSwitchControl: { + /** + * @description discriminator enum property added by openapi-typescript + * @enum {string} + */ + kind: "force_charge"; + }; /** * ForecastConfig - * @description PV forecast sources: templates and the entities of their roles. + * @description PV forecast sources (schema v2: several sources, blending, learning settings). * * Independent of the tariff so that the wizard can store the choice without prices. + * Forecast uncertainty is handled by the learned risk model and the battery section's + * ``risk`` level, not by a flat discount. */ ForecastConfig: { - /** Forecast Entities */ - forecast_entities?: { - [key: string]: components["schemas"]["EntityRef"]; - }; - /** Forecast Template Ids */ - forecast_template_ids?: string[]; + /** + * @default { + * "load_ml_mode": "shadow", + * "min_history_days": 30 + * } + */ + learning: components["schemas"]["LearningSettings"]; /** * Schema Version - * @default 1 + * @default 2 */ schema_version: number; + /** Sources */ + sources?: components["schemas"]["ForecastSourceConfig"][]; + }; + /** + * ForecastSourceConfig + * @description One PV forecast source: its template, the entities of its roles and its weight. + * + * ``weight_mode = auto`` blends by measured skill (inverse MAE, WP5); ``manual`` uses + * ``weight`` (relative, normalised over all sources by the blender). + */ + ForecastSourceConfig: { + /** Entities */ + entities?: { + [key: string]: components["schemas"]["EntityRef"]; + }; + /** Template Id */ + template_id: string; + /** Weight */ + weight?: number | null; + /** + * Weight Mode + * @default auto + * @enum {string} + */ + weight_mode: "auto" | "manual"; }; /** ForecastTemplate */ ForecastTemplate: { @@ -1333,7 +1619,18 @@ export interface components { /** Unverified */ unverified?: string[]; }; - /** GoalsConfig */ + /** + * GoalsConfig + * @description User goals (schema v2). + * + * v2 drops ``grid_charge_night_only`` (answers 2026-10-16, question 7): grid charging is + * allowed by a price level and/or a schedule in the ``battery`` section's + * :class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the + * window a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not + * lost; the planner uses it as the policy's schedule when the policy has none. **It never + * enables grid charging**: that stays off until the user enables the policy with a level + * or a schedule. ``conservative_planning`` = targets are planned on P10 PV (default on). + */ GoalsConfig: { battery_target?: components["schemas"]["BatteryTarget"] | null; /** @@ -1342,16 +1639,36 @@ export interface components { */ climate_before_export: boolean; /** - * Grid Charge Night Only + * Conservative Planning * @default true */ - grid_charge_night_only: boolean; + conservative_planning: boolean; + grid_charge_schedule?: components["schemas"]["WeeklySchedule"] | null; /** * Schema Version - * @default 1 + * @default 2 */ schema_version: number; }; + /** + * GridChargePolicy + * @description When the battery may be charged from the grid (answers 2026-10-16, question 7). + * + * Off by default. When enabled, a ``max_price`` (charge only at or below this import + * price) and/or a ``schedule`` (only inside these weekly windows) must be set; both given + * means both must hold. On a dynamic tariff the planner's spread rule + * (:meth:`BatteryModel.grid_charge_pays`) applies in addition. + */ + GridChargePolicy: { + /** + * Enabled + * @default false + */ + enabled: boolean; + /** Max Price */ + max_price?: number | null; + schedule?: components["schemas"]["WeeklySchedule"] | null; + }; /** * GridConfig * @description Grid connection: one signed sensor (+ = import) OR an import/export pair, plus limits. @@ -1375,6 +1692,15 @@ export interface components { para14a_signal?: components["schemas"]["EntityRef"] | null; power?: components["schemas"]["EntityRef"] | null; }; + /** + * GridFeeWindow + * @description A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3). + */ + GridFeeWindow: { + /** Fee */ + fee: number; + window: components["schemas"]["TouWindow"]; + }; /** HealthMessage */ HealthMessage: { health: components["schemas"]["HealthOut"]; @@ -1414,18 +1740,26 @@ export interface components { /** * ImportTariff * @description How the price of imported energy is obtained. + * + * * ``fixed``: one ``fixed_price``. + * * ``dynamic``: a price source (``template_id`` and/or ``price_entity``); ``formula`` + * turns a spot-only price into the end price (tariff v3; ``None`` = the source's price + * is used as it is, e.g. Tibber's all-in price). + * * ``tou``: fixed time-of-use prices (HT/NT, seasonal) in ``tou`` (tariff v3). */ ImportTariff: { /** Fixed Price */ fixed_price?: number | null; + formula?: components["schemas"]["PriceFormula"] | null; /** * Kind * @enum {string} */ - kind: "fixed" | "dynamic"; + kind: "fixed" | "dynamic" | "tou"; price_entity?: components["schemas"]["EntityRef"] | null; /** Template Id */ template_id?: string | null; + tou?: components["schemas"]["TouTariff"] | null; }; /** InfoOut */ InfoOut: { @@ -1445,7 +1779,7 @@ export interface components { */ Intent: { /** Action */ - action: components["schemas"]["SwitchAction"] | components["schemas"]["PowerSetpointAction"] | components["schemas"]["SetpointAction"] | components["schemas"]["StepAction"] | components["schemas"]["ModeAction"] | components["schemas"]["ReleaseAction"]; + action: components["schemas"]["SwitchAction"] | components["schemas"]["PowerSetpointAction"] | components["schemas"]["SetpointAction"] | components["schemas"]["StepAction"] | components["schemas"]["ModeAction"] | components["schemas"]["ReleaseAction"] | components["schemas"]["BatteryAction"]; /** Asset Id */ asset_id: string; /** Id */ @@ -1462,6 +1796,30 @@ export interface components { */ ts: string; }; + /** + * LearningSettings + * @description What SunCore may learn from history (forecast calibration, load model). + * + * Kept in the ``forecast`` section because it governs how forecasts are produced. Before + * ``min_history_days`` of history exist, calibration and the ML load model only collect + * statistics and change nothing. ``load_ml_mode``: ``off``; ``shadow`` (default) = the + * optional ML load model runs in parallel and is only scored and shown for comparison; + * ``auto`` = it is used when it beats the profile forecast in the accuracy statistics - + * it never replaces the profiles otherwise. + */ + LearningSettings: { + /** + * Load Ml Mode + * @default shadow + * @enum {string} + */ + load_ml_mode: "off" | "shadow" | "auto"; + /** + * Min History Days + * @default 30 + */ + min_history_days: number; + }; LiveMessage: components["schemas"]["SnapshotMessage"] | components["schemas"]["FlowsMessage"] | components["schemas"]["HealthMessage"] | components["schemas"]["DecisionsMessage"] | components["schemas"]["ControlMessage"]; /** * Mode @@ -1508,12 +1866,52 @@ export interface components { ModeRequest: { mode: components["schemas"]["OperatingMode"]; }; + /** + * ModeSelectControl + * @description A ``select`` role choosing the battery's operating mode. + */ + ModeSelectControl: { + /** + * @description discriminator enum property added by openapi-typescript + * @enum {string} + */ + kind: "mode"; + /** Options */ + options: components["schemas"]["BatteryModeOption"][]; + }; /** * NegativePriceRule * @description Feed-in remuneration during negative spot prices. * @enum {string} */ NegativePriceRule: "none" | "no_remuneration" | "curtailment_required"; + /** + * NumberLimitControl + * @description A ``number`` role that sets a battery limit. + * + * ``min_soc``: discharge floor in ``%`` (discharge lock / reserve); ``max_charge`` and + * ``max_discharge``: power limits; ``grid_charge_power``: forced charging power from the + * grid. ``min``/``max``/``step`` are the entity's range in ``unit`` (``%`` of the + * battery's maximum power for the power kinds). + */ + NumberLimitControl: { + /** + * @description discriminator enum property added by openapi-typescript + * @enum {string} + */ + kind: "grid_charge_power" | "max_charge" | "max_discharge" | "min_soc"; + /** Max */ + max: number; + /** Min */ + min: number; + /** Step */ + step?: number | null; + /** + * Unit + * @enum {string} + */ + unit: "W" | "kW" | "%"; + }; /** * OnOffControl * @description Switch on/off through ``role``. @@ -1716,6 +2114,49 @@ export interface components { /** Items */ items: components["schemas"]["PreviewItemOut"][]; }; + /** + * PriceFormula + * @description Spot price -> end price (BRIEF §4.4). + * + * ``end = (spot * spot_multiplier + grid_fee(at) + levies + margin) * (1 + vat_pct/100)`` + * + * ``grid_fee(at)`` is the fee of the first ``grid_fee_windows`` entry containing ``at``, + * else ``grid_fee``. VAT is applied to the whole sum, as on a German bill; a negative + * spot price can make the end price negative. Only for ``price_includes = spot_only`` + * sources - an all-in price never gets the formula on top. + */ + PriceFormula: { + /** + * Grid Fee + * @default 0 + */ + grid_fee: number; + /** + * Grid Fee Windows + * @default [] + */ + grid_fee_windows: components["schemas"]["GridFeeWindow"][]; + /** + * Levies + * @default 0 + */ + levies: number; + /** + * Margin + * @default 0 + */ + margin: number; + /** + * Spot Multiplier + * @default 1 + */ + spot_multiplier: number; + /** + * Vat Pct + * @default 0 + */ + vat_pct: number; + }; /** * PvArray * @description One PV array (roof plane / inverter input). Everything but ``name`` is optional. @@ -1857,6 +2298,16 @@ export interface components { */ kind: "release"; }; + /** + * RiskLevel + * @description How sure the plan must be that a battery target is met (forecast/buffer concept). + * + * The planner plans the target on a quantile of the net energy forecast (PV minus load): + * ``cautious`` meets it with probability 0.9 (10 % quantile), ``balanced`` 0.75 (25 %), + * ``bold`` 0.5 (median). + * @enum {string} + */ + RiskLevel: "cautious" | "balanced" | "bold"; /** * RoleSpec * @description One entity role of a template. @@ -2063,13 +2514,15 @@ export interface components { /** * TariffConfig * @description Import and export prices (the forecast sources live in :class:`ForecastConfig`). + * + * v3 (Phase 4): ``import.formula`` and kind ``tou``; v2 documents are valid v3 bodies. */ TariffConfig: { export: components["schemas"]["ExportTariff"]; import: components["schemas"]["ImportTariff"]; /** * Schema Version - * @default 2 + * @default 3 */ schema_version: number; }; @@ -2130,7 +2583,7 @@ export interface components { */ origin: "builtin" | "user"; /** Template */ - template: components["schemas"]["DeviceTemplate"] | components["schemas"]["TariffTemplate"] | components["schemas"]["ForecastTemplate"] | components["schemas"]["RegionTemplate"]; + template: components["schemas"]["DeviceTemplate"] | components["schemas"]["TariffTemplate"] | components["schemas"]["ForecastTemplate"] | components["schemas"]["RegionTemplate"] | components["schemas"]["BatteryTemplate"]; }; /** * TemplateErrorOut @@ -2235,6 +2688,60 @@ export interface components { */ start: string; }; + /** + * TouPeriod + * @description A price that applies inside ``window``, optionally only in some ``months``. + */ + TouPeriod: { + /** Months */ + months?: number[] | null; + /** Price */ + price: number; + window: components["schemas"]["TouWindow"]; + }; + /** + * TouTariff + * @description Fixed time-of-use import tariff (HT/NT, seasonal). + * + * The first period in list order that applies wins; outside every period ``base_price`` + * applies. Overlaps are therefore allowed and resolved by order (put the special case + * first). + */ + TouTariff: { + /** Base Price */ + base_price: number; + /** Periods */ + periods: components["schemas"]["TouPeriod"][]; + }; + /** + * TouWindow + * @description A local time window on a set of weekdays, e.g. NT Mon-Fri 22:00-06:00. + * + * * ``end`` before ``start`` crosses midnight; the part after midnight belongs to the + * weekday the window *started* on (Fri 22:00-06:00 covers Saturday 00:00-06:00, not + * Friday 00:00-06:00). + * * ``start == end`` is the whole day (00:00-00:00 = 24 h). + * * ``start`` is included, ``end`` is excluded. + */ + TouWindow: { + /** + * Days + * @default [ + * 0, + * 1, + * 2, + * 3, + * 4, + * 5, + * 6 + * ] + */ + days: number[]; + /** End */ + end: string; + /** Start */ + start: string; + }; /** * ValidationErrorOut * @description 422 body of a request that failed validation. @@ -2259,6 +2766,14 @@ export interface components { /** Loc */ loc: (string | number)[]; }; + /** + * WeeklySchedule + * @description Union of weekly windows (grid-charge schedule, HT/NT windows, §14a module 3). + */ + WeeklySchedule: { + /** Windows */ + windows: components["schemas"]["TouWindow"][]; + }; /** WizardState */ WizardState: { /** @@ -2288,6 +2803,68 @@ export interface components { } export type $defs = Record; export interface operations { + get_config_battery: { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["BatterySettings"] | null; + }; + }; + }; + }; + put_config_battery: { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["BatterySettings"]; + }; + }; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["BatterySettings"]; + }; + }; + /** @description Body too large */ + 413: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ErrorOut"]; + }; + }; + /** @description Validation error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ValidationErrorOut"]; + }; + }; + }; + }; get_config_devices: { parameters: { query?: never; @@ -3393,7 +3970,7 @@ export interface operations { templates_api_v1_templates_get: { parameters: { query?: { - kind?: ("device" | "tariff" | "forecast" | "region") | null; + kind?: ("device" | "tariff" | "forecast" | "region" | "battery") | null; }; header?: never; path?: never; diff --git a/frontend/src/lib/api/types.ts b/frontend/src/lib/api/types.ts index 09e21bd..730e340 100644 --- a/frontend/src/lib/api/types.ts +++ b/frontend/src/lib/api/types.ts @@ -28,12 +28,21 @@ export type EntityCandidate = S['EntityCandidateOut']; export type ValidationIssue = S['ValidationIssueOut']; export type ConfigSectionName = - 'site' | 'region' | 'tariff' | 'forecast' | 'devices' | 'goals' | 'operation' | 'wizard'; + | 'site' + | 'region' + | 'tariff' + | 'forecast' + | 'devices' + | 'goals' + | 'operation' + | 'wizard' + | 'battery'; export type RegionConfig = S['RegionConfig']; export type TariffConfig = S['TariffConfig']; export type ForecastConfig = S['ForecastConfig']; export type DevicesConfig = S['DevicesConfig']; export type GoalsConfig = S['GoalsConfig']; +export type BatterySettings = S['BatterySettings']; export type OperationConfig = S['OperationConfig']; export type WizardState = S['WizardState']; export type DeviceConfig = S['DeviceConfig']; @@ -55,7 +64,7 @@ export type Suggestion = S['SuggestionOut']; export type ConfigHistoryEntry = S['ConfigHistoryEntryOut']; export type ImportResult = S['ImportResultOut']; export type HistoryRestore = S['HistoryRestoreOut']; -export type TemplateKind = 'device' | 'tariff' | 'forecast' | 'region'; +export type TemplateKind = 'device' | 'tariff' | 'forecast' | 'region' | 'battery'; export type TemplateList = S['TemplateListOut']; export type TemplateSummary = S['TemplateSummaryOut']; export type TemplateDetail = S['TemplateDetailOut']; diff --git a/frontend/src/lib/control/actions.ts b/frontend/src/lib/control/actions.ts index 5ca7b57..a85168f 100644 --- a/frontend/src/lib/control/actions.ts +++ b/frontend/src/lib/control/actions.ts @@ -28,5 +28,7 @@ export function describeAction(action: Action, power: (w: number) => string): Ac return { key: 'action.mode', args: { mode: action.mode } }; case 'release': return { key: 'action.release', args: {} }; + case 'battery': + return { key: 'action.battery', args: {} }; } } diff --git a/frontend/src/lib/control/present.test.ts b/frontend/src/lib/control/present.test.ts index 117e751..b569d7c 100644 --- a/frontend/src/lib/control/present.test.ts +++ b/frontend/src/lib/control/present.test.ts @@ -90,6 +90,7 @@ describe('describeAction', () => { expect(describeAction({ kind: 'step', step: 2 }, w).key).toBe('action.step'); expect(describeAction({ kind: 'mode', mode: 'eco' }, w).key).toBe('action.mode'); expect(describeAction({ kind: 'release' }, w).key).toBe('action.release'); + expect(describeAction({ kind: 'battery', min_soc_pct: 20 }, w).key).toBe('action.battery'); }); }); diff --git a/frontend/src/lib/i18n/locales/de.json b/frontend/src/lib/i18n/locales/de.json index 97459f6..a31a06a 100644 --- a/frontend/src/lib/i18n/locales/de.json +++ b/frontend/src/lib/i18n/locales/de.json @@ -826,5 +826,24 @@ "wizard.step.pv": "Photovoltaik", "wizard.step.region": "Region und Sprache", "wizard.step.summary": "Zusammenfassung und Check", - "wizard.step.tariff": "Tarif" + "wizard.step.tariff": "Tarif", + "api.validation.battery_action_empty": "Eine Batterie-Aktion braucht mindestens ein Limit.", + "api.validation.battery_corridor": "Reserve plus Drift-Puffer ({floor} %) müssen unter dem maximalen Ladestand ({max} %) liegen.", + "api.validation.battery_entities_need_adapter": "Batterie-Entitäten brauchen einen Batterie-Adapter; „nur beobachten“ hat keinen.", + "api.validation.forecast_auto_weight": "Eine Gewichtung wird nur bei manueller Gewichtung verwendet.", + "api.validation.forecast_manual_weight": "Eine manuelle Gewichtung braucht ein Gewicht.", + "api.validation.grid_charge_needs_limit": "Laden aus dem Netz braucht eine Preisgrenze oder einen Zeitplan.", + "api.validation.import_formula_dynamic_only": "Eine Preisformel gilt nur für einen dynamischen Tarif.", + "api.validation.import_tou_extra": "Ein zeitvariabler Tarif (HT/NT) hat keinen Festpreis, keine Vorlage und keinen Preissensor.", + "api.validation.import_tou_needs_periods": "Ein zeitvariabler Tarif (HT/NT) braucht seine Zeitfenster und Preise.", + "api.validation.import_tou_only": "Zeitfenster mit Preisen brauchen einen zeitvariablen Tarif (HT/NT).", + "api.validation.price_grid": "Bezugs- und Einspeisepreise brauchen dasselbe Zeitraster.", + "api.validation.price_unit": "Preise müssen pro kWh angegeben sein.", + "api.validation.quantile_grid": "Die Quantile der Prognose brauchen dasselbe Zeitraster.", + "api.validation.quantile_order": "Die Quantile der Prognose müssen geordnet sein (P10 ≤ P50 ≤ P90).", + "api.validation.quantile_unit": "Die Quantile der Prognose brauchen dieselbe Einheit.", + "api.validation.range_order": "Der Beginn muss vor dem Ende liegen.", + "api.validation.series_step": "Die Intervalllänge muss zwischen 0 und einem Tag liegen.", + "safety.battery_not_supported": "Batteriesteuerung ist noch nicht verfügbar; es wurde nichts geschrieben.", + "action.battery": "Batterie-Limits setzen" } diff --git a/frontend/src/lib/i18n/locales/en.json b/frontend/src/lib/i18n/locales/en.json index faff812..6d375cf 100644 --- a/frontend/src/lib/i18n/locales/en.json +++ b/frontend/src/lib/i18n/locales/en.json @@ -826,5 +826,24 @@ "wizard.step.pv": "Solar", "wizard.step.region": "Region and language", "wizard.step.summary": "Summary and check", - "wizard.step.tariff": "Tariff" + "wizard.step.tariff": "Tariff", + "api.validation.battery_action_empty": "A battery action needs at least one limit.", + "api.validation.battery_corridor": "Reserve plus drift margin ({floor} %) must be below the maximum state of charge ({max} %).", + "api.validation.battery_entities_need_adapter": "Battery entities need a battery adapter; \"observe only\" has none.", + "api.validation.forecast_auto_weight": "A weight is only used with a manual weighting.", + "api.validation.forecast_manual_weight": "A manual weighting needs a weight.", + "api.validation.grid_charge_needs_limit": "Grid charging needs a price level or a schedule.", + "api.validation.import_formula_dynamic_only": "A price formula only applies to a dynamic tariff.", + "api.validation.import_tou_extra": "A time-of-use tariff takes no fixed price, template or price sensor.", + "api.validation.import_tou_needs_periods": "A time-of-use tariff needs its time windows and prices.", + "api.validation.import_tou_only": "Time windows with prices need the time-of-use tariff.", + "api.validation.price_grid": "Import and export prices need the same time grid.", + "api.validation.price_unit": "Prices must be given per kWh.", + "api.validation.quantile_grid": "The forecast quantiles need the same time grid.", + "api.validation.quantile_order": "The forecast quantiles must be ordered (P10 ≤ P50 ≤ P90).", + "api.validation.quantile_unit": "The forecast quantiles need the same unit.", + "api.validation.range_order": "The start must be before the end.", + "api.validation.series_step": "The interval length must be between 0 and one day.", + "safety.battery_not_supported": "Battery control is not available yet; nothing was written.", + "action.battery": "Set battery limits" } diff --git a/frontend/src/lib/views/Settings.svelte b/frontend/src/lib/views/Settings.svelte index 849f290..314be37 100644 --- a/frontend/src/lib/views/Settings.svelte +++ b/frontend/src/lib/views/Settings.svelte @@ -12,7 +12,8 @@ import { STEPS, type StepId } from '../wizard/steps'; const uid = $props.id(); - const SECTIONS: Exclude[] = [ + // The battery section gets its settings page in Phase 4 WP10. + const SECTIONS: Exclude[] = [ 'site', 'region', 'tariff', diff --git a/frontend/src/lib/wizard/forecastForm.ts b/frontend/src/lib/wizard/forecastForm.ts index 88dd312..ce709b6 100644 --- a/frontend/src/lib/wizard/forecastForm.ts +++ b/frontend/src/lib/wizard/forecastForm.ts @@ -1,39 +1,64 @@ // SPDX-License-Identifier: Apache-2.0 -// Forecast section <-> form: the PV forecast sources (templates plus the entities of their roles). -// Independent of the tariff, so the choice can be stored without prices. +// Forecast section (v2) <-> form: the PV forecast sources (templates plus the entities of their +// roles). Independent of the tariff, so the choice can be stored without prices. Weights and +// learning settings have no form fields yet (Phase 4 WP10); they are carried through unchanged. import type { ForecastConfig } from '../api/types'; import { emptyRef, fromRef, toRef, type RefField } from './forms'; +type Source = NonNullable[number]; + export interface ForecastForm { templateIds: string[]; /** Forecast role -> entity; rows without an entity are not sent. */ entities: Record; + /** Roles per template id, when known; a template without an entry gets every entity. */ + roles: Record; + /** Stored weighting per template id (carried through). */ + weights: Record>; + learning: ForecastConfig['learning'] | null; } export function emptyForecastForm(): ForecastForm { - return { templateIds: [], entities: {} }; + return { templateIds: [], entities: {}, roles: {}, weights: {}, learning: null }; } export function forecastFormFromConfig(cfg: ForecastConfig | null): ForecastForm { if (!cfg) return emptyForecastForm(); - return { - templateIds: [...(cfg.forecast_template_ids ?? [])], - entities: Object.fromEntries( - Object.entries(cfg.forecast_entities ?? {}).map(([role, ref]) => [role, fromRef(ref)]), - ), - }; + const form = emptyForecastForm(); + form.learning = cfg.learning; + for (const source of cfg.sources ?? []) { + form.templateIds.push(source.template_id); + form.weights[source.template_id] = { weight_mode: source.weight_mode, weight: source.weight }; + for (const [role, ref] of Object.entries(source.entities ?? {})) { + form.entities[role] ??= fromRef(ref); + } + } + return form; } export function forecastFromForm(form: ForecastForm): ForecastConfig { - const entities: ForecastConfig['forecast_entities'] = {}; + const entities: Record>> = {}; for (const [role, field] of Object.entries(form.entities)) { const ref = toRef(field); if (ref) entities[role] = ref; } + const sources: Source[] = form.templateIds.map((id) => { + const roles = form.roles[id]; + const own = roles + ? Object.fromEntries(Object.entries(entities).filter(([role]) => roles.includes(role))) + : { ...entities }; + const weight = form.weights[id]; + return { + template_id: id, + entities: own, + weight_mode: weight?.weight_mode ?? 'auto', + weight: weight?.weight ?? null, + }; + }); return { - schema_version: 1, - forecast_template_ids: [...form.templateIds], - forecast_entities: entities, + schema_version: 2, + sources, + learning: form.learning ?? { min_history_days: 30, load_ml_mode: 'shadow' }, }; } @@ -45,10 +70,13 @@ export function setForecastTemplate( roles: readonly string[], ): void { const has = form.templateIds.includes(id); + if (on) form.roles[id] = [...roles]; if (on && !has) form.templateIds.push(id); if (!on && has) { form.templateIds = form.templateIds.filter((x) => x !== id); for (const role of roles) delete form.entities[role]; + delete form.roles[id]; + delete form.weights[id]; } } diff --git a/frontend/src/lib/wizard/forms.test.ts b/frontend/src/lib/wizard/forms.test.ts index 4ea1f01..1b76990 100644 --- a/frontend/src/lib/wizard/forms.test.ts +++ b/frontend/src/lib/wizard/forms.test.ts @@ -31,6 +31,7 @@ import { import { num, slugify, uniqueId } from './forms'; import { goalsFormFromConfig, + NIGHT_SCHEDULE, goalsFromForm, operationFormFromConfig, operationFromForm, @@ -128,29 +129,46 @@ describe('region form', () => { }); describe('forecast form', () => { + const learning = { min_history_days: 30, load_ml_mode: 'shadow' as const }; it('round-trips a stored forecast without any tariff', () => { const cfg: ForecastConfig = { - schema_version: 1, - forecast_template_ids: ['solcast.pv'], - forecast_entities: { forecast_today: ref('sensor.today') }, + schema_version: 2, + sources: [ + { + template_id: 'solcast.pv', + entities: { forecast_today: ref('sensor.today') }, + weight_mode: 'manual', + weight: 0.5, + }, + ], + learning: { min_history_days: 14, load_ml_mode: 'off' }, }; expect(forecastFromForm(forecastFormFromConfig(cfg))).toEqual(cfg); expect(forecastFromForm(forecastFormFromConfig(null))).toEqual({ - schema_version: 1, - forecast_template_ids: [], - forecast_entities: {}, + schema_version: 2, + sources: [], + learning, }); }); + it('gives each source the entities of its own roles', () => { + const form = emptyForecastForm(); + setForecastTemplate(form, 'a.pv', true, ['today']); + setForecastTemplate(form, 'b.pv', true, ['b_today']); + form.entities.today = ref('sensor.a'); + form.entities.b_today = ref('sensor.b'); + const sources = forecastFromForm(form).sources!; + expect(sources.map((s) => Object.keys(s.entities ?? {}))).toEqual([['today'], ['b_today']]); + }); it('drops forecast entities of a template that is switched off', () => { const form = emptyForecastForm(); setForecastTemplate(form, 'solcast.pv', true, ['forecast_today']); ensureRoleFields(form, ['forecast_today', 'forecast_tomorrow']); form.entities.forecast_today = ref('sensor.today'); - expect(Object.keys(forecastFromForm(form).forecast_entities!)).toEqual(['forecast_today']); + expect(Object.keys(forecastFromForm(form).sources![0]!.entities!)).toEqual(['forecast_today']); setForecastTemplate(form, 'solcast.pv', false, ['forecast_today', 'forecast_tomorrow']); expect(form.templateIds).toEqual([]); expect(form.entities).toEqual({}); - expect(forecastFromForm(form).forecast_entities).toEqual({}); + expect(forecastFromForm(form).sources).toEqual([]); }); }); @@ -162,8 +180,15 @@ describe('tariff form', () => { form.exportKind = 'none'; expect(isTariffComplete(form)).toBe(true); expect(tariffFromForm(form)).toEqual({ - schema_version: 2, - import: { kind: 'fixed', template_id: null, price_entity: null, fixed_price: 0.32 }, + schema_version: 3, + import: { + kind: 'fixed', + template_id: null, + price_entity: null, + fixed_price: 0.32, + formula: null, + tou: null, + }, export: { kind: 'none', fixed_price: null, tiers: [] }, }); }); @@ -181,6 +206,8 @@ describe('tariff form', () => { template_id: 'tibber.price', price_entity: ref('sensor.tibber'), fixed_price: null, + formula: null, + tou: null, }); expect(isTariffComplete(form)).toBe(true); }); @@ -212,12 +239,21 @@ describe('tariff form', () => { }); it('round-trips a stored tariff', () => { const cfg: TariffConfig = { - schema_version: 2, + schema_version: 3, import: { kind: 'dynamic', template_id: 'tibber.price', price_entity: ref('sensor.t'), fixed_price: null, + formula: { + spot_multiplier: 1, + grid_fee: 0.1, + grid_fee_windows: [], + levies: 0.02, + margin: 0.01, + vat_pct: 19, + }, + tou: null, }, export: { kind: 'tiered', @@ -230,6 +266,27 @@ describe('tariff form', () => { }; expect(tariffFromForm(tariffFormFromConfig(cfg))).toEqual(cfg); }); + it('carries a time-of-use tariff through unchanged', () => { + const tou = { + base_price: 0.3, + periods: [{ window: { days: [0, 1, 2, 3, 4], start: '22:00', end: '06:00' }, price: 0.2 }], + }; + const cfg: TariffConfig = { + schema_version: 3, + import: { + kind: 'tou', + template_id: null, + price_entity: null, + fixed_price: null, + formula: null, + tou, + }, + export: { kind: 'none', fixed_price: null, tiers: [] }, + }; + const form = tariffFormFromConfig(cfg); + expect(isTariffComplete(form)).toBe(true); + expect(tariffFromForm(form)).toEqual(cfg); + }); it('offers region tiers as rows', () => { expect( tiersFromRegion({ @@ -775,6 +832,11 @@ describe('goals and operation', () => { climateBeforeExport: false, }); expect(goalsFromForm(form).battery_target).toBeNull(); + expect(goalsFromForm(form).grid_charge_schedule).toEqual(NIGHT_SCHEDULE); + expect(goalsFromForm(form).conservative_planning).toBe(true); + form.gridChargeNightOnly = false; + expect(goalsFromForm(form).grid_charge_schedule).toBeNull(); + expect(goalsFormFromConfig(goalsFromForm(form)).gridChargeNightOnly).toBe(false); form.targetEnabled = true; form.targetSoc = 90; form.targetTime = '17:30'; diff --git a/frontend/src/lib/wizard/goalsForm.ts b/frontend/src/lib/wizard/goalsForm.ts index 4b738e2..4e8cf1d 100644 --- a/frontend/src/lib/wizard/goalsForm.ts +++ b/frontend/src/lib/wizard/goalsForm.ts @@ -7,9 +7,17 @@ export interface GoalsForm { targetSoc: number | null; targetTime: string; climateBeforeExport: boolean; + /** Goals v2: `true` = the grid-charge schedule is the night window 22:00-06:00. It never + * enables grid charging (that is the battery section's policy). */ gridChargeNightOnly: boolean; + conservativePlanning: boolean; } +/** The night window goals v1's `grid_charge_night_only` migrates to (22:00-06:00 daily). */ +export const NIGHT_SCHEDULE: NonNullable = { + windows: [{ days: [0, 1, 2, 3, 4, 5, 6], start: '22:00', end: '06:00' }], +}; + export const DEFAULT_TARGET_TIME = '18:00'; export function goalsFormFromConfig(cfg: GoalsConfig | null): GoalsForm { @@ -18,19 +26,21 @@ export function goalsFormFromConfig(cfg: GoalsConfig | null): GoalsForm { targetSoc: cfg?.battery_target?.soc_pct ?? 100, targetTime: cfg?.battery_target?.time ?? DEFAULT_TARGET_TIME, climateBeforeExport: cfg?.climate_before_export ?? false, - gridChargeNightOnly: cfg?.grid_charge_night_only ?? true, + gridChargeNightOnly: cfg ? cfg.grid_charge_schedule != null : true, + conservativePlanning: cfg?.conservative_planning ?? true, }; } export function goalsFromForm(form: GoalsForm): GoalsConfig { return { - schema_version: 1, + schema_version: 2, battery_target: form.targetEnabled && form.targetSoc != null ? { soc_pct: form.targetSoc, time: form.targetTime } : null, climate_before_export: form.climateBeforeExport, - grid_charge_night_only: form.gridChargeNightOnly, + grid_charge_schedule: form.gridChargeNightOnly ? NIGHT_SCHEDULE : null, + conservative_planning: form.conservativePlanning, }; } diff --git a/frontend/src/lib/wizard/tariffForm.ts b/frontend/src/lib/wizard/tariffForm.ts index 1c567ea..740441f 100644 --- a/frontend/src/lib/wizard/tariffForm.ts +++ b/frontend/src/lib/wizard/tariffForm.ts @@ -1,9 +1,11 @@ // SPDX-License-Identifier: Apache-2.0 // Tariff section <-> form: import price (fixed or dynamic) and feed-in (fixed, tiered, none). -import type { ExportTier, RegionTemplate, TariffConfig } from '../api/types'; +// Tariff v3 (price formula, HT/NT kind `tou`) has no form fields yet (Phase 4 WP10): the values +// are carried through unchanged so that saving the form never drops them. +import type { ExportTier, ImportTariff, RegionTemplate, TariffConfig } from '../api/types'; import { emptyRef, fromRef, toRef, type RefField } from './forms'; -export type ImportKind = 'fixed' | 'dynamic'; +export type ImportKind = 'fixed' | 'dynamic' | 'tou'; export type DynamicSource = 'template' | 'entity'; export type ExportKind = 'fixed' | 'tiered' | 'none'; @@ -22,6 +24,9 @@ export interface TariffForm { exportKind: ExportKind; exportFixed: number | null; tiers: TierRow[]; + /** Carried through unchanged (no form fields yet). */ + importFormula: ImportTariff['formula']; + importTou: ImportTariff['tou']; } export function emptyTariffForm(): TariffForm { @@ -34,6 +39,8 @@ export function emptyTariffForm(): TariffForm { exportKind: 'fixed', exportFixed: null, tiers: [{ upToKwp: null, price: null }], + importFormula: null, + importTou: null, }; } @@ -53,17 +60,21 @@ export function tariffFormFromConfig(cfg: TariffConfig | null): TariffForm { exp.tiers && exp.tiers.length > 0 ? exp.tiers.map((t) => ({ upToKwp: t.up_to_kwp ?? null, price: t.price })) : [{ upToKwp: null, price: null }], + importFormula: imp.formula ?? null, + importTou: imp.tou ?? null, }; } /** Enough to store the tariff section (the backend requires import and export). */ export function isTariffComplete(form: TariffForm): boolean { const importOk = - form.importKind === 'fixed' - ? form.importFixed != null - : form.dynamicSource === 'template' - ? form.importTemplateId !== '' - : toRef(form.priceEntity) !== null; + form.importKind === 'tou' + ? form.importTou != null + : form.importKind === 'fixed' + ? form.importFixed != null + : form.dynamicSource === 'template' + ? form.importTemplateId !== '' + : toRef(form.priceEntity) !== null; const exportOk = form.exportKind === 'none' || (form.exportKind === 'fixed' && form.exportFixed != null) || @@ -85,12 +96,14 @@ export function tariffFromForm(form: TariffForm): TariffConfig { })) : []; return { - schema_version: 2, + schema_version: 3, import: { kind: form.importKind, template_id: byTemplate ? form.importTemplateId : null, price_entity: dynamic ? toRef(form.priceEntity) : null, - fixed_price: dynamic ? null : form.importFixed, + fixed_price: form.importKind === 'fixed' ? form.importFixed : null, + formula: dynamic ? (form.importFormula ?? null) : null, + tou: form.importKind === 'tou' ? (form.importTou ?? null) : null, }, export: { kind: form.exportKind, diff --git a/scripts/gen_compat.py b/scripts/gen_compat.py index e26cdfa..c79519a 100644 --- a/scripts/gen_compat.py +++ b/scripts/gen_compat.py @@ -24,9 +24,10 @@ EN_CATALOGUE = ROOT / "frontend" / "src" / "lib" / "i18n" / "locales" / "en.json" DEFAULT_OUTPUT = ROOT / "docs" / "user" / "compatibility.md" -KIND_ORDER = ("device", "tariff", "forecast", "region") +KIND_ORDER = ("device", "battery", "tariff", "forecast", "region") KIND_TITLES = { "device": "Devices", + "battery": "Batteries", "tariff": "Tariffs", "forecast": "PV forecasts", "region": "Regions", diff --git a/suncore/api/validation.py b/suncore/api/validation.py index bfdf9e0..a3de64f 100644 --- a/suncore/api/validation.py +++ b/suncore/api/validation.py @@ -109,6 +109,23 @@ "suncore.percent_needs_nominal": "api.validation.percent_needs_nominal", "suncore.plan_interval_order": "api.validation.plan_interval_order", "suncore.plan_overlap": "api.validation.plan_overlap", + "suncore.battery_action_empty": "api.validation.battery_action_empty", + "suncore.battery_corridor": "api.validation.battery_corridor", + "suncore.battery_entities_need_adapter": "api.validation.battery_entities_need_adapter", + "suncore.forecast_auto_weight": "api.validation.forecast_auto_weight", + "suncore.forecast_manual_weight": "api.validation.forecast_manual_weight", + "suncore.grid_charge_needs_limit": "api.validation.grid_charge_needs_limit", + "suncore.import_formula_dynamic_only": "api.validation.import_formula_dynamic_only", + "suncore.import_tou_extra": "api.validation.import_tou_extra", + "suncore.import_tou_needs_periods": "api.validation.import_tou_needs_periods", + "suncore.import_tou_only": "api.validation.import_tou_only", + "suncore.price_grid": "api.validation.price_grid", + "suncore.price_unit": "api.validation.price_unit", + "suncore.quantile_grid": "api.validation.quantile_grid", + "suncore.quantile_order": "api.validation.quantile_order", + "suncore.quantile_unit": "api.validation.quantile_unit", + "suncore.range_order": "api.validation.range_order", + "suncore.series_step": "api.validation.series_step", "suncore.power_model_counter": "api.validation.power_model_counter", "suncore.power_model_measured": "api.validation.power_model_measured", "suncore.power_model_nominal": "api.validation.power_model_nominal", diff --git a/suncore/core/battery.py b/suncore/core/battery.py new file mode 100644 index 0000000..616f99c --- /dev/null +++ b/suncore/core/battery.py @@ -0,0 +1,284 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Battery model, strategies and the ``battery`` configuration section (Phase 4 WP2). + +Units and signs per ``core/units.py``: W, Wh, percent, currency/kWh; battery + = charging. +The defaults are the maintainer's answers to plan §6 question 6 (efficiency 95 % each way, +degradation 0.04/kWh throughput, grid-charge margin 0.03/kWh, BMS-drift margin 5 %), all +expert parameters. + +:class:`BatteryAction` (the gate's battery intent) lives in :mod:`suncore.core.decisions` +with the other action variants and is re-exported here. +""" + +from __future__ import annotations + +from datetime import datetime, tzinfo +from enum import StrEnum, unique +from typing import Annotated, Final, Self + +from pydantic import BaseModel, ConfigDict, Field, model_validator + +from suncore.core.decisions import BatteryAction +from suncore.core.devices import RoleName, TemplateId +from suncore.core.site import EntityRef +from suncore.core.tariff import WeeklySchedule +from suncore.core.validation import invalid + +Pct = Annotated[float, Field(ge=0, le=100, allow_inf_nan=False)] +Efficiency = Annotated[float, Field(gt=0, le=1, allow_inf_nan=False)] +PosW = Annotated[float, Field(gt=0, le=1_000_000, allow_inf_nan=False)] +UnitCost = Annotated[float, Field(ge=0, le=10, allow_inf_nan=False)] +"""Currency per kWh.""" + +DEFAULT_EFFICIENCY: Final = 0.95 +DEFAULT_DEGRADATION_PER_KWH: Final = 0.04 +DEFAULT_GRID_CHARGE_MARGIN_PER_KWH: Final = 0.03 +DEFAULT_BMS_DRIFT_PCT: Final = 5.0 +DEFAULT_RESERVE_PCT: Final = 10.0 + + +class _Frozen(BaseModel): + model_config = ConfigDict(frozen=True, extra="forbid") + + +class BatteryModel(_Frozen): + """What the planner knows about the battery. + + The planning floor is ``reserve_pct + bms_drift_margin_pct``: the reserve is the user's + backup energy, the drift margin keeps SunCore off the BMS's own cut-off when the SoC + reading sags (the maintainer's BMS drops from 10 % to 6-7 %). SunCore never plans to + discharge below it; ``max_soc_pct`` caps charging (battery care). + """ + + capacity_wh: Annotated[float, Field(gt=0, le=100_000_000, allow_inf_nan=False)] + charge_efficiency: Efficiency = DEFAULT_EFFICIENCY + discharge_efficiency: Efficiency = DEFAULT_EFFICIENCY + max_charge_w: PosW + max_discharge_w: PosW + reserve_pct: Pct = DEFAULT_RESERVE_PCT + bms_drift_margin_pct: Pct = DEFAULT_BMS_DRIFT_PCT + max_soc_pct: Pct = 100.0 + degradation_cost_per_kwh: UnitCost = DEFAULT_DEGRADATION_PER_KWH + """Cost of one kWh of throughput (charged energy), currency/kWh.""" + grid_charge_margin_per_kwh: UnitCost = DEFAULT_GRID_CHARGE_MARGIN_PER_KWH + """Safety margin on top of losses and degradation before grid charging pays.""" + + @model_validator(mode="after") + def _corridor(self) -> Self: + if not self.min_soc_pct < self.max_soc_pct: + raise invalid( + "battery_corridor", + "battery: reserve + drift margin ({floor}) must be below max_soc_pct ({max})", + floor=self.min_soc_pct, + max=self.max_soc_pct, + ) + return self + + @property + def min_soc_pct(self) -> float: + """Planning floor: reserve plus BMS-drift margin.""" + return self.reserve_pct + self.bms_drift_margin_pct + + @property + def round_trip_efficiency(self) -> float: + return self.charge_efficiency * self.discharge_efficiency + + def soc_to_wh(self, soc_pct: float) -> float: + return self.capacity_wh * soc_pct / 100 + + def wh_to_soc(self, energy_wh: float) -> float: + return 100 * energy_wh / self.capacity_wh + + @property + def usable_wh(self) -> float: + """Stored energy between the planning floor and ``max_soc_pct``.""" + return self.soc_to_wh(self.max_soc_pct - self.min_soc_pct) + + def min_later_price(self, charge_price: float) -> float: + """Spread rule: the import price later must exceed this for grid charging to pay. + + One kWh bought at ``charge_price`` delivers ``round_trip_efficiency`` kWh later and + costs degradation and the safety margin on top: + ``later * rte > charge_price + degradation + margin``. + """ + cost = charge_price + self.degradation_cost_per_kwh + self.grid_charge_margin_per_kwh + return cost / self.round_trip_efficiency + + def grid_charge_pays(self, charge_price: float, later_price: float) -> bool: + return later_price > self.min_later_price(charge_price) + + +@unique +class BatteryStrategy(StrEnum): + """How the planner uses the battery (plan §1 item 5, answers 2026-10-16).""" + + SELF_CONSUMPTION = "self_consumption" + """Default: store surplus, discharge for the house, no grid charging unless allowed.""" + PREFER_FULL = "prefer_full" + """As full as possible: target 100 % (capped by ``max_soc_pct``) as early as the P10 + forecast allows, PV surplus to the battery before consumers until full, discharge + only for the house (never for export).""" + TARGET_BY_TIME = "target_by_time" + """Reach ``goals.battery_target`` by its time, planned on P10 PV.""" + + +@unique +class RiskLevel(StrEnum): + """How sure the plan must be that a battery target is met (forecast/buffer concept). + + The planner plans the target on a quantile of the net energy forecast (PV minus load): + ``cautious`` meets it with probability 0.9 (10 % quantile), ``balanced`` 0.75 (25 %), + ``bold`` 0.5 (median). + """ + + CAUTIOUS = "cautious" + BALANCED = "balanced" + BOLD = "bold" + + @property + def probability(self) -> float: + """Probability with which the target is met.""" + return _RISK_PROBABILITY[self] + + @property + def quantile(self) -> float: + """Quantile of the net energy forecast the target constraint uses.""" + return round(1.0 - self.probability, 6) + + +_RISK_PROBABILITY: Final[dict[RiskLevel, float]] = { + RiskLevel.CAUTIOUS: 0.9, + RiskLevel.BALANCED: 0.75, + RiskLevel.BOLD: 0.5, +} + +DEFAULT_AUTARKY_WEIGHT_PER_KWH: Final = 0.01 + + +class GridChargePolicy(_Frozen): + """When the battery may be charged from the grid (answers 2026-10-16, question 7). + + Off by default. When enabled, a ``max_price`` (charge only at or below this import + price) and/or a ``schedule`` (only inside these weekly windows) must be set; both given + means both must hold. On a dynamic tariff the planner's spread rule + (:meth:`BatteryModel.grid_charge_pays`) applies in addition. + """ + + enabled: bool = False + max_price: Annotated[float, Field(ge=-10, le=100, allow_inf_nan=False)] | None = None + """Currency/kWh; may be negative (charge only when paid to import).""" + schedule: WeeklySchedule | None = None + + @model_validator(mode="after") + def _limited(self) -> Self: + if self.enabled and self.max_price is None and self.schedule is None: + raise invalid( + "grid_charge_needs_limit", + "battery.grid_charge: enabling needs a price level or a schedule", + ) + return self + + def allows(self, at: datetime, import_price: float | None, tz: tzinfo | None = None) -> bool: + """Whether this policy admits grid charging at ``at`` with ``import_price``. + + An unknown price never satisfies a price level. + """ + if not self.enabled: + return False + if self.schedule is not None and not self.schedule.contains(at, tz): + return False + if self.max_price is not None: + return import_price is not None and import_price <= self.max_price + return True + + +class CalibrationCharge(_Frozen): + """Periodic full charge for the BMS's SoC calibration (off by default, question 11). + + When enabled the planner schedules a charge to ``target_soc_pct`` once every + ``interval_days`` into a sunny day (from PV; grid only if the policy allows it). + """ + + enabled: bool = False + interval_days: Annotated[int, Field(ge=1, le=365)] = 30 + target_soc_pct: Pct = 100.0 + + +BATTERY_SCHEMA_VERSION: Final = 1 + + +class BatterySettings(_Frozen): + """Configuration section ``battery`` (schema v1). + + ``adapter_template_id = None`` means *observe only*: SunCore reads the battery through + the ``site`` section and plans with it, but never writes. With an adapter, ``entities`` + maps the template's roles; the battery starts in shadow like every device (the mode + lives in the ``operation`` section). ``target_by_time`` uses ``goals.battery_target`` + (target SoC and time); reserve and max SoC are part of ``model``. + """ + + schema_version: int = BATTERY_SCHEMA_VERSION + adapter_template_id: TemplateId | None = None + entities: dict[RoleName, EntityRef] = Field(default_factory=dict) + model: BatteryModel | None = None + """``None`` until the user (wizard step 5) provides capacity and limits.""" + strategy: BatteryStrategy = BatteryStrategy.SELF_CONSUMPTION + grid_charge: GridChargePolicy = GridChargePolicy() + calibration: CalibrationCharge = CalibrationCharge() + risk: RiskLevel = RiskLevel.CAUTIOUS + """Maintainer's choice: targets are met with probability 0.9 by default.""" + morning_buffer_pct: Pct | None = None + """Manual floor of "battery first until X %" in the morning; ``None`` = automatic from + the forecast uncertainty.""" + bridge_next_day: bool = True + """Raise the evening target up to ``max_soc_pct`` when the 48 h horizon shows a poor + next day.""" + autarky_weight_per_kwh: Annotated[float, Field(ge=0, le=1, allow_inf_nan=False)] = ( + DEFAULT_AUTARKY_WEIGHT_PER_KWH + ) + """Currency/kWh penalty on grid import added to the cost objective. The default + (0.01) is a tie-breaker: it prefers self-sufficiency only where the cost difference is + below 1 ct/kWh, which also removes LP degeneracy on flat tariffs; 0 = pure cost.""" + + @model_validator(mode="after") + def _consistent(self) -> Self: + if self.entities and self.adapter_template_id is None: + raise invalid( + "battery_entities_need_adapter", + "battery: role entities need an adapter template (observe only has none)", + ) + seen: set[str] = set() + for ref in self.entities.values(): + if ref.entity_id in seen: + raise invalid( + "entity_reused", + "entity {entity_id} is used more than once", + entity_id=ref.entity_id, + ) + seen.add(ref.entity_id) + return self + + @property + def observe_only(self) -> bool: + return self.adapter_template_id is None + + def entity_refs(self) -> list[EntityRef]: + return list(self.entities.values()) + + +__all__ = [ + "BATTERY_SCHEMA_VERSION", + "DEFAULT_AUTARKY_WEIGHT_PER_KWH", + "DEFAULT_BMS_DRIFT_PCT", + "DEFAULT_DEGRADATION_PER_KWH", + "DEFAULT_EFFICIENCY", + "DEFAULT_GRID_CHARGE_MARGIN_PER_KWH", + "DEFAULT_RESERVE_PCT", + "BatteryAction", + "BatteryModel", + "BatterySettings", + "BatteryStrategy", + "CalibrationCharge", + "GridChargePolicy", + "RiskLevel", +] diff --git a/suncore/core/capabilities.py b/suncore/core/capabilities.py index f96e700..06569dc 100644 --- a/suncore/core/capabilities.py +++ b/suncore/core/capabilities.py @@ -28,3 +28,10 @@ class Capability(StrEnum): PV_QUANTILES = "pv_quantiles" FEED_IN_TIERS = "feed_in_tiers" NEGATIVE_PRICE_RULE = "negative_price_rule" + # Phase 4 (battery kind, additive): + BATTERY_SOC = "battery_soc" + MIN_SOC_LIMIT = "min_soc_limit" + CHARGE_LIMIT = "charge_limit" + DISCHARGE_LIMIT = "discharge_limit" + GRID_CHARGE = "grid_charge" + BATTERY_MODES = "battery_modes" diff --git a/suncore/core/config.py b/suncore/core/config.py index b7e6e27..e2a2631 100644 --- a/suncore/core/config.py +++ b/suncore/core/config.py @@ -9,7 +9,7 @@ import itertools from collections.abc import Mapping -from typing import Annotated, Final, Literal, Self +from typing import Annotated, Any, Final, Literal, Self from zoneinfo import ZoneInfo, ZoneInfoNotFoundError from pydantic import ( @@ -21,15 +21,17 @@ model_validator, ) +from suncore.core.battery import BatterySettings from suncore.core.decisions import OperatingMode from suncore.core.devices import DeviceConfig, DeviceId, RoleName, TemplateId from suncore.core.site import EntityRef, SiteConfig +from suncore.core.tariff import HHMM_PATTERN, NIGHT_WINDOW, PriceFormula, TouTariff, WeeklySchedule from suncore.core.validation import invalid REGION_ID_PATTERN: Final = r"^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)?$" """Region id: ``de`` or ``.``; also the region template id pattern.""" STEP_PATTERN: Final = r"^[a-z][a-z0-9_]{0,47}$" -TIME_PATTERN: Final = r"^([01]\d|2[0-3]):[0-5]\d$" +TIME_PATTERN: Final = HHMM_PATTERN Price = Annotated[float, Field(ge=0, le=100)] """Currency per kWh.""" @@ -70,12 +72,21 @@ def _known_zone(cls, value: str) -> str: class ImportTariff(_Frozen): - """How the price of imported energy is obtained.""" + """How the price of imported energy is obtained. - kind: Literal["fixed", "dynamic"] + * ``fixed``: one ``fixed_price``. + * ``dynamic``: a price source (``template_id`` and/or ``price_entity``); ``formula`` + turns a spot-only price into the end price (tariff v3; ``None`` = the source's price + is used as it is, e.g. Tibber's all-in price). + * ``tou``: fixed time-of-use prices (HT/NT, seasonal) in ``tou`` (tariff v3). + """ + + kind: Literal["fixed", "dynamic", "tou"] template_id: TemplateId | None = None price_entity: EntityRef | None = None fixed_price: Price | None = None + formula: PriceFormula | None = None + tou: TouTariff | None = None @model_validator(mode="after") def _consistent(self) -> Self: @@ -89,15 +100,34 @@ def _consistent(self) -> Self: "import_fixed_extra", "tariff.import: kind `fixed` takes no `template_id`/`price_entity`", ) - elif self.template_id is None and self.price_entity is None: - raise invalid( - "import_dynamic_needs_source", - "tariff.import: kind `dynamic` needs `template_id` or `price_entity`", - ) - elif self.fixed_price is not None: + elif self.kind == "dynamic": + if self.template_id is None and self.price_entity is None: + raise invalid( + "import_dynamic_needs_source", + "tariff.import: kind `dynamic` needs `template_id` or `price_entity`", + ) + if self.fixed_price is not None: + raise invalid( + "import_dynamic_extra", "tariff.import: kind `dynamic` takes no `fixed_price`" + ) + else: + if self.tou is None: + raise invalid("import_tou_needs_periods", "tariff.import: kind `tou` needs `tou`") + if ( + self.fixed_price is not None + or self.template_id is not None + or self.price_entity is not None + ): + raise invalid( + "import_tou_extra", + "tariff.import: kind `tou` takes no fixed price, template or price entity", + ) + if self.formula is not None and self.kind != "dynamic": raise invalid( - "import_dynamic_extra", "tariff.import: kind `dynamic` takes no `fixed_price`" + "import_formula_dynamic_only", "tariff.import: `formula` needs kind `dynamic`" ) + if self.tou is not None and self.kind != "tou": + raise invalid("import_tou_only", "tariff.import: `tou` needs kind `tou`") return self @@ -145,7 +175,10 @@ def _consistent(self) -> Self: class TariffConfig(_Frozen): - """Import and export prices (the forecast sources live in :class:`ForecastConfig`).""" + """Import and export prices (the forecast sources live in :class:`ForecastConfig`). + + v3 (Phase 4): ``import.formula`` and kind ``tou``; v2 documents are valid v3 bodies. + """ model_config = ConfigDict(frozen=True, extra="forbid", populate_by_name=True) """``import`` is a Python keyword: the JSON key is ``import``, ``import_`` is accepted too. @@ -154,7 +187,7 @@ class TariffConfig(_Frozen): before the alias existed (key ``import_``) still load. """ - schema_version: int = 2 + schema_version: int = 3 import_: ImportTariff = Field(alias="import") export: ExportTariff @@ -165,27 +198,98 @@ def entity_refs(self) -> list[EntityRef]: # --- forecast ----------------------------------------------------------------- +class ForecastSourceConfig(_Frozen): + """One PV forecast source: its template, the entities of its roles and its weight. + + ``weight_mode = auto`` blends by measured skill (inverse MAE, WP5); ``manual`` uses + ``weight`` (relative, normalised over all sources by the blender). + """ + + template_id: TemplateId + entities: dict[RoleName, EntityRef] = Field(default_factory=dict) + """Role of the template (e.g. ``forecast_today``) to entity.""" + weight_mode: Literal["auto", "manual"] = "auto" + weight: Annotated[float, Field(gt=0, le=1)] | None = None + + @model_validator(mode="after") + def _weight(self) -> Self: + if self.weight_mode == "manual" and self.weight is None: + raise invalid("forecast_manual_weight", "forecast: a manual weight needs `weight`") + if self.weight_mode == "auto" and self.weight is not None: + raise invalid( + "forecast_auto_weight", "forecast: `weight` is only used with weight_mode manual" + ) + return self + + +class LearningSettings(_Frozen): + """What SunCore may learn from history (forecast calibration, load model). + + Kept in the ``forecast`` section because it governs how forecasts are produced. Before + ``min_history_days`` of history exist, calibration and the ML load model only collect + statistics and change nothing. ``load_ml_mode``: ``off``; ``shadow`` (default) = the + optional ML load model runs in parallel and is only scored and shown for comparison; + ``auto`` = it is used when it beats the profile forecast in the accuracy statistics - + it never replaces the profiles otherwise. + """ + + min_history_days: Annotated[int, Field(ge=1, le=365)] = 30 + load_ml_mode: Literal["off", "shadow", "auto"] = "shadow" + + class ForecastConfig(_Frozen): - """PV forecast sources: templates and the entities of their roles. + """PV forecast sources (schema v2: several sources, blending, learning settings). Independent of the tariff so that the wizard can store the choice without prices. + Forecast uncertainty is handled by the learned risk model and the battery section's + ``risk`` level, not by a flat discount. """ - schema_version: int = 1 - forecast_template_ids: list[TemplateId] = Field(default_factory=list) - forecast_entities: dict[RoleName, EntityRef] = Field(default_factory=dict) - """Forecast role (e.g. ``today``, ``tomorrow``) to entity.""" + schema_version: int = 2 + sources: Annotated[list[ForecastSourceConfig], Field(max_length=8)] = Field( + default_factory=list + ) + learning: LearningSettings = LearningSettings() @model_validator(mode="after") def _unique_templates(self) -> Self: - if len(set(self.forecast_template_ids)) != len(self.forecast_template_ids): - raise invalid( - "forecast_templates_unique", "forecast: forecast_template_ids must be unique" - ) + ids = [s.template_id for s in self.sources] + if len(set(ids)) != len(ids): + raise invalid("forecast_templates_unique", "forecast: every template only once") return self + @property + def template_ids(self) -> list[str]: + return [s.template_id for s in self.sources] + def entity_refs(self) -> list[EntityRef]: - return list(self.forecast_entities.values()) + refs: dict[str, EntityRef] = {} + for source in self.sources: + for ref in source.entities.values(): + refs.setdefault(ref.entity_id, ref) + return list(refs.values()) + + +def migrate_forecast_v1_to_v2(body: dict[str, Any]) -> dict[str, Any]: + """v1 (template ids + one role map) -> v2 (one source per template, weights auto). + + v1 kept a single role map for all templates, so it cannot say which entity belongs to + which template. Every source gets the whole map: the adapter reads only the roles its + template defines, nothing is lost, and the wizard writes clean per-source maps on the + next save. (Only one forecast template, ``solcast.pv``, existed in v1.) + """ + ids = body.pop("forecast_template_ids", None) or [] + entities = body.pop("forecast_entities", None) or {} + if not isinstance(ids, list) or not isinstance(entities, dict): + msg = "forecast v1: `forecast_template_ids` must be a list, `forecast_entities` a map" + raise TypeError(msg) + body["sources"] = [{"template_id": i, "entities": dict(entities)} for i in ids] + return body + + +def migrate_tariff_v2_to_v3(body: dict[str, Any]) -> dict[str, Any]: + """Tariff v3 only adds optional fields (``formula``, kind ``tou``): unchanged body.""" + return body # --- devices ------------------------------------------------------------------ @@ -226,10 +330,35 @@ class BatteryTarget(_Frozen): class GoalsConfig(_Frozen): - schema_version: int = 1 + """User goals (schema v2). + + v2 drops ``grid_charge_night_only`` (answers 2026-10-16, question 7): grid charging is + allowed by a price level and/or a schedule in the ``battery`` section's + :class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the + window a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not + lost; the planner uses it as the policy's schedule when the policy has none. **It never + enables grid charging**: that stays off until the user enables the policy with a level + or a schedule. ``conservative_planning`` = targets are planned on P10 PV (default on). + """ + + schema_version: int = 2 battery_target: BatteryTarget | None = None climate_before_export: bool = False - grid_charge_night_only: bool = True + grid_charge_schedule: WeeklySchedule | None = None + conservative_planning: bool = True + + +def migrate_goals_v1_to_v2(body: dict[str, Any]) -> dict[str, Any]: + """``grid_charge_night_only`` (v1 default ``True``) -> ``grid_charge_schedule``. + + ``True`` becomes the night window 22:00-06:00 every day, ``False`` no schedule. Grid + charging itself is *not* enabled by this migration (see :class:`GoalsConfig`). + """ + night_only = body.pop("grid_charge_night_only", True) + body["grid_charge_schedule"] = ( + NIGHT_WINDOW.model_dump(mode="json") if night_only is True else None + ) + return body # --- operation ---------------------------------------------------------------- @@ -280,5 +409,6 @@ class WizardState(_Frozen): "goals": GoalsConfig, "operation": OperationConfig, "wizard": WizardState, + "battery": BatterySettings, } """All configuration sections by name and their document model (API and storage share it).""" diff --git a/suncore/core/control.py b/suncore/core/control.py index 8f0d9c2..38f392a 100644 --- a/suncore/core/control.py +++ b/suncore/core/control.py @@ -13,9 +13,16 @@ from dataclasses import dataclass, field from datetime import datetime from enum import StrEnum, unique -from typing import Annotated, Final, Self +from typing import Annotated, Any, Final, Literal, Self -from pydantic import AwareDatetime, BaseModel, ConfigDict, Field, model_validator +from pydantic import ( + AwareDatetime, + BaseModel, + ConfigDict, + Field, + StringConstraints, + model_validator, +) from suncore.core.balance import FlowSnapshot from suncore.core.decisions import Reason @@ -181,6 +188,11 @@ class BatteryBand(_Frozen): max_soc_pct: Annotated[float, Field(ge=0, le=100)] | None = None charge_from_surplus: bool = True """``False`` = surplus goes to consumers before the battery (``climate_before_export``).""" + priority_until_soc_pct: Annotated[float, Field(ge=0, le=100)] | None = None + """Plan v2: battery first until the *live* SoC reaches this value, then surplus may go + to consumers. Evaluated by the controller against the measured SoC every tick (robust + against plan deviation); ``None`` = no such rule. Accepted and stored now, the + controller wiring follows in Phase 4 WP9 (until then the controller ignores it).""" @model_validator(mode="after") def _ordered(self) -> Self: @@ -193,8 +205,46 @@ def _ordered(self) -> Self: return self +class IntervalFlows(_Frozen): + """Plan v2: the flows the planner expects in an interval (mean power, W). + + Signs per ``core/units.py``: import/export are magnitudes, battery + = charging. + ``pv_p50_w`` is what the objective used; ``pv_p10_w`` the conservative value the + target constraints used (``None`` = the source has no quantiles). ``soc_pct`` is the + planned state of charge at the *end* of the interval (``None`` = no battery). + """ + + pv_p10_w: float | None = Field(default=None, ge=0, allow_inf_nan=False) + pv_p50_w: float = Field(ge=0, allow_inf_nan=False) + pv_p90_w: float | None = Field(default=None, ge=0, allow_inf_nan=False) + load_w: float = Field(ge=0, allow_inf_nan=False) + import_w: float = Field(ge=0, allow_inf_nan=False) + export_w: float = Field(ge=0, allow_inf_nan=False) + battery_w: float = Field(default=0.0, allow_inf_nan=False) + soc_pct: Annotated[float, Field(ge=0, le=100)] | None = None + + @model_validator(mode="after") + def _quantiles(self) -> Self: + lo = self.pv_p10_w if self.pv_p10_w is not None else self.pv_p50_w + hi = self.pv_p90_w if self.pv_p90_w is not None else self.pv_p50_w + if not lo <= self.pv_p50_w <= hi: + raise invalid("quantile_order", "quantiles: p10 <= p50 <= p90 is violated") + return self + + +class IntervalPrices(_Frozen): + """Plan v2: end prices (currency/kWh) the planner used; may be negative.""" + + import_price: float = Field(allow_inf_nan=False) + export_price: float = Field(allow_inf_nan=False) + + class PlanInterval(_Frozen): - """One planning interval (15 min grid in Phase 4; any length is accepted).""" + """One planning interval (15 min grid in Phase 4; any length is accepted). + + ``expected``, ``prices`` and ``reasons`` are the plan v2 additions for the UI's + "Why?"; the controller reads only the bands and the import limit. + """ start: AwareDatetime end: AwareDatetime @@ -202,6 +252,9 @@ class PlanInterval(_Frozen): """Device id -> band; a missing device gets :data:`DEFAULT_BAND`.""" battery: BatteryBand = BatteryBand() grid_import_limit_w: float | None = Field(default=None, allow_inf_nan=False, ge=0) + expected: IntervalFlows | None = None + prices: IntervalPrices | None = None + reasons: tuple[Reason, ...] = () @model_validator(mode="after") def _ordered(self) -> Self: @@ -217,20 +270,48 @@ def covers(self, at: datetime) -> bool: DEFAULT_BATTERY_BAND: Final = BatteryBand() +PLAN_SCHEMA_VERSION: Final = 2 +"""Version of new plans; v1 (Phase 3, no expected flows/prices/source/stats) still loads.""" + +PlanSource = Literal["solver", "heuristic", "default"] +"""Who made the plan: the HiGHS model, the heuristic fallback, or nobody (allow all).""" + +StatusSlug = Annotated[str, StringConstraints(pattern=r"^[a-z][a-z0-9_]*$", max_length=48)] + + +class SolverStats(_Frozen): + """Plan v2: how the plan was computed (the UI explains long solves with it).""" + + solve_time_s: float = Field(ge=0, allow_inf_nan=False) + status: StatusSlug + """Solver or heuristic status, e.g. ``optimal``, ``time_limit``, ``infeasible``.""" + objective: float | None = Field(default=None, allow_inf_nan=False) + """Objective value in currency (``None`` = not available, e.g. heuristic).""" + n_vars: int = Field(default=0, ge=0) + n_binaries: int = Field(default=0, ge=0) + mip_gap: float | None = Field(default=None, ge=0, allow_inf_nan=False) + + class Plan(_Frozen): """The planner's output as the controller consumes it (ADR-0011). Phase 3 only ever uses :meth:`Plan.default`, which has no intervals and therefore allows everything; the Phase 4 planner fills ``intervals`` without changing the controller. Intervals must not overlap and are kept in start order. + + Schema v2 (Phase 4 WP2, additive): ``source``, ``stats`` and per interval + ``expected``/``prices``/``reasons``. A v1 payload loads unchanged (it keeps + ``schema_version`` 1, ``source = "default"``); :func:`plan_from_payload` stamps v2. """ - schema_version: int = 1 + schema_version: Annotated[int, Field(ge=1)] = PLAN_SCHEMA_VERSION created_at: AwareDatetime | None = None intervals: tuple[PlanInterval, ...] = () reasons: tuple[Reason, ...] = () assumptions: dict[str, str | int | float | bool] = Field(default_factory=dict) """Forecast and price sources, mode, solver status - scalars only.""" + source: PlanSource = "default" + stats: SolverStats | None = None @model_validator(mode="after") def _non_overlapping(self) -> Self: @@ -269,6 +350,24 @@ def battery_band_at(self, at: datetime) -> BatteryBand: return DEFAULT_BATTERY_BAND if interval is None else interval.battery +def plan_from_payload(payload: Mapping[str, Any]) -> Plan: + """Load a stored plan of any known schema version (v1 rows are stamped v2). + + Every v1 field exists unchanged in v2, so the upgrade only stamps the version. A + payload newer than :data:`PLAN_SCHEMA_VERSION` raises ``ValueError``. + """ + data = dict(payload) + version = data.get("schema_version", 1) + if not isinstance(version, int) or isinstance(version, bool) or version < 1: + msg = f"plan payload has an invalid schema_version {version!r}" + raise ValueError(msg) + if version > PLAN_SCHEMA_VERSION: + msg = f"plan payload schema_version {version} is newer than {PLAN_SCHEMA_VERSION}" + raise ValueError(msg) + data["schema_version"] = PLAN_SCHEMA_VERSION + return Plan.model_validate(data) + + TEMPERATURE_ROLE: Final = "temperature" """The role of a thermal device's own temperature probe (water heater tank).""" @@ -302,6 +401,7 @@ def heat_capped(device: DeviceConfig, state: DeviceState | None) -> bool: __all__ = [ # RESUME_TIME_DEFAULT is re-exported from core.devices (single constant) "DEFAULT_BAND", "DEFAULT_BATTERY_BAND", + "PLAN_SCHEMA_VERSION", "RESUME_TIME_DEFAULT", "TEMPERATURE_ROLE", "BandPreference", @@ -310,9 +410,14 @@ def heat_capped(device: DeviceConfig, state: DeviceState | None) -> bool: "DeviceBand", "DeviceState", "GateInputs", + "IntervalFlows", + "IntervalPrices", "Plan", "PlanInterval", + "PlanSource", "PowerEstimate", + "SolverStats", "has_heat_limit", "heat_capped", + "plan_from_payload", ] diff --git a/suncore/core/decisions.py b/suncore/core/decisions.py index 1f0e2a0..60a0ee7 100644 --- a/suncore/core/decisions.py +++ b/suncore/core/decisions.py @@ -20,9 +20,11 @@ import re from collections.abc import Mapping from enum import StrEnum, unique -from typing import Annotated, Any, Final, Literal +from typing import Annotated, Any, Final, Literal, Self -from pydantic import AwareDatetime, BaseModel, ConfigDict, Field +from pydantic import AwareDatetime, BaseModel, ConfigDict, Field, model_validator + +from suncore.core.validation import invalid ReasonArg = str | int | float | bool """Scalar placeholder values a catalogue message may interpolate.""" @@ -113,6 +115,7 @@ class ActionKind(StrEnum): STEP = "step" MODE = "mode" RELEASE = "release" + BATTERY = "battery" class SwitchAction(_FrozenModel): @@ -171,8 +174,44 @@ class ReleaseAction(_FrozenModel): kind: Literal[ActionKind.RELEASE] = ActionKind.RELEASE +class BatteryAction(_FrozenModel): + """Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP9). + + Every field is optional, at least one must be set; a missing field leaves that limit + as it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a + charging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the + inverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at + or above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the + grid. Until the battery path exists the gate rejects this action + (``safety.battery_not_supported``); nothing is written. + """ + + kind: Literal[ActionKind.BATTERY] = ActionKind.BATTERY + max_charge_w: float | None = Field(default=None, ge=0, le=1_000_000, allow_inf_nan=False) + max_discharge_w: float | None = Field(default=None, ge=0, le=1_000_000, allow_inf_nan=False) + min_soc_pct: float | None = Field(default=None, ge=0, le=100, allow_inf_nan=False) + grid_charge_w: float | None = Field(default=None, ge=0, le=1_000_000, allow_inf_nan=False) + + @model_validator(mode="after") + def _not_empty(self) -> Self: + if ( + self.max_charge_w is None + and self.max_discharge_w is None + and self.min_soc_pct is None + and self.grid_charge_w is None + ): + raise invalid("battery_action_empty", "battery action: at least one limit must be set") + return self + + Action = Annotated[ - SwitchAction | PowerSetpointAction | SetpointAction | StepAction | ModeAction | ReleaseAction, + SwitchAction + | PowerSetpointAction + | SetpointAction + | StepAction + | ModeAction + | ReleaseAction + | BatteryAction, Field(discriminator="kind"), ] """Typed action target; the ``kind`` field selects the variant.""" @@ -263,6 +302,7 @@ def decision_from_json(text: str) -> Decision: "REASON_KEY_PATTERN", "Action", "ActionKind", + "BatteryAction", "Decision", "DecisionKind", "Intent", diff --git a/suncore/core/devices.py b/suncore/core/devices.py index fc01915..5d86ea8 100644 --- a/suncore/core/devices.py +++ b/suncore/core/devices.py @@ -399,6 +399,11 @@ class DeviceConfig(_Frozen): switch_on_hvac_mode: Literal["heat", "cool", "heat_cool", "auto"] | None = None """Climate: the HVAC mode used for switching on; required with ``allow_switch_on``.""" override_policy: OverridePolicy = OverridePolicy() + planner_value_per_kwh: Annotated[float, Field(ge=0, le=100, allow_inf_nan=False)] | None = None + """The device's utility for the planner, currency/kWh (Phase 4). ``None`` = automatic: + just above the export price (self-consumption beats export) and never above the import + price (SunCore never imports for a device). An explicit value is capped the same way + by the planner: the import price stays the ceiling.""" @model_validator(mode="after") def _consistent(self) -> Self: # noqa: PLR0912 diff --git a/suncore/core/forecast.py b/suncore/core/forecast.py new file mode 100644 index 0000000..089375a --- /dev/null +++ b/suncore/core/forecast.py @@ -0,0 +1,97 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Forecast and price source contracts (Phase 4 WP2). + +Adapters in ``suncore/forecast`` and ``suncore/tariff`` implement these protocols; the +planner only sees the resulting series. A source returns what it has for the requested +window - it may be shorter (``None`` values or a shorter series), never padded with +invented data. +""" + +from __future__ import annotations + +from datetime import timedelta +from enum import StrEnum, unique +from typing import Annotated, Final, Protocol, runtime_checkable + +from pydantic import BaseModel, ConfigDict, Field + +from suncore.core.series import PriceSeries, QuantileSeries, TimeRange + +SourceId = Annotated[str, Field(min_length=1, max_length=64, pattern=r"^[a-z0-9_.:-]+$")] +"""Stable id of a forecast or price source, usually its template id.""" + + +@runtime_checkable +class ForecastSource(Protocol): + """A PV (or load) forecast source; values in W (mean power per interval).""" + + @property + def source_id(self) -> str: ... + + async def fetch(self, window: TimeRange) -> QuantileSeries: + """The forecast for ``window`` (P10/P50/P90; equal quantiles if the source has none).""" + ... + + +@runtime_checkable +class PriceSource(Protocol): + """An import (and optionally export) price source; currency/kWh per interval.""" + + @property + def source_id(self) -> str: ... + + async def fetch(self, window: TimeRange) -> PriceSeries: + """Prices for ``window`` as far as the source publishes them.""" + ... + + +@unique +class HorizonBucket(StrEnum): + """Lead-time bucket the accuracy of a forecast is tracked in.""" + + H0_6 = "0_6h" + H6_24 = "6_24h" + H24_48 = "24_48h" + H48_PLUS = "48h_plus" + + @classmethod + def of(cls, lead: timedelta) -> HorizonBucket: + """Bucket of a lead time (issue time to interval start); negative counts as 0.""" + hours = lead / timedelta(hours=1) + if hours < 6: + return cls.H0_6 + if hours < 24: + return cls.H6_24 + if hours < 48: + return cls.H24_48 + return cls.H48_PLUS + + +class AccuracyRecord(BaseModel): + """Measured error of one source in one horizon bucket (WP5 bookkeeping). + + ``mae_w`` is the mean absolute error and ``bias_w`` the mean signed error + (forecast - actual, so + = over-forecast) of P50 over ``n`` intervals. + """ + + model_config = ConfigDict(frozen=True, extra="forbid") + + source_id: SourceId + horizon: HorizonBucket + mae_w: float = Field(ge=0, allow_inf_nan=False) + bias_w: float = Field(allow_inf_nan=False) + n: int = Field(ge=0) + + +ACCURACY_MIN_SAMPLES: Final = 96 +"""Intervals (one day of quarter hours) before a record counts for blending weights.""" + + +__all__ = [ + "ACCURACY_MIN_SAMPLES", + "AccuracyRecord", + "ForecastSource", + "HorizonBucket", + "PriceSource", + "SourceId", +] diff --git a/suncore/core/series.py b/suncore/core/series.py new file mode 100644 index 0000000..a0624dc --- /dev/null +++ b/suncore/core/series.py @@ -0,0 +1,505 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Time series contracts and resampling onto the planning grid (Phase 4 WP2). + +A :class:`TimeSeries` is a regular grid: ``values[i]`` belongs to the half-open interval +``[start + i*step, start + (i+1)*step)``. ``None`` marks a missing value; it is never +replaced by zero. All arithmetic is done in UTC so a DST day (23 h or 25 h local) needs no +special case: the planning grid is a run of 15-minute intervals in absolute time. + +Meaning of a value by unit (``core/units.py`` base units, signs as there): + +* ``W``: mean power over the interval (Solcast-style "average power over the slot"). +* ``Wh``: energy *in* the interval. +* ``currency/kWh``: price that applies to every instant of the interval. +* ``%`` and ``°C``: state averaged over the interval. + +Resampling (all pure functions): + +* power and state: time-weighted mean over the overlap (energy-preserving for power); +* energy: each source value is split by the share of its interval that overlaps; +* prices: step-hold (a coarse price holds for every finer interval inside it; several + finer prices inside one target interval are averaged by time). + +A target interval gets ``None`` unless known source values cover it completely. Input may +consist of several series with different steps (e.g. 15-minute prices today, hourly prices +tomorrow); they must not overlap. +""" + +from __future__ import annotations + +import itertools +from collections.abc import Iterable, Iterator, Sequence +from datetime import UTC, datetime, timedelta +from enum import StrEnum, unique +from typing import Annotated, Final, Literal, Self + +from pydantic import AwareDatetime, BaseModel, ConfigDict, Field, field_validator, model_validator + +from suncore.core.validation import invalid + +PLANNING_STEP: Final = timedelta(minutes=15) +"""Step of the planning grid (BRIEF §4.8).""" + +QUANTILE_TOLERANCE: Final = 1e-6 +"""Rounding slack allowed when checking p10 <= p50 <= p90.""" + +MAX_POINTS: Final = 20_000 +"""Upper bound of values per series (two weeks of minutes); protects the Pi's memory.""" + +FiniteFloat = Annotated[float, Field(allow_inf_nan=False)] + +PriceIncludes = Literal["spot_only", "all_in"] +"""Whether a price is the bare spot price or already the all-in end price (ADR-0012).""" + + +@unique +class SeriesUnit(StrEnum): + """Unit of a series; always a base unit (adapters convert kW, kWh at the edge).""" + + W = "W" + WH = "Wh" + PERCENT = "%" + CELSIUS = "°C" + PRICE = "currency/kWh" + """Region currency per kWh (``RegionConfig.currency``).""" + + +class _Frozen(BaseModel): + model_config = ConfigDict(frozen=True, extra="forbid") + + +def _utc(value: datetime) -> datetime: + return value.astimezone(UTC) + + +class TimeRange(_Frozen): + """Half-open absolute time range ``[start, end)``, e.g. the window a source is asked for.""" + + start: AwareDatetime + end: AwareDatetime + + @model_validator(mode="after") + def _ordered(self) -> Self: + if not self.start < self.end: + raise invalid("range_order", "time range: start must be before end") + return self + + @property + def duration(self) -> timedelta: + return self.end - self.start + + +class TimeSeries(_Frozen): + """A regular series: ``values[i]`` covers ``[start + i*step, start + (i+1)*step)``.""" + + start: AwareDatetime + """Normalised to UTC.""" + step: timedelta + values: Annotated[tuple[FiniteFloat | None, ...], Field(max_length=MAX_POINTS)] + unit: SeriesUnit + + @field_validator("start") + @classmethod + def _to_utc(cls, value: datetime) -> datetime: + return _utc(value) + + @model_validator(mode="after") + def _step(self) -> Self: + if self.step <= timedelta(0) or self.step > timedelta(days=1): + raise invalid("series_step", "series: step must be between 0 and one day") + return self + + @property + def end(self) -> datetime: + return self.start + self.step * len(self.values) + + def __len__(self) -> int: + return len(self.values) + + def slot_start(self, index: int) -> datetime: + return self.start + self.step * index + + def slots(self) -> Iterator[tuple[datetime, datetime, float | None]]: + """``(start, end, value)`` of every interval, in order.""" + for i, value in enumerate(self.values): + begin = self.slot_start(i) + yield begin, begin + self.step, value + + def same_grid(self, other: TimeSeries) -> bool: + return (self.start, self.step, len(self.values)) == ( + other.start, + other.step, + len(other.values), + ) + + def total(self) -> float | None: + """Energy (Wh) of a ``W`` or ``Wh`` series; ``None`` if a value is missing.""" + if self.unit not in (SeriesUnit.W, SeriesUnit.WH): + msg = f"total() needs a power or energy series, not {self.unit.value}" + raise ValueError(msg) + if any(v is None for v in self.values): + return None + hours = self.step / timedelta(hours=1) + factor = hours if self.unit is SeriesUnit.W else 1.0 + return sum(v * factor for v in self.values if v is not None) + + +class QuantileSeries(_Frozen): + """P10/P50/P90 of one quantity on one grid (PV or load forecast). + + Where all three values of an interval are known, ``p10 <= p50 <= p90`` holds. A source + without quantiles sets all three to the same series. + """ + + p10: TimeSeries + p50: TimeSeries + p90: TimeSeries + + @model_validator(mode="after") + def _consistent(self) -> Self: + if not (self.p50.same_grid(self.p10) and self.p50.same_grid(self.p90)): + raise invalid("quantile_grid", "quantiles: p10, p50 and p90 need the same grid") + if not self.p10.unit is self.p50.unit is self.p90.unit: + raise invalid("quantile_unit", "quantiles: p10, p50 and p90 need the same unit") + for lo, mid, hi in zip(self.p10.values, self.p50.values, self.p90.values, strict=True): + if lo is None or mid is None or hi is None: + continue + if lo > mid + QUANTILE_TOLERANCE or mid > hi + QUANTILE_TOLERANCE: + raise invalid("quantile_order", "quantiles: p10 <= p50 <= p90 is violated") + return self + + @classmethod + def deterministic(cls, series: TimeSeries) -> QuantileSeries: + """A forecast without spread (all quantiles equal).""" + return cls(p10=series, p50=series, p90=series) + + @property + def start(self) -> datetime: + return self.p50.start + + @property + def step(self) -> timedelta: + return self.p50.step + + @property + def unit(self) -> SeriesUnit: + return self.p50.unit + + def __len__(self) -> int: + return len(self.p50) + + +class PriceSeries(_Frozen): + """Import (and optionally export) prices per interval from one source. + + ``import_price`` and ``export_price`` are in currency/kWh and may be negative. + ``price_includes`` says whether the import price still needs the price formula + (``spot_only``) or is the end price (``all_in``). ``export_price = None`` means the + export price comes from the tariff configuration, not from this source. + """ + + source_id: Annotated[str, Field(min_length=1, max_length=64)] + price_includes: PriceIncludes + import_price: TimeSeries + export_price: TimeSeries | None = None + + @model_validator(mode="after") + def _consistent(self) -> Self: + series = ( + [self.import_price] + if self.export_price is None + else [ + self.import_price, + self.export_price, + ] + ) + if any(s.unit is not SeriesUnit.PRICE for s in series): + raise invalid("price_unit", "prices: unit must be currency/kWh") + if self.export_price is not None and not self.import_price.same_grid(self.export_price): + raise invalid("price_grid", "prices: import and export need the same grid") + return self + + @property + def start(self) -> datetime: + return self.import_price.start + + @property + def step(self) -> timedelta: + return self.import_price.step + + @property + def resolution_min(self) -> float: + """Interval length in minutes (15 or 60 for the sources SunCore knows).""" + return self.step / timedelta(minutes=1) + + def __len__(self) -> int: + return len(self.import_price) + + +# --- planning grid ------------------------------------------------------------------- + + +class Grid(_Frozen): + """The target grid of a resampling: ``count`` intervals of ``step`` from ``start``.""" + + start: AwareDatetime + step: timedelta = PLANNING_STEP + count: Annotated[int, Field(ge=1, le=MAX_POINTS)] + + @field_validator("start") + @classmethod + def _to_utc(cls, value: datetime) -> datetime: + return _utc(value) + + @model_validator(mode="after") + def _step(self) -> Self: + if self.step <= timedelta(0) or self.step > timedelta(days=1): + raise invalid("series_step", "series: step must be between 0 and one day") + return self + + @property + def end(self) -> datetime: + return self.start + self.step * self.count + + def starts(self) -> list[datetime]: + return [self.start + self.step * i for i in range(self.count)] + + +def floor_to_step(at: datetime, step: timedelta = PLANNING_STEP) -> datetime: + """``at`` rounded down to a multiple of ``step`` since the UNIX epoch (in UTC). + + For 15 minutes this is also the local quarter hour in every zone whose offset is a + multiple of 15 minutes (all current IANA zones). + """ + utc = _utc(at) + epoch = datetime(1970, 1, 1, tzinfo=UTC) + return epoch + step * ((utc - epoch) // step) + + +def planning_grid(now: datetime, horizon: timedelta, step: timedelta = PLANNING_STEP) -> Grid: + """The grid that starts at the interval containing ``now`` and covers ``horizon``.""" + start = floor_to_step(now, step) + end = _utc(now) + horizon + count = max(1, -(-(end - start) // step)) + return Grid(start=start, step=step, count=count) + + +# --- resampling ---------------------------------------------------------------------- + +Slot = tuple[datetime, datetime, float | None] + + +def _segments(series: TimeSeries | Sequence[TimeSeries]) -> list[TimeSeries]: + items = [series] if isinstance(series, TimeSeries) else list(series) + if not items: + return [] + units = {s.unit for s in items} + if len(units) != 1: + msg = "resample: all segments need the same unit" + raise ValueError(msg) + items.sort(key=lambda s: s.start) + for a, b in itertools.pairwise(items): + if b.start < a.end: + msg = "resample: segments overlap" + raise ValueError(msg) + return items + + +def _slots(items: Iterable[TimeSeries]) -> list[Slot]: + return [slot for s in items for slot in s.slots()] + + +def _seconds(delta: timedelta) -> float: + return delta.total_seconds() + + +def _resample(slots: list[Slot], grid: Grid, *, split: bool) -> tuple[float | None, ...]: + """Core of all resamplers. + + ``split`` = energy semantics (source value shared by overlap share, then summed); + otherwise intensive semantics (time-weighted mean over the overlap). + """ + out: list[float | None] = [] + j = 0 + for target_start in grid.starts(): + target_end = target_start + grid.step + while j < len(slots) and slots[j][1] <= target_start: + j += 1 + covered = 0.0 + acc = 0.0 + known = True + single: float | None = None + parts = 0 + k = j + while k < len(slots) and slots[k][0] < target_end: + s_start, s_end, value = slots[k] + overlap = _seconds(min(s_end, target_end) - max(s_start, target_start)) + if overlap > 0: + if value is None: + known = False + break + covered += overlap + parts += 1 + single = value + if split: + acc += value * overlap / _seconds(s_end - s_start) + else: + acc += value * overlap + k += 1 + full = _seconds(grid.step) + if not known or abs(covered - full) > 1e-6: + out.append(None) + elif split: + out.append(acc) + else: # one source value holds the whole interval: keep it exactly (step-hold) + out.append(single if parts == 1 else acc / full) + return tuple(out) + + +def resample_power(series: TimeSeries | Sequence[TimeSeries], grid: Grid) -> TimeSeries: + """Mean power (W) per grid interval; energy over fully covered intervals is preserved. + + Also used for ``%`` and ``°C`` series (time-weighted mean of a state). + """ + items = _segments(series) + unit = items[0].unit if items else SeriesUnit.W + if unit in (SeriesUnit.WH, SeriesUnit.PRICE): + msg = f"resample_power: unit {unit.value} is not intensive, use the matching resampler" + raise ValueError(msg) + values = _resample(_slots(items), grid, split=False) + return TimeSeries(start=grid.start, step=grid.step, values=values, unit=unit) + + +def resample_energy(series: TimeSeries | Sequence[TimeSeries], grid: Grid) -> TimeSeries: + """Energy (Wh) per grid interval; each source value is split by its overlap share.""" + items = _segments(series) + if items and items[0].unit is not SeriesUnit.WH: + msg = "resample_energy: needs a Wh series" + raise ValueError(msg) + values = _resample(_slots(items), grid, split=True) + return TimeSeries(start=grid.start, step=grid.step, values=values, unit=SeriesUnit.WH) + + +def energy_to_power(series: TimeSeries) -> TimeSeries: + """A Wh-per-interval series as mean power (W).""" + if series.unit is not SeriesUnit.WH: + msg = "energy_to_power: needs a Wh series" + raise ValueError(msg) + hours = series.step / timedelta(hours=1) + values = tuple(None if v is None else v / hours for v in series.values) + return series.model_copy(update={"values": values, "unit": SeriesUnit.W}) + + +def resample_quantiles( + series: QuantileSeries | Sequence[QuantileSeries], grid: Grid +) -> QuantileSeries: + """Resample P10/P50/P90 alike; the order of the quantiles is kept (same weights).""" + items = [series] if isinstance(series, QuantileSeries) else list(series) + unit = items[0].unit if items else SeriesUnit.W + resample = resample_energy if unit is SeriesUnit.WH else resample_power + return QuantileSeries( + p10=resample([q.p10 for q in items], grid), + p50=resample([q.p50 for q in items], grid), + p90=resample([q.p90 for q in items], grid), + ) + + +def _hold(series: Sequence[TimeSeries], grid: Grid) -> TimeSeries: + items = _segments(series) + if items and items[0].unit is not SeriesUnit.PRICE: + msg = "resample_prices: needs a currency/kWh series" + raise ValueError(msg) + values = _resample(_slots(items), grid, split=False) + return TimeSeries(start=grid.start, step=grid.step, values=values, unit=SeriesUnit.PRICE) + + +def resample_prices(series: PriceSeries | Sequence[PriceSeries], grid: Grid) -> PriceSeries: + """Step-hold prices onto ``grid``; segments of one source may mix 15 and 60 minutes. + + All segments must come from the same source with the same ``price_includes``; the + export series is kept only if every segment has one. + """ + items = [series] if isinstance(series, PriceSeries) else list(series) + if not items: + msg = "resample_prices: no input" + raise ValueError(msg) + if len({(p.source_id, p.price_includes) for p in items}) != 1: + msg = "resample_prices: segments from different sources" + raise ValueError(msg) + exports = [p.export_price for p in items] + export = ( + None + if any(e is None for e in exports) + else _hold([e for e in exports if e is not None], grid) + ) + return PriceSeries( + source_id=items[0].source_id, + price_includes=items[0].price_includes, + import_price=_hold([p.import_price for p in items], grid), + export_price=export, + ) + + +def segments_from_points( + points: Sequence[tuple[datetime, float | None]], + unit: SeriesUnit, + *, + last_step: timedelta | None = None, +) -> tuple[TimeSeries, ...]: + """Regular segments from timestamped points (e.g. an integration's price list). + + A new segment starts wherever the distance between two points changes (15 -> 60 + minutes) - this is how mixed resolutions enter the resamplers. The last point lasts as + long as the distance before it, or ``last_step`` for a single point. Points must be + strictly increasing in time; duplicates must be removed by the adapter. + """ + if not points: + return () + stamps = [_utc(t) for t, _ in points] + if any(b <= a for a, b in itertools.pairwise(stamps)): + msg = "segments_from_points: timestamps must be strictly increasing" + raise ValueError(msg) + steps = [b - a for a, b in itertools.pairwise(stamps)] + if not steps: + if last_step is None: + msg = "segments_from_points: a single point needs last_step" + raise ValueError(msg) + steps = [last_step] + else: + steps.append(last_step if last_step is not None else steps[-1]) + segments: list[TimeSeries] = [] + begin = 0 + for i in range(1, len(points) + 1): + if i == len(points) or steps[i] != steps[begin]: + segments.append( + TimeSeries( + start=stamps[begin], + step=steps[begin], + values=tuple(v for _, v in points[begin:i]), + unit=unit, + ) + ) + begin = i + return tuple(segments) + + +__all__ = [ + "MAX_POINTS", + "PLANNING_STEP", + "QUANTILE_TOLERANCE", + "Grid", + "PriceIncludes", + "PriceSeries", + "QuantileSeries", + "SeriesUnit", + "TimeRange", + "TimeSeries", + "energy_to_power", + "floor_to_step", + "planning_grid", + "resample_energy", + "resample_power", + "resample_prices", + "resample_quantiles", + "segments_from_points", +] diff --git a/suncore/core/tariff.py b/suncore/core/tariff.py new file mode 100644 index 0000000..472901b --- /dev/null +++ b/suncore/core/tariff.py @@ -0,0 +1,262 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Tariff mathematics (Phase 4 WP2): weekly time windows, price formula, HT/NT, export tiers. + +Pure and frozen. Prices are currency/kWh in the region's currency; spot prices may be +negative. Time windows are *local* wall-clock windows; callers pass the region's time zone, +the conversion happens here (``datetime.astimezone``), so DST days need no special case. +""" + +from __future__ import annotations + +import itertools +from collections.abc import Sequence +from datetime import datetime, time, timedelta, tzinfo +from typing import Annotated, Final, Protocol, Self + +from pydantic import BaseModel, ConfigDict, Field, StringConstraints, model_validator + +from suncore.core.forecast import PriceSource +from suncore.core.validation import invalid + +HHMM_PATTERN: Final = r"^([01]\d|2[0-3]):[0-5]\d$" +"""Local wall-clock time ``HH:MM`` (24 h).""" + +ALL_DAYS: Final = frozenset(range(7)) + +Hhmm = Annotated[str, StringConstraints(pattern=HHMM_PATTERN)] +Weekday = Annotated[int, Field(ge=0, le=6)] +"""0 = Monday ... 6 = Sunday (``datetime.weekday()``).""" +Month = Annotated[int, Field(ge=1, le=12)] +UnitPrice = Annotated[float, Field(ge=0, le=100, allow_inf_nan=False)] +"""Non-negative currency/kWh component (fees, levies, fixed prices).""" + + +class _Frozen(BaseModel): + model_config = ConfigDict(frozen=True, extra="forbid") + + +def _minutes(hhmm: str) -> int: + hours, minutes = hhmm.split(":") + return int(hours) * 60 + int(minutes) + + +def _local(at: datetime, tz: tzinfo | None) -> datetime: + if at.tzinfo is None: + msg = "time windows need an aware datetime" + raise ValueError(msg) + return at.astimezone(tz) if tz is not None else at + + +# --- weekly windows -------------------------------------------------------------------- + + +class TouWindow(_Frozen): + """A local time window on a set of weekdays, e.g. NT Mon-Fri 22:00-06:00. + + * ``end`` before ``start`` crosses midnight; the part after midnight belongs to the + weekday the window *started* on (Fri 22:00-06:00 covers Saturday 00:00-06:00, not + Friday 00:00-06:00). + * ``start == end`` is the whole day (00:00-00:00 = 24 h). + * ``start`` is included, ``end`` is excluded. + """ + + days: frozenset[Weekday] = ALL_DAYS + start: Hhmm + end: Hhmm + + @model_validator(mode="after") + def _days(self) -> Self: + if not self.days: + raise invalid("window_no_days", "window: at least one weekday is required") + return self + + @property + def crosses_midnight(self) -> bool: + return _minutes(self.end) < _minutes(self.start) + + def contains(self, at: datetime, tz: tzinfo | None = None) -> bool: + """Whether ``at`` (aware) lies in the window, judged in ``tz`` (default: ``at``'s).""" + local = _local(at, tz) + minute = local.hour * 60 + local.minute + start, end = _minutes(self.start), _minutes(self.end) + today = local.weekday() + yesterday = (today - 1) % 7 + if start == end: + return today in self.days + if start < end: + return today in self.days and start <= minute < end + return (today in self.days and minute >= start) or (yesterday in self.days and minute < end) + + +class WeeklySchedule(_Frozen): + """Union of weekly windows (grid-charge schedule, HT/NT windows, §14a module 3).""" + + windows: Annotated[tuple[TouWindow, ...], Field(min_length=1, max_length=28)] + + def contains(self, at: datetime, tz: tzinfo | None = None) -> bool: + return any(w.contains(at, tz) for w in self.windows) + + @classmethod + def daily(cls, start: str, end: str) -> WeeklySchedule: + """One window every day, e.g. ``daily("22:00", "06:00")``.""" + return cls(windows=(TouWindow(start=start, end=end),)) + + +NIGHT_WINDOW: Final = WeeklySchedule.daily("22:00", "06:00") +"""Plan §6 question 7 default night window, the target of the ``goals`` v1 migration.""" + + +# --- price formula --------------------------------------------------------------------- + + +class GridFeeWindow(_Frozen): + """A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3).""" + + window: TouWindow + fee: UnitPrice + + +class PriceFormula(_Frozen): + """Spot price -> end price (BRIEF §4.4). + + ``end = (spot * spot_multiplier + grid_fee(at) + levies + margin) * (1 + vat_pct/100)`` + + ``grid_fee(at)`` is the fee of the first ``grid_fee_windows`` entry containing ``at``, + else ``grid_fee``. VAT is applied to the whole sum, as on a German bill; a negative + spot price can make the end price negative. Only for ``price_includes = spot_only`` + sources - an all-in price never gets the formula on top. + """ + + spot_multiplier: float = Field(default=1.0, gt=0, le=10, allow_inf_nan=False) + """E.g. 1.0; some contracts pass the spot price on with a factor.""" + grid_fee: UnitPrice = 0.0 + grid_fee_windows: Annotated[tuple[GridFeeWindow, ...], Field(max_length=12)] = () + levies: UnitPrice = 0.0 + """Taxes and levies per kWh (e.g. electricity tax, surcharges), before VAT.""" + margin: UnitPrice = 0.0 + """Supplier margin per kWh, before VAT.""" + vat_pct: float = Field(default=0.0, ge=0, le=100, allow_inf_nan=False) + + def grid_fee_at(self, at: datetime | None, tz: tzinfo | None = None) -> float: + if at is not None: + for entry in self.grid_fee_windows: + if entry.window.contains(at, tz): + return entry.fee + return self.grid_fee + + def apply(self, spot: float, at: datetime | None = None, tz: tzinfo | None = None) -> float: + """End price for ``spot`` at ``at`` (``None``: the base grid fee is used).""" + net = spot * self.spot_multiplier + self.grid_fee_at(at, tz) + self.levies + self.margin + return net * (1 + self.vat_pct / 100) + + +# --- HT/NT (time of use) --------------------------------------------------------------- + + +class TouPeriod(_Frozen): + """A price that applies inside ``window``, optionally only in some ``months``.""" + + window: TouWindow + price: UnitPrice + months: frozenset[Month] | None = None + """``None`` = all year (seasonal tariffs list the months).""" + + def applies(self, at: datetime, tz: tzinfo | None = None) -> bool: + local = _local(at, tz) + if self.months is not None and local.month not in self.months: + return False + return self.window.contains(local) + + +class TouTariff(_Frozen): + """Fixed time-of-use import tariff (HT/NT, seasonal). + + The first period in list order that applies wins; outside every period ``base_price`` + applies. Overlaps are therefore allowed and resolved by order (put the special case + first). + """ + + base_price: UnitPrice + periods: Annotated[tuple[TouPeriod, ...], Field(min_length=1, max_length=24)] + + def price_at(self, at: datetime, tz: tzinfo | None = None) -> float: + for period in self.periods: + if period.applies(at, tz): + return period.price + return self.base_price + + +# --- export tiers ---------------------------------------------------------------------- + + +class PriceTier(Protocol): + """A feed-in band up to ``up_to_kwp`` (``None`` = unbounded) with its rate.""" + + @property + def up_to_kwp(self) -> float | None: ... + + @property + def price(self) -> float: ... + + +def blended_export_price(tiers: Sequence[PriceTier], total_kwp: float) -> float: + """Feed-in rate of a plant split into kWp bands (German EEG "anteilig" rule). + + A 17 kWp plant with ``<= 10 kWp: a``, ``rest: b`` earns ``(10 a + 7 b) / 17`` per kWh. + Tiers must be ordered with strictly increasing bounds and only the last one unbounded + (``ExportTariff`` validates that). Raises ``ValueError`` if ``total_kwp`` is not + positive or the tiers do not cover it - SunCore never guesses a rate. + """ + if not total_kwp > 0: + msg = "blended_export_price: total_kwp must be positive" + raise ValueError(msg) + if not tiers: + msg = "blended_export_price: no tiers" + raise ValueError(msg) + bounds = [t.up_to_kwp for t in tiers] + if None in bounds[:-1] or any( + a is not None and b is not None and a >= b for a, b in itertools.pairwise(bounds) + ): + msg = "blended_export_price: tiers must be ordered, only the last may be unbounded" + raise ValueError(msg) + revenue = 0.0 + lower = 0.0 + for tier in tiers: + upper = total_kwp if tier.up_to_kwp is None else min(tier.up_to_kwp, total_kwp) + if upper > lower: + revenue += (upper - lower) * tier.price + lower = upper + if lower >= total_kwp: + return revenue / total_kwp + msg = f"blended_export_price: tiers end at {lower} kWp below the plant's {total_kwp} kWp" + raise ValueError(msg) + + +def hhmm(value: time) -> str: + """``time`` -> ``HH:MM`` (seconds are dropped).""" + return f"{value.hour:02d}:{value.minute:02d}" + + +def window_duration(window: TouWindow) -> timedelta: + """Length of one occurrence of the window.""" + start, end = _minutes(window.start), _minutes(window.end) + minutes = (end - start) % (24 * 60) or 24 * 60 + return timedelta(minutes=minutes) + + +__all__ = [ + "ALL_DAYS", + "HHMM_PATTERN", + "NIGHT_WINDOW", + "GridFeeWindow", + "PriceFormula", + "PriceSource", + "PriceTier", + "TouPeriod", + "TouTariff", + "TouWindow", + "WeeklySchedule", + "blended_export_price", + "hhmm", + "window_duration", +] diff --git a/suncore/safety/gate.py b/suncore/safety/gate.py index e4c41a2..dc98c52 100644 --- a/suncore/safety/gate.py +++ b/suncore/safety/gate.py @@ -55,6 +55,7 @@ from suncore.core.control import ControlSettings, DeviceState, GateInputs from suncore.core.decisions import ( Action, + BatteryAction, Decision, DecisionKind, Intent, @@ -137,6 +138,8 @@ """The device SunCore controlled was removed or re-mapped and could not be released.""" REASON_ORPHAN_DROPPED: Final = Reason(key="safety.orphan_dropped") """An orphaned runtime without the old actuator: reported, then dropped (manual check needed).""" +REASON_BATTERY_NOT_SUPPORTED: Final = Reason(key="safety.battery_not_supported") +"""A battery action before the battery path exists (Phase 4 WP9): rejected, never written.""" MAX_CONSECUTIVE_FAILURES: Final = 3 """After this many failed writes in a row the device scope trips (watchdog).""" @@ -515,7 +518,7 @@ def dismiss(self, intent_id: str) -> bool: # --- the gate ------------------------------------------------------------------------ - async def submit(self, intent: Intent) -> Decision: # noqa: PLR0911 - one return per verdict + async def submit(self, intent: Intent) -> Decision: # noqa: PLR0911, PLR0912 - one return per verdict """Judge one intent, emit the decision to the sink and return it.""" now = self._now() self._prune_proposals(now) @@ -529,6 +532,11 @@ async def submit(self, intent: Intent) -> Decision: # noqa: PLR0911 - one retur return await self._decide( intent, DecisionKind.REJECTED, now, (REASON_EMERGENCY_STOP,), mode ) + if isinstance(intent.action, BatteryAction): + # Explicitly unsupported until WP9 (control-safety): no invariants exist yet. + return await self._decide( + intent, DecisionKind.REJECTED, now, (REASON_BATTERY_NOT_SUPPORTED,), mode + ) current = await self._runtime(device_id) runtime = overrides.expire(current, now) if runtime is not current: # only the expiry is saved, never a stale copy diff --git a/suncore/safety/invariants.py b/suncore/safety/invariants.py index 770e94b..353db9a 100644 --- a/suncore/safety/invariants.py +++ b/suncore/safety/invariants.py @@ -24,6 +24,7 @@ from suncore.core.conversion import value_to_power_w from suncore.core.decisions import ( Action, + BatteryAction, ModeAction, PowerSetpointAction, Reason, @@ -107,7 +108,7 @@ def action_turns_on(action: Action, device: DeviceConfig) -> bool | None: # noq return power > 0 case SetpointAction(): return True - case ReleaseAction(): + case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP9) return None return None # pragma: no cover - exhaustive @@ -149,7 +150,7 @@ def expected_power_w( # noqa: PLR0911 - one branch per action kind return max(0.0, power) case SetpointAction(): return nominal if nominal is not None else _upper(estimate) - case ReleaseAction(): + case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP9) return None return None # pragma: no cover - exhaustive @@ -430,7 +431,7 @@ def action_matches_state( # noqa: PLR0911 if device is not None and isinstance(device.control, ContinuousControl): current = _continuous_power_w(current, device.control, device.power_model.nominal_w) return abs(current - power) <= POWER_TOLERANCE_W - case ReleaseAction(): + case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP9) return False return False # pragma: no cover - exhaustive diff --git a/suncore/sim/demo_config.py b/suncore/sim/demo_config.py index b10c6ab..6d5b3f2 100644 --- a/suncore/sim/demo_config.py +++ b/suncore/sim/demo_config.py @@ -12,11 +12,13 @@ from typing import Final +from suncore.core.battery import BatterySettings from suncore.core.config import ( BatteryTarget, DevicesConfig, ExportTariff, ForecastConfig, + ForecastSourceConfig, GoalsConfig, ImportTariff, OperationConfig, @@ -172,11 +174,15 @@ async def seed_demo_config(config: ConfigStore, catalog: TemplateCatalog) -> Non await config.put( "forecast", ForecastConfig( - forecast_template_ids=["solcast.pv"], - forecast_entities={ - "forecast_today": EntityRef(entity_id=FORECAST_TODAY_ID), - "forecast_tomorrow": EntityRef(entity_id=FORECAST_TOMORROW_ID), - }, + sources=[ + ForecastSourceConfig( + template_id="solcast.pv", + entities={ + "forecast_today": EntityRef(entity_id=FORECAST_TODAY_ID), + "forecast_tomorrow": EntityRef(entity_id=FORECAST_TOMORROW_ID), + }, + ) + ], ), ) await config.put("devices", demo_devices(catalog)) @@ -188,4 +194,6 @@ async def seed_demo_config(config: ConfigStore, catalog: TemplateCatalog) -> Non ), ) await config.put("operation", OperationConfig(mode=OperatingMode.SHADOW)) + # Observe only: the demo battery is read, never written; the model follows with WP8. + await config.put("battery", BatterySettings()) await config.put("wizard", WizardState(completed=True, completed_steps=list(WIZARD_STEPS))) diff --git a/suncore/storage/config_store.py b/suncore/storage/config_store.py index 3acc3c8..d9190d9 100644 --- a/suncore/storage/config_store.py +++ b/suncore/storage/config_store.py @@ -12,6 +12,7 @@ from pydantic import BaseModel, ValidationError from suncore import __version__ +from suncore.core.battery import BatterySettings from suncore.core.config import ( DevicesConfig, ForecastConfig, @@ -20,6 +21,9 @@ RegionConfig, TariffConfig, WizardState, + migrate_forecast_v1_to_v2, + migrate_goals_v1_to_v2, + migrate_tariff_v2_to_v3, ) from suncore.core.decisions import OperatingMode from suncore.core.devices import DeviceConfig @@ -83,7 +87,9 @@ def migrate_tariff_v1_to_v2(body: Body) -> Body: def legacy_forecast(body: Body) -> Body | None: - """The ``forecast`` section body hidden in a tariff v1 body; ``None`` if it had no sources.""" + """The ``forecast`` section body (schema v1) hidden in a tariff v1 body; ``None`` if it had + no sources. Upgrade it with the ``forecast`` migrations (``_upgrade("forecast", 1, ...)``). + """ ids = body.get("forecast_template_ids") or [] entities = body.get("forecast_entities") or {} if not ids and not entities: @@ -94,12 +100,13 @@ def legacy_forecast(body: Body) -> Body | None: DEFAULT_SECTIONS: Final[Mapping[str, SectionSpec]] = { "site": SectionSpec(SiteConfig, (migrate_site_v1_to_v2,)), "region": SectionSpec(RegionConfig), - "tariff": SectionSpec(TariffConfig, (migrate_tariff_v1_to_v2,)), - "forecast": SectionSpec(ForecastConfig), + "tariff": SectionSpec(TariffConfig, (migrate_tariff_v1_to_v2, migrate_tariff_v2_to_v3)), + "forecast": SectionSpec(ForecastConfig, (migrate_forecast_v1_to_v2,)), "devices": SectionSpec(DevicesConfig), - "goals": SectionSpec(GoalsConfig), + "goals": SectionSpec(GoalsConfig, (migrate_goals_v1_to_v2,)), "operation": SectionSpec(OperationConfig), "wizard": SectionSpec(WizardState), + "battery": SectionSpec(BatterySettings), } @@ -406,8 +413,8 @@ async def migrate_legacy_forecast(self) -> bool: now = self._clock().isoformat() if moved is not None and have is None: try: - forecast = ForecastConfig.model_validate(moved) - except ValidationError: + forecast: BaseModel | None = self._upgrade("forecast", 1, moved) + except ConfigCorruptError: forecast = None if forecast is not None: await self._write(tx, "forecast", forecast, now) diff --git a/suncore/templates/__init__.py b/suncore/templates/__init__.py index 7d24c7d..49cae6a 100644 --- a/suncore/templates/__init__.py +++ b/suncore/templates/__init__.py @@ -22,6 +22,7 @@ TEMPLATE_ADAPTER, ActionSeries, AttributeSeries, + BatteryTemplate, DeviceTemplate, EntityMatcher, ForecastTemplate, @@ -36,6 +37,7 @@ "TEMPLATE_ADAPTER", "ActionSeries", "AttributeSeries", + "BatteryTemplate", "BuiltinTemplateError", "DeviceTemplate", "EntityMatcher", diff --git a/suncore/templates/loader.py b/suncore/templates/loader.py index b62ee5c..63a66c2 100644 --- a/suncore/templates/loader.py +++ b/suncore/templates/loader.py @@ -22,6 +22,7 @@ TEMPLATE_ADAPTER, TEMPLATE_KINDS, VERSION_PATTERN, + BatteryTemplate, DeviceTemplate, ForecastTemplate, RegionTemplate, @@ -80,6 +81,9 @@ def forecasts(self) -> list[ForecastTemplate]: def regions(self) -> list[RegionTemplate]: return [t for t in self.of_kind("region") if isinstance(t, RegionTemplate)] + def batteries(self) -> list[BatteryTemplate]: + return [t for t in self.of_kind("battery") if isinstance(t, BatteryTemplate)] + def _format_validation(exc: ValidationError) -> str: parts = [] @@ -96,6 +100,7 @@ def _is_union_tag(part: str | int) -> bool: "TariffTemplate", "ForecastTemplate", "RegionTemplate", + "BatteryTemplate", } diff --git a/suncore/templates/models.py b/suncore/templates/models.py index 555c77f..66e6920 100644 --- a/suncore/templates/models.py +++ b/suncore/templates/models.py @@ -53,13 +53,22 @@ VERSION_PATTERN: Final = r"^\d+(\.\d+)*((a|b|rc)\d+)?(\.post\d+)?(\.dev\d+)?$" """PEP 440 subset accepted for ``min_app_version`` (no epochs, no local versions).""" -TemplateKind = Literal["device", "tariff", "forecast", "region"] -TEMPLATE_KINDS: Final[tuple[TemplateKind, ...]] = ("device", "tariff", "forecast", "region") +TemplateKind = Literal["device", "tariff", "forecast", "region", "battery"] +TEMPLATE_KINDS: Final[tuple[TemplateKind, ...]] = ( + "device", + "tariff", + "forecast", + "region", + "battery", +) Slug = Annotated[str, StringConstraints(pattern=r"^[a-z][a-z0-9_]*$", max_length=48)] MessageKey = Annotated[str, StringConstraints(pattern=REASON_KEY_PATTERN, max_length=120)] Text = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1, max_length=200)] Url = Annotated[str, StringConstraints(pattern=r"^https?://\S+$", max_length=500)] +DottedPath = Annotated[ + str, StringConstraints(pattern=r"^[A-Za-z0-9_]+(\.[A-Za-z0-9_]+)*$", max_length=200) +] RegexText = Annotated[str, StringConstraints(min_length=1, max_length=200)] @@ -216,12 +225,24 @@ class SeriesFields(_Frozen): class AttributeSeries(_Frozen): - """Series held in an attribute of the entities of ``roles`` (concatenated in order).""" + """Series held in an attribute of the entities of ``roles`` (concatenated in order). + + ``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's + value to the list of points when the attribute is a mapping (``None`` = the attribute + itself is the list). The period length of a point is the template's + ``resolution_min``. + """ source: Literal["attribute"] roles: Annotated[tuple[RoleName, ...], Field(min_length=1, max_length=16)] attribute: Text fields: SeriesFields + list_path: ( + Annotated[ + str, StringConstraints(pattern=r"^[A-Za-z0-9_]+(\.[A-Za-z0-9_]+)*$", max_length=200) + ] + | None + ) = None @model_validator(mode="after") def _unique(self) -> Self: @@ -544,14 +565,181 @@ def _region(self) -> Self: return self +# --- battery ----------------------------------------------------------------- + + +class NumberLimitControl(_Frozen): + """A ``number`` role that sets a battery limit. + + ``min_soc``: discharge floor in ``%`` (discharge lock / reserve); ``max_charge`` and + ``max_discharge``: power limits; ``grid_charge_power``: forced charging power from the + grid. ``min``/``max``/``step`` are the entity's range in ``unit`` (``%`` of the + battery's maximum power for the power kinds). + """ + + kind: Literal["min_soc", "max_charge", "max_discharge", "grid_charge_power"] + unit: Literal["W", "kW", "%"] + min: Annotated[float, Field(ge=0)] + max: Annotated[float, Field(gt=0)] + step: Annotated[float, Field(gt=0)] | None = None + + @model_validator(mode="after") + def _range(self) -> Self: + if not self.min < self.max: + msg = "controls: `min` must be below `max`" + raise ValueError(msg) + if self.unit == "%" and self.max > 100: + msg = "controls: a value in % allows at most 100" + raise ValueError(msg) + if self.kind == "min_soc" and self.unit != "%": + msg = "controls: `min_soc` is set in %" + raise ValueError(msg) + return self + + +@unique +class BatteryModeMeaning(StrEnum): + """What selecting a battery mode option does.""" + + AUTO = "auto" + """The inverter's own control (the release baseline).""" + FORCE_CHARGE = "force_charge" + """Charge, also from the grid.""" + NO_DISCHARGE = "no_discharge" + NO_CHARGE = "no_charge" + OTHER = "other" + """Any other option; SunCore never selects it.""" + + +class BatteryModeOption(_Frozen): + value: Text + """The select option written to the entity.""" + meaning: BatteryModeMeaning + + +class ModeSelectControl(_Frozen): + """A ``select`` role choosing the battery's operating mode.""" + + kind: Literal["mode"] + options: Annotated[tuple[BatteryModeOption, ...], Field(min_length=2, max_length=16)] + + @model_validator(mode="after") + def _options(self) -> Self: + values = [o.value for o in self.options] + if len(set(values)) != len(values): + msg = "controls: mode values must be unique" + raise ValueError(msg) + autos = [o for o in self.options if o.meaning is BatteryModeMeaning.AUTO] + if len(autos) != 1: + msg = "controls: a mode select needs exactly one option with meaning `auto`" + raise ValueError(msg) + return self + + +class ForceChargeSwitchControl(_Frozen): + """A ``switch`` role: on = force charging (from the grid if PV is short), off = auto.""" + + kind: Literal["force_charge"] + + +BatteryControl = Annotated[ + NumberLimitControl | ModeSelectControl | ForceChargeSwitchControl, + Field(discriminator="kind"), +] +"""What a writable battery role can express (``controls`` of a battery template).""" + +_CONTROL_DOMAINS: Final = { + "min_soc": "number", + "max_charge": "number", + "max_discharge": "number", + "grid_charge_power": "number", + "mode": "select", + "force_charge": "switch", +} + + +class BatteryTemplate(TemplateHeader): + """Battery adapter template (Phase 4, ADR-0006/ADR-0012 kind ``battery``). + + ``roles`` are the readable and writable entities (e.g. ``soc``, ``power``, + ``min_soc_number``, ``max_charge_number``, ``max_discharge_number``, ``mode_select``, + ``force_charge_switch``); ``controls`` says what each *writable* role expresses. A + role without a control is read only. Writes go through the safety gate only. + """ + + kind: Literal["battery"] + roles: dict[RoleName, RoleSpec] + controls: Annotated[dict[RoleName, BatteryControl], Field(max_length=8)] = Field( + default_factory=dict + ) + settle_s: Annotated[int, Field(ge=0, le=600)] | None = None + """Delay before the read-back after a write.""" + + @model_validator(mode="after") + def _battery(self) -> Self: + if "." not in self.id: + msg = "id: must be `.`" + raise ValueError(msg) + _check_roles(self.roles) + kinds: list[str] = [] + for role, control in self.controls.items(): + if role not in self.roles: + msg = f"controls.{role}: unknown role" + raise ValueError(msg) + domain = _CONTROL_DOMAINS[control.kind] + if self.roles[role].matcher.domain != domain: + msg = f"controls.{role}: kind `{control.kind}` needs a `{domain}` entity" + raise ValueError(msg) + kinds.append(control.kind) + if len(set(kinds)) != len(kinds): + msg = "controls: every control kind at most once" + raise ValueError(msg) + self._capabilities(set(kinds)) + return self + + def _capabilities(self, kinds: set[str]) -> None: + caps = self.capabilities + classes = {r.matcher.device_class for r in self.roles.values()} + modes = { + o.meaning + for c in self.controls.values() + if isinstance(c, ModeSelectControl) + for o in c.options + } + _need( + caps, Capability.BATTERY_SOC, "battery" in classes, "no role with device_class battery" + ) + _need( + caps, Capability.POWER_MEASURED, "power" in classes, "no role with device_class power" + ) + _need(caps, Capability.MIN_SOC_LIMIT, "min_soc" in kinds, "no `min_soc` control") + _need(caps, Capability.CHARGE_LIMIT, "max_charge" in kinds, "no `max_charge` control") + _need( + caps, Capability.DISCHARGE_LIMIT, "max_discharge" in kinds, "no `max_discharge` control" + ) + _need( + caps, + Capability.GRID_CHARGE, + bool(kinds & {"grid_charge_power", "force_charge"}) + or BatteryModeMeaning.FORCE_CHARGE in modes, + "no grid charge control", + ) + _need(caps, Capability.BATTERY_MODES, "mode" in kinds, "no `mode` control") + + def message_keys(self) -> list[str]: + return [self.name, *(r.description for r in self.roles.values())] + + Template = Annotated[ - DeviceTemplate | TariffTemplate | ForecastTemplate | RegionTemplate, + DeviceTemplate | TariffTemplate | ForecastTemplate | RegionTemplate | BatteryTemplate, Field(discriminator="kind"), ] """Discriminated union on ``kind``.""" TEMPLATE_ADAPTER: Final[ - TypeAdapter[DeviceTemplate | TariffTemplate | ForecastTemplate | RegionTemplate] + TypeAdapter[ + DeviceTemplate | TariffTemplate | ForecastTemplate | RegionTemplate | BatteryTemplate + ] ] = TypeAdapter(Template) KIND_MODELS: Final[dict[str, type[BaseModel]]] = { @@ -559,6 +747,7 @@ def _region(self) -> Self: "tariff": TariffTemplate, "forecast": ForecastTemplate, "region": RegionTemplate, + "battery": BatteryTemplate, } __all__ = [ @@ -571,12 +760,19 @@ def _region(self) -> Self: "VERSION_PATTERN", "ActionSeries", "AttributeSeries", + "BatteryControl", + "BatteryModeMeaning", + "BatteryModeOption", + "BatteryTemplate", "DeviceDefaults", "DeviceTemplate", "EntityMatcher", "FeedInTier", + "ForceChargeSwitchControl", "ForecastTemplate", + "ModeSelectControl", "NegativePriceRule", + "NumberLimitControl", "PowerModelDefault", "RegionTemplate", "RoleSpec", diff --git a/templates/README.md b/templates/README.md index e2deace..c4c4c28 100644 --- a/templates/README.md +++ b/templates/README.md @@ -2,9 +2,8 @@ Declarative device, tariff, forecast and region templates (ADR-0006, ADR-0012, schema v1). Layout: `templates//.yaml` with `kind` in `device`, `tariff`, `forecast`, -`region`; user templates in `/data/templates//` override built-in ones by `id`. -A `*.yaml` file in any other sub directory (for example `battery/` before that kind exists) -is reported as an error. The JSON Schemas in `schema/` are generated from the Pydantic +`region`, `battery`; user templates in `/data/templates//` override built-in ones by +`id`. A `*.yaml` file in any other sub directory is reported as an error. The JSON Schemas in `schema/` are generated from the Pydantic models in `suncore/templates/models.py` and `suncore/core/devices.py` – never edit them by hand (`uv run python scripts/gen_schemas.py`; CI fails on drift). @@ -60,7 +59,8 @@ unverified: [] # facts you could not verify; required if tes - `capabilities` is a closed list: `power_measured`, `energy_counter`, `on_off`, `power_setpoint`, `temperature_setpoint`, `hvac_modes`, `preset_modes`, `fan_modes`, `price_now`, `price_curve`, `pv_curve`, `pv_quantiles`, `feed_in_tiers`, - `negative_price_rule`. Where a validator can check the claim (for example + `negative_price_rule`, and for batteries `battery_soc`, `min_soc_limit`, `charge_limit`, + `discharge_limit`, `grid_charge`, `battery_modes`. Where a validator can check the claim (for example `pv_quantiles` needs `p10_field` and `p90_field`) an unfounded one is an error. - Devices write through the role(s) named by `defaults.control` (`on_off`, `steps`, `relays` (1–3 switches), `continuous` in W/A/%, `setpoint` with a maximum shift, `mode`). Mode of operation @@ -73,7 +73,17 @@ unverified: [] # facts you could not verify; required if tes `solcast_solar.query_forecast_data`, `weather.get_forecasts`) are accepted; any other service is refused at load time with the key `template.error.action_not_allowed`. Power units in a forecast mean the average power over the slot, energy units the energy - per slot. + per slot. When the attribute is a mapping, `list_path` (dotted) leads to the list of + points; the period of a point is `resolution_min`. +- Battery templates (`kind: battery`) list the readable and writable roles (for example + `soc`, `power`, `min_soc_number`, `max_charge_number`, `max_discharge_number`, + `mode_select`, `force_charge_switch`) and in `controls` what each writable role + expresses: `min_soc` (number in %), `max_charge` / `max_discharge` / + `grid_charge_power` (number in W, kW or %), `mode` (select; every option has a + `meaning`: `auto` - exactly one, the release baseline -, `force_charge`, + `no_discharge`, `no_charge`, `other`) or `force_charge` (switch). The control kind + fixes the entity domain (`number`, `select`, `switch`). Every write goes through the + safety gate; a battery starts in shadow mode like every device. - Do not invent attributes, services or registers. Cite the integration source or vendor document in `source`, otherwise list the fact under `unverified`. - Every template needs a simulator conformance scenario under @@ -116,3 +126,4 @@ Every addition is recorded below with the SunCore version that introduced it. | 0.3.0 (unreleased, Phase 2) | Initial schema v1 after the ADR-0012 review: matcher with `platforms`, `translation_key_pattern`, `unit_pattern`, digits in `domain`; `RoleSpec.same_device_as`; `PowerModelDefault` without required `nominal_w`, `counter_update_s`; `defaults.control` as `ControlSpec` (`on_off`, `steps`, `continuous`, `setpoint`, `mode`); `hysteresis_w`, `hysteresis_delay_s`, `settle_s`; `series` (`attribute` or `action`) for tariff and forecast, `price_includes`, optional `resolution_min` for tariffs; closed `capabilities` vocabulary; `min_app_version`; loader lint for unknown kind directories. | | 0.3.0 (unreleased, Phase 2) | `region.currency` is optional for `country: generic` (`None` = currency of the Home Assistant configuration). | | 0.4.0 (unreleased, Phase 3) | New `defaults.control` kind `relays` (additive union member, ADR-0012 policy): 1–3 switch roles (`roles`) and `steps`, each step naming the relays that are on (`relays`) and the power it draws (`power_w`, strictly increasing, relay sets distinct). `DeviceConfig` accepts the same control. Used by `generic.water_heater_relays`. | +| 0.5.0 (unreleased, Phase 4) | New template kind `battery` (`battery.schema.json`): `roles`, `controls` (union on `kind`: `min_soc`, `max_charge`, `max_discharge`, `grid_charge_power` numbers; `mode` select with option meanings; `force_charge` switch), optional `settle_s`. New capabilities `battery_soc`, `min_soc_limit`, `charge_limit`, `discharge_limit`, `grid_charge`, `battery_modes`. `series` of `source: attribute` takes an optional `list_path` for the generic attribute adapter. HT/NT needs no template change: time windows and prices are contract data in the `tariff` configuration section (v3). | diff --git a/templates/schema/battery.schema.json b/templates/schema/battery.schema.json new file mode 100644 index 0000000..b80c294 --- /dev/null +++ b/templates/schema/battery.schema.json @@ -0,0 +1,513 @@ +{ + "$defs": { + "BatteryModeMeaning": { + "description": "What selecting a battery mode option does.", + "enum": [ + "auto", + "force_charge", + "no_discharge", + "no_charge", + "other" + ], + "title": "BatteryModeMeaning", + "type": "string" + }, + "BatteryModeOption": { + "additionalProperties": false, + "properties": { + "meaning": { + "$ref": "#/$defs/BatteryModeMeaning" + }, + "value": { + "maxLength": 200, + "minLength": 1, + "title": "Value", + "type": "string" + } + }, + "required": [ + "value", + "meaning" + ], + "title": "BatteryModeOption", + "type": "object" + }, + "Capability": { + "description": "What a template provides.", + "enum": [ + "power_measured", + "energy_counter", + "on_off", + "power_setpoint", + "temperature_setpoint", + "hvac_modes", + "preset_modes", + "fan_modes", + "price_now", + "price_curve", + "pv_curve", + "pv_quantiles", + "feed_in_tiers", + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" + ], + "title": "Capability", + "type": "string" + }, + "EntityMatcher": { + "additionalProperties": false, + "description": "Role matcher; every given criterion must match, ``domain`` is mandatory.\n\n``units`` lists acceptable ``unit_of_measurement`` strings (exact, case-sensitive);\n``unit_pattern`` is a regular expression matched with ``re.fullmatch`` on the stripped\nunit. ``translation_key`` / ``translation_key_pattern`` likewise. Exact and pattern\nforms exclude each other.", + "properties": { + "device_class": { + "anyOf": [ + { + "maxLength": 64, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Device Class" + }, + "domain": { + "pattern": "^[a-z0-9_]+$", + "title": "Domain", + "type": "string" + }, + "platforms": { + "anyOf": [ + { + "items": { + "maxLength": 48, + "pattern": "^[a-z][a-z0-9_]*$", + "type": "string" + }, + "maxItems": 8, + "minItems": 1, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Platforms" + }, + "state_class": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "State Class" + }, + "translation_key": { + "anyOf": [ + { + "maxLength": 128, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Translation Key" + }, + "translation_key_pattern": { + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Translation Key Pattern" + }, + "unit_pattern": { + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Unit Pattern" + }, + "units": { + "anyOf": [ + { + "items": { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + "maxItems": 8, + "minItems": 1, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Units" + } + }, + "required": [ + "domain" + ], + "title": "EntityMatcher", + "type": "object" + }, + "ForceChargeSwitchControl": { + "additionalProperties": false, + "description": "A ``switch`` role: on = force charging (from the grid if PV is short), off = auto.", + "properties": { + "kind": { + "const": "force_charge", + "title": "Kind", + "type": "string" + } + }, + "required": [ + "kind" + ], + "title": "ForceChargeSwitchControl", + "type": "object" + }, + "ModeSelectControl": { + "additionalProperties": false, + "description": "A ``select`` role choosing the battery's operating mode.", + "properties": { + "kind": { + "const": "mode", + "title": "Kind", + "type": "string" + }, + "options": { + "items": { + "$ref": "#/$defs/BatteryModeOption" + }, + "maxItems": 16, + "minItems": 2, + "title": "Options", + "type": "array" + } + }, + "required": [ + "kind", + "options" + ], + "title": "ModeSelectControl", + "type": "object" + }, + "NumberLimitControl": { + "additionalProperties": false, + "description": "A ``number`` role that sets a battery limit.\n\n``min_soc``: discharge floor in ``%`` (discharge lock / reserve); ``max_charge`` and\n``max_discharge``: power limits; ``grid_charge_power``: forced charging power from the\ngrid. ``min``/``max``/``step`` are the entity's range in ``unit`` (``%`` of the\nbattery's maximum power for the power kinds).", + "properties": { + "kind": { + "enum": [ + "min_soc", + "max_charge", + "max_discharge", + "grid_charge_power" + ], + "title": "Kind", + "type": "string" + }, + "max": { + "exclusiveMinimum": 0, + "title": "Max", + "type": "number" + }, + "min": { + "minimum": 0, + "title": "Min", + "type": "number" + }, + "step": { + "anyOf": [ + { + "exclusiveMinimum": 0, + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Step" + }, + "unit": { + "enum": [ + "W", + "kW", + "%" + ], + "title": "Unit", + "type": "string" + } + }, + "required": [ + "kind", + "unit", + "min", + "max" + ], + "title": "NumberLimitControl", + "type": "object" + }, + "RoleSpec": { + "additionalProperties": false, + "description": "One entity role of a template.", + "properties": { + "description": { + "maxLength": 120, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$", + "title": "Description", + "type": "string" + }, + "matcher": { + "$ref": "#/$defs/EntityMatcher" + }, + "required": { + "default": true, + "title": "Required", + "type": "boolean" + }, + "same_device_as": { + "anyOf": [ + { + "pattern": "^[a-z][a-z0-9_]{0,31}$", + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Same Device As" + } + }, + "required": [ + "matcher", + "description" + ], + "title": "RoleSpec", + "type": "object" + }, + "SourceRef": { + "additionalProperties": false, + "description": "Documentation a template (or one of its facts) is based on.", + "properties": { + "date": { + "format": "date", + "title": "Date", + "type": "string" + }, + "title": { + "maxLength": 200, + "minLength": 1, + "title": "Title", + "type": "string" + }, + "url": { + "maxLength": 500, + "pattern": "^https?://\\S+$", + "title": "Url", + "type": "string" + } + }, + "required": [ + "title", + "url", + "date" + ], + "title": "SourceRef", + "type": "object" + }, + "TestedWith": { + "additionalProperties": false, + "description": "Integration version the template was verified against.", + "properties": { + "integration": { + "maxLength": 200, + "minLength": 1, + "title": "Integration", + "type": "string" + }, + "version": { + "maxLength": 200, + "minLength": 1, + "title": "Version", + "type": "string" + } + }, + "required": [ + "integration", + "version" + ], + "title": "TestedWith", + "type": "object" + } + }, + "$id": "https://raw.githubusercontent.com/pxljake/suncore/main/templates/schema/battery.schema.json", + "$schema": "https://json-schema.org/draft/2020-12/schema", + "additionalProperties": false, + "description": "Battery adapter template (Phase 4, ADR-0006/ADR-0012 kind ``battery``).\n\n``roles`` are the readable and writable entities (e.g. ``soc``, ``power``,\n``min_soc_number``, ``max_charge_number``, ``max_discharge_number``, ``mode_select``,\n``force_charge_switch``); ``controls`` says what each *writable* role expresses. A\nrole without a control is read only. Writes go through the safety gate only.", + "properties": { + "capabilities": { + "items": { + "$ref": "#/$defs/Capability" + }, + "maxItems": 32, + "title": "Capabilities", + "type": "array" + }, + "controls": { + "maxProperties": 8, + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "discriminator": { + "mapping": { + "force_charge": "#/$defs/ForceChargeSwitchControl", + "grid_charge_power": "#/$defs/NumberLimitControl", + "max_charge": "#/$defs/NumberLimitControl", + "max_discharge": "#/$defs/NumberLimitControl", + "min_soc": "#/$defs/NumberLimitControl", + "mode": "#/$defs/ModeSelectControl" + }, + "propertyName": "kind" + }, + "oneOf": [ + { + "$ref": "#/$defs/NumberLimitControl" + }, + { + "$ref": "#/$defs/ModeSelectControl" + }, + { + "$ref": "#/$defs/ForceChargeSwitchControl" + } + ] + } + }, + "title": "Controls", + "type": "object" + }, + "id": { + "maxLength": 64, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)?$", + "title": "Id", + "type": "string" + }, + "kind": { + "const": "battery", + "title": "Kind", + "type": "string" + }, + "min_app_version": { + "anyOf": [ + { + "pattern": "^\\d+(\\.\\d+)*((a|b|rc)\\d+)?(\\.post\\d+)?(\\.dev\\d+)?$", + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Min App Version" + }, + "name": { + "maxLength": 120, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$", + "title": "Name", + "type": "string" + }, + "roles": { + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "$ref": "#/$defs/RoleSpec" + } + }, + "title": "Roles", + "type": "object" + }, + "schema_version": { + "const": 1, + "title": "Schema Version", + "type": "integer" + }, + "settle_s": { + "anyOf": [ + { + "maximum": 600, + "minimum": 0, + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Settle S" + }, + "source": { + "items": { + "$ref": "#/$defs/SourceRef" + }, + "minItems": 1, + "title": "Source", + "type": "array" + }, + "tested_with": { + "items": { + "$ref": "#/$defs/TestedWith" + }, + "title": "Tested With", + "type": "array" + }, + "unverified": { + "items": { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + "title": "Unverified", + "type": "array" + } + }, + "required": [ + "id", + "schema_version", + "name", + "source", + "tested_with", + "capabilities", + "kind", + "roles" + ], + "title": "BatteryTemplate", + "type": "object" +} diff --git a/templates/schema/device.schema.json b/templates/schema/device.schema.json index 319a8f0..65a4352 100644 --- a/templates/schema/device.schema.json +++ b/templates/schema/device.schema.json @@ -16,7 +16,13 @@ "pv_curve", "pv_quantiles", "feed_in_tiers", - "negative_price_rule" + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" ], "title": "Capability", "type": "string" diff --git a/templates/schema/forecast.schema.json b/templates/schema/forecast.schema.json index 7150c23..a80d5d5 100644 --- a/templates/schema/forecast.schema.json +++ b/templates/schema/forecast.schema.json @@ -78,7 +78,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -89,6 +89,20 @@ "fields": { "$ref": "#/$defs/SeriesFields" }, + "list_path": { + "anyOf": [ + { + "maxLength": 200, + "pattern": "^[A-Za-z0-9_]+(\\.[A-Za-z0-9_]+)*$", + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "List Path" + }, "roles": { "items": { "pattern": "^[a-z][a-z0-9_]{0,31}$", @@ -130,7 +144,13 @@ "pv_curve", "pv_quantiles", "feed_in_tiers", - "negative_price_rule" + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" ], "title": "Capability", "type": "string" diff --git a/templates/schema/region.schema.json b/templates/schema/region.schema.json index 06ea6dd..172aa79 100644 --- a/templates/schema/region.schema.json +++ b/templates/schema/region.schema.json @@ -16,7 +16,13 @@ "pv_curve", "pv_quantiles", "feed_in_tiers", - "negative_price_rule" + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" ], "title": "Capability", "type": "string" diff --git a/templates/schema/tariff.schema.json b/templates/schema/tariff.schema.json index 0806c2d..b791525 100644 --- a/templates/schema/tariff.schema.json +++ b/templates/schema/tariff.schema.json @@ -78,7 +78,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -89,6 +89,20 @@ "fields": { "$ref": "#/$defs/SeriesFields" }, + "list_path": { + "anyOf": [ + { + "maxLength": 200, + "pattern": "^[A-Za-z0-9_]+(\\.[A-Za-z0-9_]+)*$", + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "List Path" + }, "roles": { "items": { "pattern": "^[a-z][a-z0-9_]{0,31}$", @@ -130,7 +144,13 @@ "pv_curve", "pv_quantiles", "feed_in_tiers", - "negative_price_rule" + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" ], "title": "Capability", "type": "string" diff --git a/templates/schema/template.schema.json b/templates/schema/template.schema.json index 61e7ff1..e0da0b2 100644 --- a/templates/schema/template.schema.json +++ b/templates/schema/template.schema.json @@ -78,7 +78,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -89,6 +89,20 @@ "fields": { "$ref": "#/$defs/SeriesFields" }, + "list_path": { + "anyOf": [ + { + "maxLength": 200, + "pattern": "^[A-Za-z0-9_]+(\\.[A-Za-z0-9_]+)*$", + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "List Path" + }, "roles": { "items": { "pattern": "^[a-z][a-z0-9_]{0,31}$", @@ -114,6 +128,177 @@ "title": "AttributeSeries", "type": "object" }, + "BatteryModeMeaning": { + "description": "What selecting a battery mode option does.", + "enum": [ + "auto", + "force_charge", + "no_discharge", + "no_charge", + "other" + ], + "title": "BatteryModeMeaning", + "type": "string" + }, + "BatteryModeOption": { + "additionalProperties": false, + "properties": { + "meaning": { + "$ref": "#/$defs/BatteryModeMeaning" + }, + "value": { + "maxLength": 200, + "minLength": 1, + "title": "Value", + "type": "string" + } + }, + "required": [ + "value", + "meaning" + ], + "title": "BatteryModeOption", + "type": "object" + }, + "BatteryTemplate": { + "additionalProperties": false, + "description": "Battery adapter template (Phase 4, ADR-0006/ADR-0012 kind ``battery``).\n\n``roles`` are the readable and writable entities (e.g. ``soc``, ``power``,\n``min_soc_number``, ``max_charge_number``, ``max_discharge_number``, ``mode_select``,\n``force_charge_switch``); ``controls`` says what each *writable* role expresses. A\nrole without a control is read only. Writes go through the safety gate only.", + "properties": { + "capabilities": { + "items": { + "$ref": "#/$defs/Capability" + }, + "maxItems": 32, + "title": "Capabilities", + "type": "array" + }, + "controls": { + "maxProperties": 8, + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "discriminator": { + "mapping": { + "force_charge": "#/$defs/ForceChargeSwitchControl", + "grid_charge_power": "#/$defs/NumberLimitControl", + "max_charge": "#/$defs/NumberLimitControl", + "max_discharge": "#/$defs/NumberLimitControl", + "min_soc": "#/$defs/NumberLimitControl", + "mode": "#/$defs/ModeSelectControl" + }, + "propertyName": "kind" + }, + "oneOf": [ + { + "$ref": "#/$defs/NumberLimitControl" + }, + { + "$ref": "#/$defs/ModeSelectControl" + }, + { + "$ref": "#/$defs/ForceChargeSwitchControl" + } + ] + } + }, + "title": "Controls", + "type": "object" + }, + "id": { + "maxLength": 64, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)?$", + "title": "Id", + "type": "string" + }, + "kind": { + "const": "battery", + "title": "Kind", + "type": "string" + }, + "min_app_version": { + "anyOf": [ + { + "pattern": "^\\d+(\\.\\d+)*((a|b|rc)\\d+)?(\\.post\\d+)?(\\.dev\\d+)?$", + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Min App Version" + }, + "name": { + "maxLength": 120, + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$", + "title": "Name", + "type": "string" + }, + "roles": { + "patternProperties": { + "^[a-z][a-z0-9_]{0,31}$": { + "$ref": "#/$defs/RoleSpec" + } + }, + "title": "Roles", + "type": "object" + }, + "schema_version": { + "const": 1, + "title": "Schema Version", + "type": "integer" + }, + "settle_s": { + "anyOf": [ + { + "maximum": 600, + "minimum": 0, + "type": "integer" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Settle S" + }, + "source": { + "items": { + "$ref": "#/$defs/SourceRef" + }, + "minItems": 1, + "title": "Source", + "type": "array" + }, + "tested_with": { + "items": { + "$ref": "#/$defs/TestedWith" + }, + "title": "Tested With", + "type": "array" + }, + "unverified": { + "items": { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + "title": "Unverified", + "type": "array" + } + }, + "required": [ + "id", + "schema_version", + "name", + "source", + "tested_with", + "capabilities", + "kind", + "roles" + ], + "title": "BatteryTemplate", + "type": "object" + }, "Capability": { "description": "What a template provides.", "enum": [ @@ -130,7 +315,13 @@ "pv_curve", "pv_quantiles", "feed_in_tiers", - "negative_price_rule" + "negative_price_rule", + "battery_soc", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes" ], "title": "Capability", "type": "string" @@ -662,6 +853,22 @@ "title": "FeedInTier", "type": "object" }, + "ForceChargeSwitchControl": { + "additionalProperties": false, + "description": "A ``switch`` role: on = force charging (from the grid if PV is short), off = auto.", + "properties": { + "kind": { + "const": "force_charge", + "title": "Kind", + "type": "string" + } + }, + "required": [ + "kind" + ], + "title": "ForceChargeSwitchControl", + "type": "object" + }, "ForecastTemplate": { "additionalProperties": false, "properties": { @@ -845,6 +1052,32 @@ "title": "ModeControl", "type": "object" }, + "ModeSelectControl": { + "additionalProperties": false, + "description": "A ``select`` role choosing the battery's operating mode.", + "properties": { + "kind": { + "const": "mode", + "title": "Kind", + "type": "string" + }, + "options": { + "items": { + "$ref": "#/$defs/BatteryModeOption" + }, + "maxItems": 16, + "minItems": 2, + "title": "Options", + "type": "array" + } + }, + "required": [ + "kind", + "options" + ], + "title": "ModeSelectControl", + "type": "object" + }, "NegativePriceRule": { "description": "Feed-in remuneration during negative spot prices.", "enum": [ @@ -855,6 +1088,62 @@ "title": "NegativePriceRule", "type": "string" }, + "NumberLimitControl": { + "additionalProperties": false, + "description": "A ``number`` role that sets a battery limit.\n\n``min_soc``: discharge floor in ``%`` (discharge lock / reserve); ``max_charge`` and\n``max_discharge``: power limits; ``grid_charge_power``: forced charging power from the\ngrid. ``min``/``max``/``step`` are the entity's range in ``unit`` (``%`` of the\nbattery's maximum power for the power kinds).", + "properties": { + "kind": { + "enum": [ + "min_soc", + "max_charge", + "max_discharge", + "grid_charge_power" + ], + "title": "Kind", + "type": "string" + }, + "max": { + "exclusiveMinimum": 0, + "title": "Max", + "type": "number" + }, + "min": { + "minimum": 0, + "title": "Min", + "type": "number" + }, + "step": { + "anyOf": [ + { + "exclusiveMinimum": 0, + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Step" + }, + "unit": { + "enum": [ + "W", + "kW", + "%" + ], + "title": "Unit", + "type": "string" + } + }, + "required": [ + "kind", + "unit", + "min", + "max" + ], + "title": "NumberLimitControl", + "type": "object" + }, "OnOffControl": { "additionalProperties": false, "description": "Switch on/off through ``role``.", @@ -1577,6 +1866,7 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "discriminator": { "mapping": { + "battery": "#/$defs/BatteryTemplate", "device": "#/$defs/DeviceTemplate", "forecast": "#/$defs/ForecastTemplate", "region": "#/$defs/RegionTemplate", @@ -1596,6 +1886,9 @@ }, { "$ref": "#/$defs/RegionTemplate" + }, + { + "$ref": "#/$defs/BatteryTemplate" } ], "title": "SunCore template v1" diff --git a/tests/api/test_config_api.py b/tests/api/test_config_api.py index 5e3a6b3..206d5c4 100644 --- a/tests/api/test_config_api.py +++ b/tests/api/test_config_api.py @@ -19,13 +19,18 @@ "export": {"kind": "fixed", "fixed_price": 0.08}, }, "forecast": { - "forecast_template_ids": ["solcast.pv"], - "forecast_entities": {"today": {"entity_id": "sensor.fc_today"}}, + "sources": [ + {"template_id": "solcast.pv", "entities": {"today": {"entity_id": "sensor.fc_today"}}} + ], }, "devices": {"devices": []}, "goals": {"climate_before_export": True}, "operation": {"mode": "shadow", "device_modes": {}}, "wizard": {"completed": False, "completed_steps": ["region"]}, + "battery": { + "model": {"capacity_wh": 10_000, "max_charge_w": 5000, "max_discharge_w": 5000}, + "strategy": "prefer_full", + }, } diff --git a/tests/api/test_templates_api.py b/tests/api/test_templates_api.py index ee90e1b..0ead5fc 100644 --- a/tests/api/test_templates_api.py +++ b/tests/api/test_templates_api.py @@ -45,7 +45,7 @@ async def test_filter_by_kind(services: FakeServices, kind: str) -> None: async def test_unknown_kind_is_rejected(services: FakeServices) -> None: async with client_for(create_app(services)) as client: - r = await client.get("/api/v1/templates", params={"kind": "battery"}) + r = await client.get("/api/v1/templates", params={"kind": "inverter"}) assert r.status_code == 422 diff --git a/tests/app/test_demo.py b/tests/app/test_demo.py index e951df3..d461795 100644 --- a/tests/app/test_demo.py +++ b/tests/app/test_demo.py @@ -139,7 +139,7 @@ async def test_demo_starts_fully_configured(demo_runtime: Runtime) -> None: assert tariff.import_.price_entity == EntityRef(entity_id="sensor.sim_electricity_price") forecast = await demo_runtime.get_section("forecast") assert isinstance(forecast, ForecastConfig) - assert forecast.forecast_template_ids == ["solcast.pv"] + assert forecast.template_ids == ["solcast.pv"] operation = await demo_runtime.get_section("operation") assert isinstance(operation, OperationConfig) assert operation.mode is OperatingMode.SHADOW diff --git a/tests/core/test_battery.py b/tests/core/test_battery.py new file mode 100644 index 0000000..5e29de9 --- /dev/null +++ b/tests/core/test_battery.py @@ -0,0 +1,170 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Battery model, strategies, grid-charge policy, risk levels and the ``battery`` section.""" + +from __future__ import annotations + +from datetime import datetime +from zoneinfo import ZoneInfo + +import pytest +from hypothesis import given +from hypothesis import strategies as st +from pydantic import TypeAdapter, ValidationError + +from suncore.core import battery +from suncore.core.battery import ( + BatteryAction, + BatteryModel, + BatterySettings, + BatteryStrategy, + CalibrationCharge, + GridChargePolicy, + RiskLevel, +) +from suncore.core.decisions import Action, ActionKind +from suncore.core.decisions import BatteryAction as CoreBatteryAction +from suncore.core.site import EntityRef +from suncore.core.tariff import NIGHT_WINDOW + +BERLIN = ZoneInfo("Europe/Berlin") + + +def model(**kw: float) -> BatteryModel: + base: dict[str, float] = { + "capacity_wh": 29_600, + "max_charge_w": 10_000, + "max_discharge_w": 10_000, + } + return BatteryModel.model_validate(base | kw) + + +def test_defaults_are_the_maintainers_answers() -> None: + m = model() + assert (m.charge_efficiency, m.discharge_efficiency) == (0.95, 0.95) + assert m.degradation_cost_per_kwh == 0.04 + assert m.grid_charge_margin_per_kwh == 0.03 + assert m.bms_drift_margin_pct == 5.0 + assert m.reserve_pct == 10.0 + assert m.max_soc_pct == 100.0 + assert m.min_soc_pct == 15.0 + assert m.round_trip_efficiency == pytest.approx(0.9025) + assert m.usable_wh == pytest.approx(29_600 * 0.85) + assert m.soc_to_wh(50) == 14_800 + assert m.wh_to_soc(14_800) == 50 + + +def test_corridor_must_stay_open() -> None: + with pytest.raises(ValidationError, match="must be below max_soc_pct"): + model(reserve_pct=60, bms_drift_margin_pct=5, max_soc_pct=65) + for bad in ({"charge_efficiency": 0}, {"charge_efficiency": 1.1}, {"capacity_wh": 0}): + with pytest.raises(ValidationError): + model(**bad) + + +def test_spread_rule() -> None: + m = model() + threshold = (0.20 + 0.04 + 0.03) / 0.9025 + assert m.min_later_price(0.20) == pytest.approx(threshold) + assert m.grid_charge_pays(0.20, threshold + 0.001) + assert not m.grid_charge_pays(0.20, threshold) + + +@given(charge=st.floats(-0.5, 1.0), later=st.floats(-0.5, 2.0)) +def test_grid_charging_never_pays_below_charge_price(charge: float, later: float) -> None: + if model().grid_charge_pays(charge, later): + assert later > charge + + +def test_strategies_and_risk_levels() -> None: + assert [s.value for s in BatteryStrategy] == [ + "self_consumption", + "prefer_full", + "target_by_time", + ] + assert [(r.value, r.probability, r.quantile) for r in RiskLevel] == [ + ("cautious", 0.9, 0.1), + ("balanced", 0.75, 0.25), + ("bold", 0.5, 0.5), + ] + + +def test_grid_charge_policy() -> None: + off = GridChargePolicy() + friday_night = datetime(2026, 10, 16, 23, tzinfo=BERLIN) + noon = friday_night.replace(hour=12) + assert not off.allows(friday_night, 0.0) + with pytest.raises(ValidationError, match="price level or a schedule"): + GridChargePolicy(enabled=True) + level = GridChargePolicy(enabled=True, max_price=0.15) + assert level.allows(noon, 0.15) + assert not level.allows(noon, 0.16) + assert not level.allows(noon, None) # unknown price never satisfies a level + sched = GridChargePolicy(enabled=True, schedule=NIGHT_WINDOW) + assert sched.allows(friday_night, None) + assert not sched.allows(noon, 0.0) + both = GridChargePolicy(enabled=True, max_price=-0.01, schedule=NIGHT_WINDOW) + assert both.allows(friday_night, -0.05) + assert not both.allows(friday_night, 0.0) + assert GridChargePolicy(max_price=0.1).enabled is False # a level alone enables nothing + + +def test_calibration_is_off_by_default() -> None: + c = CalibrationCharge() + assert (c.enabled, c.interval_days, c.target_soc_pct) == (False, 30, 100.0) + with pytest.raises(ValidationError): + CalibrationCharge(interval_days=0) + + +def test_battery_settings_defaults_and_rules() -> None: + s = BatterySettings() + assert s.schema_version == 1 + assert s.observe_only + assert s.strategy is BatteryStrategy.SELF_CONSUMPTION + assert s.risk is RiskLevel.CAUTIOUS + assert s.morning_buffer_pct is None + assert s.bridge_next_day is True + assert s.autarky_weight_per_kwh == 0.01 + assert not s.grid_charge.enabled + assert not s.calibration.enabled + assert s.entity_refs() == [] + with pytest.raises(ValidationError, match="adapter template"): + BatterySettings(entities={"min_soc_number": EntityRef(entity_id="number.min_soc")}) + with pytest.raises(ValidationError, match="more than once"): + BatterySettings( + adapter_template_id="generic.battery", + entities={ + "a": EntityRef(entity_id="number.x"), + "b": EntityRef(entity_id="number.x"), + }, + ) + for bad in ({"autarky_weight_per_kwh": 1.5}, {"morning_buffer_pct": 101}, {"risk": "yolo"}): + with pytest.raises(ValidationError): + BatterySettings.model_validate(bad) + s2 = BatterySettings( + adapter_template_id="generic.battery", + entities={"min_soc_number": EntityRef(entity_id="number.min_soc")}, + model=model(), + strategy=BatteryStrategy.PREFER_FULL, + ) + assert not s2.observe_only + assert [r.entity_id for r in s2.entity_refs()] == ["number.min_soc"] + assert BatterySettings.model_validate_json(s2.model_dump_json()) == s2 + + +def test_battery_action_contract() -> None: + assert battery.BatteryAction is CoreBatteryAction + a = BatteryAction(min_soc_pct=20) + assert a.kind is ActionKind.BATTERY + parsed = TypeAdapter(Action).validate_python({"kind": "battery", "grid_charge_w": 3000}) + assert isinstance(parsed, BatteryAction) + assert parsed.grid_charge_w == 3000 + with pytest.raises(ValidationError, match="at least one limit"): + BatteryAction() + for bad in ( + {"max_charge_w": -1}, + {"max_discharge_w": float("inf")}, + {"min_soc_pct": 101}, + {"grid_charge_w": -5}, + ): + with pytest.raises(ValidationError): + BatteryAction.model_validate(bad) diff --git a/tests/core/test_config_phase4.py b/tests/core/test_config_phase4.py new file mode 100644 index 0000000..e739551 --- /dev/null +++ b/tests/core/test_config_phase4.py @@ -0,0 +1,130 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Phase 4 config documents: tariff v3, forecast v2, goals v2, battery section, device value.""" + +from __future__ import annotations + +import copy +from typing import Any + +import pytest +from pydantic import ValidationError + +from suncore.core.battery import BatterySettings +from suncore.core.config import ( + SECTION_MODELS, + ForecastConfig, + GoalsConfig, + ImportTariff, + TariffConfig, + migrate_forecast_v1_to_v2, + migrate_goals_v1_to_v2, + migrate_tariff_v2_to_v3, +) +from suncore.core.devices import DeviceConfig +from suncore.core.tariff import NIGHT_WINDOW, PriceFormula, TouPeriod, TouTariff, TouWindow + +TOU = TouTariff( + base_price=0.32, + periods=(TouPeriod(window=TouWindow(start="22:00", end="06:00"), price=0.24),), +) + + +def test_tariff_v3_formula_and_tou() -> None: + dyn = ImportTariff( + kind="dynamic", template_id="nordpool.spot", formula=PriceFormula(grid_fee=0.08, vat_pct=19) + ) + assert dyn.formula is not None + tou = ImportTariff(kind="tou", tou=TOU) + t = TariffConfig.model_validate( + {"import": tou.model_dump(mode="json"), "export": {"kind": "none"}} + ) + assert t.schema_version == 3 + assert TariffConfig.model_validate_json(t.model_dump_json(by_alias=True)) == t + + +@pytest.mark.parametrize( + ("doc", "key"), + [ + ({"kind": "tou"}, "needs `tou`"), + ({"kind": "tou", "tou": TOU, "fixed_price": 0.3}, "takes no fixed price"), + ({"kind": "fixed", "fixed_price": 0.3, "formula": PriceFormula()}, "needs kind `dynamic`"), + ({"kind": "fixed", "fixed_price": 0.3, "tou": TOU}, "needs kind `tou`"), + ({"kind": "dynamic", "template_id": "a.b", "tou": TOU}, "needs kind `tou`"), + ], +) +def test_tariff_v3_rejects(doc: dict[str, Any], key: str) -> None: + with pytest.raises(ValidationError, match=key): + ImportTariff.model_validate(doc) + + +def test_tariff_v2_body_is_a_valid_v3_body() -> None: + v2 = { + "schema_version": 2, + "import": {"kind": "dynamic", "template_id": "tibber.price"}, + "export": {"kind": "fixed", "fixed_price": 0.08}, + } + body = migrate_tariff_v2_to_v3(copy.deepcopy(v2)) + assert body == v2 + t = TariffConfig.model_validate({**body, "schema_version": 3}) + assert t.import_.formula is None + + +def test_forecast_v1_to_v2() -> None: + body = migrate_forecast_v1_to_v2( + { + "schema_version": 1, + "forecast_template_ids": ["solcast.pv", "other.pv"], + "forecast_entities": {"forecast_today": {"entity_id": "sensor.today"}}, + } + ) + f = ForecastConfig.model_validate({**body, "schema_version": 2}) + assert f.template_ids == ["solcast.pv", "other.pv"] + assert all(s.weight_mode == "auto" for s in f.sources) + assert [list(s.entities) for s in f.sources] == [["forecast_today"]] * 2 + assert migrate_forecast_v1_to_v2({"schema_version": 1}) == { + "schema_version": 1, + "sources": [], + } + with pytest.raises(TypeError): + migrate_forecast_v1_to_v2({"forecast_template_ids": "solcast.pv"}) + + +@pytest.mark.parametrize( + ("v1", "schedule"), + [ + ({"grid_charge_night_only": True}, NIGHT_WINDOW), + ({}, NIGHT_WINDOW), # v1 default was True + ({"grid_charge_night_only": False}, None), + ], +) +def test_goals_v1_to_v2(v1: dict[str, Any], schedule: object) -> None: + body = migrate_goals_v1_to_v2({"schema_version": 1, "climate_before_export": True, **v1}) + g = GoalsConfig.model_validate({**body, "schema_version": 2}) + assert g.grid_charge_schedule == schedule + assert g.climate_before_export is True + assert g.conservative_planning is True + with pytest.raises(ValidationError): # the v1 field is gone in v2 + GoalsConfig.model_validate({"grid_charge_night_only": True}) + + +def test_battery_is_a_section() -> None: + assert SECTION_MODELS["battery"] is BatterySettings + + +def test_device_planner_value_is_optional() -> None: + doc = { + "id": "pool", + "template_id": "generic.switch", + "name": "Pool", + "kind": "switch", + "entities": {"switch": {"entity_id": "switch.pool"}}, + "power_model": {"kind": "nominal", "nominal_w": 800}, + "control": {"kind": "on_off", "role": "switch"}, + } + assert DeviceConfig.model_validate(doc).planner_value_per_kwh is None + assert ( + DeviceConfig.model_validate(doc | {"planner_value_per_kwh": 0.1}).planner_value_per_kwh + == 0.1 + ) + with pytest.raises(ValidationError): + DeviceConfig.model_validate(doc | {"planner_value_per_kwh": -0.1}) diff --git a/tests/core/test_config_sections.py b/tests/core/test_config_sections.py index 2f5b2df..102a193 100644 --- a/tests/core/test_config_sections.py +++ b/tests/core/test_config_sections.py @@ -14,6 +14,7 @@ DevicesConfig, ExportTariff, ForecastConfig, + ForecastSourceConfig, GoalsConfig, ImportTariff, OperationConfig, @@ -193,7 +194,7 @@ def test_tariff_section() -> None: ), ) assert [r.entity_id for r in t.entity_refs()] == ["sensor.p"] - assert t.schema_version == 2 + assert t.schema_version == 3 assert TariffConfig.model_validate_json(t.model_dump_json()) == t with pytest.raises(ValidationError): # forecast fields moved to ForecastConfig TariffConfig.model_validate( @@ -202,15 +203,29 @@ def test_tariff_section() -> None: def test_forecast_section() -> None: + today = {"today": EntityRef(entity_id="sensor.fc_today")} f = ForecastConfig( - forecast_template_ids=["forecast.solcast"], - forecast_entities={"today": EntityRef(entity_id="sensor.fc_today")}, + sources=[ + ForecastSourceConfig(template_id="forecast.solcast", entities=today), + ForecastSourceConfig( + template_id="b.pv", entities=today, weight_mode="manual", weight=0.3 + ), + ] ) - assert ForecastConfig().forecast_template_ids == [] - assert [r.entity_id for r in f.entity_refs()] == ["sensor.fc_today"] + assert ForecastConfig().sources == [] + assert ForecastConfig().learning.min_history_days == 30 + assert ForecastConfig().learning.load_ml_mode == "shadow" + assert f.template_ids == ["forecast.solcast", "b.pv"] + assert [r.entity_id for r in f.entity_refs()] == ["sensor.fc_today"] # shared, listed once assert ForecastConfig.model_validate_json(f.model_dump_json()) == f - with pytest.raises(ValidationError, match="unique"): - ForecastConfig(forecast_template_ids=["a.b", "a.b"]) + with pytest.raises(ValidationError, match="every template only once"): + ForecastConfig(sources=[ForecastSourceConfig(template_id="a.b")] * 2) + with pytest.raises(ValidationError, match="needs `weight`"): + ForecastSourceConfig(template_id="a.b", weight_mode="manual") + with pytest.raises(ValidationError, match="only used with"): + ForecastSourceConfig(template_id="a.b", weight=0.5) + with pytest.raises(ValidationError): + ForecastConfig.model_validate({"safety_discount_pct": 10}) # --- devices ----------------------------------------------------------------------------- @@ -249,7 +264,9 @@ def test_operation_without_modes_of() -> None: def test_goals() -> None: g = GoalsConfig() assert g.battery_target is None - assert g.grid_charge_night_only is True + assert g.grid_charge_schedule is None + assert g.conservative_planning is True + assert g.schema_version == 2 assert BatteryTarget(soc_pct=80, time="06:30").time == "06:30" for bad in ("6:30", "24:00", "06:60", "06:30:00", ""): with pytest.raises(ValidationError): diff --git a/tests/core/test_decisions.py b/tests/core/test_decisions.py index 78f99b6..208cd55 100644 --- a/tests/core/test_decisions.py +++ b/tests/core/test_decisions.py @@ -119,6 +119,7 @@ def test_enums_are_stable_strings() -> None: "step", "mode", "release", + "battery", ] diff --git a/tests/core/test_forecast_contracts.py b/tests/core/test_forecast_contracts.py new file mode 100644 index 0000000..ea387cd --- /dev/null +++ b/tests/core/test_forecast_contracts.py @@ -0,0 +1,78 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Forecast/price source protocols and accuracy records (Phase 4 WP2).""" + +from __future__ import annotations + +import asyncio +from datetime import UTC, datetime, timedelta + +import pytest +from pydantic import ValidationError + +from suncore.core.forecast import AccuracyRecord, ForecastSource, HorizonBucket, PriceSource +from suncore.core.series import ( + PriceSeries, + QuantileSeries, + SeriesUnit, + TimeRange, + TimeSeries, +) + +T0 = datetime(2026, 6, 1, tzinfo=UTC) + + +class FlatPv: + source_id = "test.flat" + + async def fetch(self, window: TimeRange) -> QuantileSeries: + n = int(window.duration / timedelta(minutes=15)) + series = TimeSeries( + start=window.start, step=timedelta(minutes=15), values=(1000.0,) * n, unit=SeriesUnit.W + ) + return QuantileSeries.deterministic(series) + + +class FixedPrice: + source_id = "test.fixed" + + async def fetch(self, window: TimeRange) -> PriceSeries: + series = TimeSeries( + start=window.start, step=window.duration, values=(0.3,), unit=SeriesUnit.PRICE + ) + return PriceSeries(source_id=self.source_id, price_includes="all_in", import_price=series) + + +def test_adapters_satisfy_the_protocols() -> None: + pv: ForecastSource = FlatPv() + price: PriceSource = FixedPrice() + assert isinstance(pv, ForecastSource) + assert isinstance(price, PriceSource) + window = TimeRange(start=T0, end=T0 + timedelta(hours=1)) + assert len(asyncio.run(pv.fetch(window))) == 4 + assert asyncio.run(price.fetch(window)).import_price.values == (0.3,) + + +@pytest.mark.parametrize( + ("hours", "bucket"), + [ + (-1, HorizonBucket.H0_6), + (0, HorizonBucket.H0_6), + (5.99, HorizonBucket.H0_6), + (6, HorizonBucket.H6_24), + (24, HorizonBucket.H24_48), + (47.9, HorizonBucket.H24_48), + (48, HorizonBucket.H48_PLUS), + ], +) +def test_horizon_buckets(hours: float, bucket: HorizonBucket) -> None: + assert HorizonBucket.of(timedelta(hours=hours)) is bucket + + +def test_accuracy_record() -> None: + r = AccuracyRecord( + source_id="solcast.pv", horizon=HorizonBucket.H6_24, mae_w=120, bias_w=-30, n=96 + ) + assert AccuracyRecord.model_validate_json(r.model_dump_json()) == r + for bad in ({"mae_w": -1}, {"n": -1}, {"source_id": "Bad Id"}, {"bias_w": float("nan")}): + with pytest.raises(ValidationError): + AccuracyRecord.model_validate(r.model_dump() | bad) diff --git a/tests/core/test_plan_v2.py b/tests/core/test_plan_v2.py new file mode 100644 index 0000000..1431527 --- /dev/null +++ b/tests/core/test_plan_v2.py @@ -0,0 +1,131 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Plan schema v2 (Phase 4 WP2): additive fields, v1 payloads keep loading.""" + +from __future__ import annotations + +import json +from datetime import UTC, datetime, timedelta + +import pytest +from pydantic import ValidationError + +from suncore.core.control import ( + PLAN_SCHEMA_VERSION, + BatteryBand, + IntervalFlows, + IntervalPrices, + Plan, + PlanInterval, + SolverStats, + plan_from_payload, +) +from suncore.core.decisions import Reason + +T0 = datetime(2026, 6, 1, tzinfo=UTC) +Q = timedelta(minutes=15) + +V1_PAYLOAD = { + "schema_version": 1, + "created_at": "2026-06-01T00:00:00Z", + "intervals": [ + { + "start": "2026-06-01T00:00:00Z", + "end": "2026-06-01T00:15:00Z", + "devices": {"pool": {"allowed": False, "preference": "neutral", "max_power_w": None}}, + "battery": { + "min_power_w": None, + "max_power_w": None, + "min_soc_pct": 15, + "max_soc_pct": None, + "charge_from_surplus": True, + }, + "grid_import_limit_w": None, + } + ], + "reasons": [], + "assumptions": {"mode": "shadow"}, +} + + +def test_new_plans_are_v2_and_default_is_default() -> None: + assert PLAN_SCHEMA_VERSION == 2 + p = Plan.default() + assert (p.schema_version, p.source, p.stats, p.is_default) == (2, "default", None, True) + + +def test_v1_payload_loads_unchanged_and_through_the_loader() -> None: + plain = Plan.model_validate(V1_PAYLOAD) + assert plain.schema_version == 1 + assert plain.source == "default" + assert plain.intervals[0].expected is None + assert plain.intervals[0].reasons == () + assert plain.intervals[0].battery.priority_until_soc_pct is None + stamped = plan_from_payload(V1_PAYLOAD) + assert stamped.schema_version == 2 + assert stamped.band_for("pool", T0).allowed is False + assert plan_from_payload(json.loads(stamped.model_dump_json())) == stamped + + +@pytest.mark.parametrize("version", [3, 0, "2", True]) +def test_loader_refuses_unknown_versions(version: object) -> None: + with pytest.raises(ValueError, match="schema_version"): + plan_from_payload({**V1_PAYLOAD, "schema_version": version}) + + +def test_v2_interval_round_trip() -> None: + flows = IntervalFlows( + pv_p10_w=800, + pv_p50_w=1200, + pv_p90_w=1500, + load_w=600, + import_w=0, + export_w=100, + battery_w=500, + soc_pct=42.5, + ) + interval = PlanInterval( + start=T0, + end=T0 + Q, + battery=BatteryBand(min_soc_pct=15, priority_until_soc_pct=60), + expected=flows, + prices=IntervalPrices(import_price=0.31, export_price=-0.01), + reasons=(Reason(key="planner.battery.charge_from_pv", args={"pv_p10_kwh": 0.2}),), + ) + plan = Plan( + created_at=T0, + intervals=(interval,), + source="solver", + stats=SolverStats( + solve_time_s=0.8, status="optimal", objective=1.23, n_vars=960, n_binaries=0, mip_gap=0 + ), + ) + assert Plan.model_validate_json(plan.model_dump_json()) == plan + assert plan.battery_band_at(T0).priority_until_soc_pct == 60 + + +@pytest.mark.parametrize( + "bad", + [ + {"pv_p10_w": 1300}, # above p50 + {"pv_p90_w": 1100}, # below p50 + {"import_w": -1}, + {"soc_pct": 101}, + {"load_w": float("nan")}, + ], +) +def test_interval_flows_reject(bad: dict[str, float]) -> None: + doc = {"pv_p50_w": 1200, "load_w": 600, "import_w": 0, "export_w": 0} | bad + with pytest.raises(ValidationError): + IntervalFlows.model_validate(doc) + + +def test_plan_source_stats_and_band_validation() -> None: + with pytest.raises(ValidationError): + Plan.model_validate({"source": "magic"}) + with pytest.raises(ValidationError): + SolverStats(solve_time_s=-1, status="optimal") + with pytest.raises(ValidationError): + SolverStats(solve_time_s=1, status="Not A Slug") + with pytest.raises(ValidationError): + BatteryBand(priority_until_soc_pct=120) + assert IntervalFlows(pv_p50_w=0, load_w=0, import_w=0, export_w=0).battery_w == 0 diff --git a/tests/core/test_series.py b/tests/core/test_series.py new file mode 100644 index 0000000..f6bbba4 --- /dev/null +++ b/tests/core/test_series.py @@ -0,0 +1,315 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Series contracts and resampling onto the 15-minute planning grid (Phase 4 WP2).""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from zoneinfo import ZoneInfo + +import pytest +from hypothesis import given, settings +from hypothesis import strategies as st +from pydantic import ValidationError + +from suncore.core.series import ( + PLANNING_STEP, + Grid, + PriceSeries, + QuantileSeries, + SeriesUnit, + TimeRange, + TimeSeries, + energy_to_power, + floor_to_step, + planning_grid, + resample_energy, + resample_power, + resample_prices, + resample_quantiles, + segments_from_points, +) + +T0 = datetime(2026, 6, 1, tzinfo=UTC) +BERLIN = ZoneInfo("Europe/Berlin") +MIN15 = timedelta(minutes=15) +HOUR = timedelta(hours=1) + + +def ts( + values: list[float | None], + step: timedelta = HOUR, + start: datetime = T0, + unit: SeriesUnit = SeriesUnit.W, +) -> TimeSeries: + return TimeSeries(start=start, step=step, values=tuple(values), unit=unit) + + +def prices(values: list[float | None], step: timedelta, start: datetime = T0) -> PriceSeries: + return PriceSeries( + source_id="tibber.price", + price_includes="all_in", + import_price=ts(values, step, start, SeriesUnit.PRICE), + ) + + +# --- contracts ------------------------------------------------------------------------- + + +def test_time_series_basics() -> None: + s = ts([1000.0, None, 500.0], start=T0.astimezone(BERLIN)) + assert s.start.tzinfo is UTC # normalised + assert s.end == T0 + 3 * HOUR + assert len(s) == 3 + assert [b for b, _, _ in s.slots()] == [T0, T0 + HOUR, T0 + 2 * HOUR] + assert s.total() is None + assert ts([1000.0, 500.0]).total() == 1500.0 + assert ts([250.0, 250.0], MIN15, unit=SeriesUnit.WH).total() == 500.0 + with pytest.raises(ValueError, match="power or energy"): + ts([1.0], unit=SeriesUnit.PERCENT).total() + + +@pytest.mark.parametrize( + "bad", + [ + {"step": timedelta(0)}, + {"step": timedelta(days=2)}, + {"values": (float("nan"),)}, + {"values": (float("inf"),)}, + {"start": datetime(2026, 1, 1)}, # noqa: DTZ001 - naive on purpose + ], +) +def test_time_series_rejects(bad: dict[str, object]) -> None: + doc: dict[str, object] = {"start": T0, "step": HOUR, "values": (1.0,), "unit": "W"} | bad + with pytest.raises(ValidationError): + TimeSeries.model_validate(doc) + + +def test_time_range() -> None: + assert TimeRange(start=T0, end=T0 + HOUR).duration == HOUR + with pytest.raises(ValidationError, match="start must be before end"): + TimeRange(start=T0, end=T0) + + +def test_quantiles_validate_order_grid_and_unit() -> None: + p10, p50, p90 = ts([1.0, None]), ts([2.0, 5.0]), ts([1.5, 6.0]) + q = QuantileSeries(p10=p10, p50=p50, p90=ts([3.0, None])) + assert (q.start, q.step, q.unit, len(q)) == (T0, HOUR, SeriesUnit.W, 2) + with pytest.raises(ValidationError, match="p10 <= p50 <= p90"): + QuantileSeries(p10=p10, p50=p50, p90=p90) + with pytest.raises(ValidationError, match="same grid"): + QuantileSeries(p10=ts([1.0]), p50=p50, p90=p50) + with pytest.raises(ValidationError, match="same unit"): + QuantileSeries(p10=ts([1.0, 1.0], unit=SeriesUnit.WH), p50=p50, p90=p50) + flat = QuantileSeries.deterministic(p50) + assert flat.p10 == flat.p90 == p50 + + +def test_price_series_validation() -> None: + p = prices([0.3, 0.2], HOUR) + assert p.resolution_min == 60 + assert len(p) == 2 + assert p.start == T0 + assert p.step == HOUR + with pytest.raises(ValidationError, match="currency/kWh"): + PriceSeries(source_id="x", price_includes="all_in", import_price=ts([0.3])) + with pytest.raises(ValidationError, match="same grid"): + PriceSeries( + source_id="x", + price_includes="spot_only", + import_price=ts([0.3, 0.2], unit=SeriesUnit.PRICE), + export_price=ts([0.1], unit=SeriesUnit.PRICE), + ) + + +# --- grid ------------------------------------------------------------------------------ + + +def test_floor_and_planning_grid() -> None: + at = datetime(2026, 6, 1, 10, 7, 30, tzinfo=BERLIN) + assert floor_to_step(at) == datetime(2026, 6, 1, 8, 0, tzinfo=UTC) + grid = planning_grid(at, timedelta(hours=48)) + assert grid.step == PLANNING_STEP + assert grid.start == datetime(2026, 6, 1, 8, 0, tzinfo=UTC) + assert grid.end >= at + timedelta(hours=48) + assert grid.count == 193 # 48 h plus the started quarter hour + with pytest.raises(ValidationError): + Grid(start=T0, step=timedelta(0), count=1) + + +@pytest.mark.parametrize( + ("day", "hours"), + [(datetime(2026, 3, 29, tzinfo=BERLIN), 23), (datetime(2026, 10, 25, tzinfo=BERLIN), 25)], +) +def test_dst_days_have_23_and_25_hours_of_quarter_hours(day: datetime, hours: int) -> None: + next_day = datetime(day.year, day.month, day.day + 1, tzinfo=BERLIN) + grid = Grid(start=day, count=int((next_day.astimezone(UTC) - day.astimezone(UTC)) / MIN15)) + assert grid.count == hours * 4 + # An hourly price list in local time covers the DST day exactly. + local_hours = [day.astimezone(UTC) + HOUR * i for i in range(hours)] + held = resample_prices(prices([float(i) for i in range(hours)], HOUR, local_hours[0]), grid) + assert None not in held.import_price.values + assert held.import_price.values[-1] == hours - 1 + + +# --- resampling ------------------------------------------------------------------------ + + +def test_power_hourly_to_quarter_hours_holds_the_mean() -> None: + grid = Grid(start=T0, count=8) + out = resample_power(ts([1000.0, 2000.0]), grid) + assert out.values == (1000.0,) * 4 + (2000.0,) * 4 + assert out.total() == ts([1000.0, 2000.0]).total() + + +def test_power_five_minutes_to_quarter_hours_averages() -> None: + src = ts([0.0, 300.0, 600.0, 900.0, 900.0, 900.0], timedelta(minutes=5)) + out = resample_power(src, Grid(start=T0, count=2)) + assert out.values == (300.0, 900.0) + + +def test_partial_coverage_and_missing_values_give_none() -> None: + src = ts([1000.0, None], MIN15, start=T0 + MIN15) + out = resample_power(src, Grid(start=T0, count=4)) + assert out.values == (None, 1000.0, None, None) + + +def test_energy_is_split_by_overlap() -> None: + src = ts([400.0], HOUR, unit=SeriesUnit.WH) + out = resample_energy(src, Grid(start=T0, count=4)) + assert out.values == (100.0,) * 4 + assert energy_to_power(out).values == (400.0,) * 4 + with pytest.raises(ValueError, match="Wh"): + resample_energy(ts([1.0]), Grid(start=T0, count=1)) + with pytest.raises(ValueError, match="Wh"): + energy_to_power(ts([1.0])) + with pytest.raises(ValueError, match="not intensive"): + resample_power(src, Grid(start=T0, count=1)) + + +def test_overlapping_or_mixed_unit_segments_are_refused() -> None: + grid = Grid(start=T0, count=4) + with pytest.raises(ValueError, match="overlap"): + resample_power([ts([1.0, 1.0]), ts([1.0], start=T0 + MIN15)], grid) + with pytest.raises(ValueError, match="same unit"): + resample_power([ts([1.0]), ts([1.0], start=T0 + HOUR, unit=SeriesUnit.PERCENT)], grid) + assert resample_power([], grid).values == (None,) * 4 + + +def test_prices_step_hold_with_mixed_15_and_60_minutes() -> None: + today = prices([0.30, 0.31, 0.32, 0.33], MIN15) + tomorrow = prices([0.20, 0.10], HOUR, start=T0 + HOUR) + out = resample_prices([tomorrow, today], Grid(start=T0, count=12)) + assert out.import_price.values == (0.30, 0.31, 0.32, 0.33) + (0.20,) * 4 + (0.10,) * 4 + assert out.export_price is None + assert out.source_id == "tibber.price" + with pytest.raises(ValueError, match="different sources"): + resample_prices( + [today, tomorrow.model_copy(update={"source_id": "x"})], Grid(start=T0, count=1) + ) + with pytest.raises(ValueError, match="no input"): + resample_prices([], Grid(start=T0, count=1)) + + +def test_export_prices_are_kept_when_every_segment_has_them() -> None: + p = PriceSeries( + source_id="nordpool", + price_includes="spot_only", + import_price=ts([0.1, -0.05], HOUR, unit=SeriesUnit.PRICE), + export_price=ts([0.05, -0.05], HOUR, unit=SeriesUnit.PRICE), + ) + out = resample_prices(p, Grid(start=T0, count=8)) + assert out.export_price is not None + assert out.export_price.values == (0.05,) * 4 + (-0.05,) * 4 + assert out.price_includes == "spot_only" + + +def test_segments_from_points_split_on_resolution_change() -> None: + points = [(T0 + MIN15 * i, float(i)) for i in range(4)] + [ + (T0 + HOUR, 10.0), + (T0 + 2 * HOUR, 20.0), + ] + segments = segments_from_points(points, SeriesUnit.PRICE) + assert [(s.step, len(s)) for s in segments] == [(MIN15, 4), (HOUR, 2)] + assert segments[-1].end == T0 + 3 * HOUR + single = segments_from_points([(T0, 1.0)], SeriesUnit.W, last_step=MIN15) + assert single[0].end == T0 + MIN15 + assert segments_from_points([], SeriesUnit.W) == () + with pytest.raises(ValueError, match="last_step"): + segments_from_points([(T0, 1.0)], SeriesUnit.W) + with pytest.raises(ValueError, match="strictly increasing"): + segments_from_points([(T0, 1.0), (T0, 2.0)], SeriesUnit.W) + + +# --- properties -------------------------------------------------------------------------- + +_power = st.floats(min_value=0, max_value=30_000, allow_nan=False) +_step_min = st.sampled_from([5, 15, 30, 60]) +_offset_q = st.integers(min_value=-8, max_value=8) + + +@settings(max_examples=150, deadline=None) +@given(values=st.lists(_power, min_size=1, max_size=96), step_min=_step_min) +def test_resampling_power_preserves_energy(values: list[float], step_min: int) -> None: + step = timedelta(minutes=step_min) + src = ts(list(values), step) + # Source is aligned to the hour, so a grid that spans it exactly covers every interval. + span = step * len(values) + count = -(-span // MIN15) + grid = Grid(start=T0, count=count) + out = resample_power(src, grid) + known = [v for v in out.values if v is not None] + covered = len(known) * MIN15 + # Only the last, partially covered quarter hour may be None. + assert len(out.values) - len(known) <= 1 + energy_out = sum(known) * MIN15 / HOUR + energy_in = sum( + v * (min(b + step, T0 + covered) - b) / HOUR + for v, (b, _, _) in zip(values, src.slots(), strict=True) + if b < T0 + covered + ) + assert energy_out == pytest.approx(energy_in, rel=1e-9, abs=1e-6) + + +@settings(max_examples=150, deadline=None) +@given( + values=st.lists(st.floats(0, 10_000, allow_nan=False), min_size=1, max_size=48), + step_min=st.sampled_from([15, 30, 60]), +) +def test_resampling_energy_preserves_energy(values: list[float], step_min: int) -> None: + src = ts(list(values), timedelta(minutes=step_min), unit=SeriesUnit.WH) + grid = Grid(start=T0, count=len(values) * step_min // 15) + out = resample_energy(src, grid) + assert out.total() == pytest.approx(src.total(), rel=1e-9, abs=1e-6) + + +@settings(max_examples=100, deadline=None) +@given( + rows=st.lists(st.tuples(_power, _power, _power).map(sorted), min_size=1, max_size=48), + step_min=_step_min, + offset=_offset_q, +) +def test_resampling_keeps_quantile_order( + rows: list[list[float]], step_min: int, offset: int +) -> None: + step = timedelta(minutes=step_min) + q = QuantileSeries( + p10=ts([r[0] for r in rows], step), + p50=ts([r[1] for r in rows], step), + p90=ts([r[2] for r in rows], step), + ) + grid = Grid(start=T0 + MIN15 * offset, count=len(rows) * 4) + out = resample_quantiles(q, grid) # the validator re-checks the order + assert len(out) == grid.count + + +@settings(max_examples=100, deadline=None) +@given( + values=st.lists(st.floats(-0.5, 1.0, allow_nan=False), min_size=1, max_size=48), + step_min=st.sampled_from([15, 60]), +) +def test_price_hold_never_invents_prices(values: list[float], step_min: int) -> None: + step = timedelta(minutes=step_min) + grid = Grid(start=T0, count=len(values) * step_min // 15) + out = resample_prices(prices(list(values), step), grid) + for i, v in enumerate(out.import_price.values): + assert v == values[(i * 15) // step_min] diff --git a/tests/core/test_tariff.py b/tests/core/test_tariff.py new file mode 100644 index 0000000..e5cd5bc --- /dev/null +++ b/tests/core/test_tariff.py @@ -0,0 +1,186 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Weekly windows, price formula, HT/NT and blended export tiers (Phase 4 WP2).""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime, time, timedelta +from zoneinfo import ZoneInfo + +import pytest +from hypothesis import given +from hypothesis import strategies as st +from pydantic import ValidationError + +from suncore.core.config import ExportTier +from suncore.core.forecast import PriceSource +from suncore.core.tariff import ( + NIGHT_WINDOW, + GridFeeWindow, + PriceFormula, + TouPeriod, + TouTariff, + TouWindow, + WeeklySchedule, + blended_export_price, + hhmm, + window_duration, +) +from suncore.core.tariff import PriceSource as ReExported + +BERLIN = ZoneInfo("Europe/Berlin") +FRI = datetime(2026, 10, 16, tzinfo=BERLIN) # a Friday + + +def at(day: datetime, hh: int, mm: int = 0) -> datetime: + return day.replace(hour=hh, minute=mm) + + +def test_price_source_is_re_exported() -> None: + assert ReExported is PriceSource + + +def test_window_same_day() -> None: + w = TouWindow(days=frozenset({4}), start="08:00", end="20:00") + assert not w.crosses_midnight + assert w.contains(at(FRI, 8)) + assert w.contains(at(FRI, 19, 59)) + assert not w.contains(at(FRI, 20)) + assert not w.contains(at(FRI, 7, 59)) + assert not w.contains(at(FRI + timedelta(days=1), 12)) + + +def test_window_crossing_midnight_belongs_to_its_start_day() -> None: + w = TouWindow(days=frozenset({4}), start="22:00", end="06:00") # Friday night + assert w.crosses_midnight + assert w.contains(at(FRI, 22)) + assert w.contains(at(FRI + timedelta(days=1), 5, 59)) # Saturday morning + assert not w.contains(at(FRI, 5)) # Friday morning belongs to Thursday's window + assert not w.contains(at(FRI + timedelta(days=1), 22)) + + +def test_whole_day_window_and_time_zone_conversion() -> None: + w = TouWindow(days=frozenset({5, 6}), start="00:00", end="00:00") + sat = FRI + timedelta(days=1) + assert w.contains(at(sat, 0)) + assert w.contains(at(sat, 23, 59)) + assert not w.contains(at(FRI, 23, 59)) + # 21:30 UTC on Friday is 23:30 in Berlin (summer time): still Friday. + assert not w.contains(datetime(2026, 10, 16, 21, 30, tzinfo=UTC), BERLIN) + assert w.contains(datetime(2026, 10, 16, 22, 30, tzinfo=UTC), BERLIN) + with pytest.raises(ValueError, match="aware"): + w.contains(datetime(2026, 1, 1)) # noqa: DTZ001 + + +@pytest.mark.parametrize( + "bad", + [ + {"start": "24:00", "end": "01:00"}, + {"start": "6:00", "end": "07:00"}, + {"start": "06:00", "end": "07:00", "days": []}, + {"start": "06:00", "end": "07:00", "days": [7]}, + ], +) +def test_window_rejects(bad: dict[str, object]) -> None: + with pytest.raises(ValidationError): + TouWindow.model_validate(bad) + + +def test_night_window_and_durations() -> None: + assert NIGHT_WINDOW.contains(at(FRI, 23)) + assert NIGHT_WINDOW.contains(at(FRI, 3)) + assert not NIGHT_WINDOW.contains(at(FRI, 12)) + assert window_duration(NIGHT_WINDOW.windows[0]) == timedelta(hours=8) + assert window_duration(TouWindow(start="05:00", end="05:00")) == timedelta(hours=24) + assert hhmm(time(6, 5, 59)) == "06:05" + with pytest.raises(ValidationError): + WeeklySchedule(windows=()) + + +def test_dst_night_is_judged_in_local_time() -> None: + # 2026-03-29: clocks jump 02:00 -> 03:00; 01:30 UTC is 03:30 local, inside 22-06. + assert NIGHT_WINDOW.contains(datetime(2026, 3, 29, 1, 30, tzinfo=UTC), BERLIN) + assert not NIGHT_WINDOW.contains(datetime(2026, 3, 29, 4, 30, tzinfo=UTC), BERLIN) + + +def test_price_formula() -> None: + f = PriceFormula(grid_fee=0.08, levies=0.03, margin=0.02, vat_pct=19) + assert f.apply(0.10) == pytest.approx((0.10 + 0.13) * 1.19) + assert f.apply(-0.20) == pytest.approx((-0.20 + 0.13) * 1.19) # negative end price + assert PriceFormula().apply(0.1) == pytest.approx(0.1) + assert PriceFormula(spot_multiplier=1.1).apply(0.1) == pytest.approx(0.11) + + +def test_price_formula_time_variable_grid_fee() -> None: + low = GridFeeWindow(window=TouWindow(start="00:00", end="06:00"), fee=0.02) + high = GridFeeWindow(window=TouWindow(start="17:00", end="20:00"), fee=0.15) + f = PriceFormula(grid_fee=0.08, grid_fee_windows=(low, high)) + assert f.grid_fee_at(None) == 0.08 + assert f.apply(0.1, at(FRI, 3)) == pytest.approx(0.12) + assert f.apply(0.1, at(FRI, 18)) == pytest.approx(0.25) + assert f.apply(0.1, at(FRI, 12)) == pytest.approx(0.18) + assert f.apply(0.1, datetime(2026, 10, 16, 1, tzinfo=UTC), BERLIN) == pytest.approx(0.12) + with pytest.raises(ValidationError): + PriceFormula(spot_multiplier=0) + with pytest.raises(ValidationError): + PriceFormula(vat_pct=-1) + + +def test_tou_tariff_first_period_wins_and_seasons() -> None: + nt = TouPeriod(window=TouWindow(start="22:00", end="06:00"), price=0.22) + winter_peak = TouPeriod( + window=TouWindow(days=frozenset(range(5)), start="17:00", end="20:00"), + price=0.40, + months=frozenset({11, 12, 1, 2}), + ) + tariff = TouTariff(base_price=0.30, periods=(nt, winter_peak)) + assert tariff.price_at(at(FRI, 23)) == 0.22 + assert tariff.price_at(at(FRI, 18)) == 0.30 # October: not winter + assert tariff.price_at(datetime(2026, 12, 4, 18, tzinfo=BERLIN)) == 0.40 + assert tariff.price_at(datetime(2026, 12, 5, 18, tzinfo=BERLIN)) == 0.30 # Saturday + with pytest.raises(ValidationError): + TouTariff(base_price=0.3, periods=()) + with pytest.raises(ValidationError): + TouPeriod(window=nt.window, price=0.2, months=frozenset({13})) + + +@dataclass(frozen=True) +class Tier: + up_to_kwp: float | None + price: float + + +def test_blended_export_price_eeg_example() -> None: + tiers = [Tier(10, 0.0786), Tier(None, 0.0680)] + assert blended_export_price(tiers, 17) == pytest.approx((10 * 0.0786 + 7 * 0.0680) / 17) + assert blended_export_price(tiers, 8) == pytest.approx(0.0786) + config_tiers = (ExportTier(up_to_kwp=10, price=0.08), ExportTier(price=0.07)) + assert blended_export_price(config_tiers, 20) == pytest.approx(0.075) + + +def test_blended_export_price_refuses_to_guess() -> None: + with pytest.raises(ValueError, match="below the plant"): + blended_export_price([Tier(10, 0.08)], 17) + with pytest.raises(ValueError, match="positive"): + blended_export_price([Tier(None, 0.08)], 0) + with pytest.raises(ValueError, match="no tiers"): + blended_export_price([], 5) + with pytest.raises(ValueError, match="ordered"): + blended_export_price([Tier(None, 0.08), Tier(10, 0.07)], 5) + with pytest.raises(ValueError, match="ordered"): + blended_export_price([Tier(10, 0.08), Tier(5, 0.07), Tier(None, 0.06)], 5) + + +@given( + bounds=st.lists(st.floats(0.5, 100), min_size=1, max_size=4, unique=True).map(sorted), + rates=st.lists(st.floats(0, 0.5), min_size=5, max_size=5), + total=st.floats(0.1, 200), +) +def test_blended_rate_lies_between_the_tier_rates( + bounds: list[float], rates: list[float], total: float +) -> None: + tiers = [Tier(b, r) for b, r in zip(bounds, rates, strict=False)] + tiers.append(Tier(None, rates[len(bounds)])) + used = [t.price for t in tiers] + blended = blended_export_price(tiers, total) + assert min(used) - 1e-12 <= blended <= max(used) + 1e-12 diff --git a/tests/safety/test_gate.py b/tests/safety/test_gate.py index 6575c09..3d0af22 100644 --- a/tests/safety/test_gate.py +++ b/tests/safety/test_gate.py @@ -21,6 +21,7 @@ from suncore.core.control import ControlSettings, DeviceState from suncore.core.decisions import ( Action, + BatteryAction, Decision, DecisionKind, Intent, @@ -1203,3 +1204,30 @@ def test_n1_a_deferred_release_is_not_re_deferred_on_every_tick() -> None: rt = h.gate.runtime_of("pool") assert rt is not None assert rt.release_pending + + +# --- battery actions (Phase 4 WP2): explicitly unsupported until WP9 --------------------- + + +@pytest.mark.parametrize("mode", [OperatingMode.SHADOW, OperatingMode.ASSISTED, AUTO]) +def test_battery_action_is_rejected_and_never_written(mode: OperatingMode) -> None: + h = make_harness(global_mode=mode, device_mode=mode) + d = asyncio.run(h.gate.submit(make_intent(BatteryAction(grid_charge_w=3000)))) + assert d.kind is DecisionKind.REJECTED + assert [r.key for r in d.reasons] == ["safety.battery_not_supported"] + assert h.apply_calls == 0 + assert h.gate.proposals() == () + + +def test_invariant_helpers_treat_battery_actions_as_no_device_change() -> None: + from suncore.safety.invariants import ( # noqa: PLC0415 + action_matches_state, + action_turns_on, + expected_power_w, + ) + + action = BatteryAction(min_soc_pct=20) + device = switch_device() + assert action_turns_on(action, device) is None + assert expected_power_w(action, device, None) is None + assert action_matches_state(action, DeviceState(on=True)) is False diff --git a/tests/storage/fixtures/export_phase3.json b/tests/storage/fixtures/export_phase3.json new file mode 100644 index 0000000..205d191 --- /dev/null +++ b/tests/storage/fixtures/export_phase3.json @@ -0,0 +1,35 @@ +{ + "suncore_export": 1, + "app_version": "0.3.0-beta.74", + "db_schema_version": 3, + "sections": { + "tariff": { + "schema_version": 2, + "body": { + "schema_version": 2, + "import": {"kind": "dynamic", "template_id": "tibber.price", "price_entity": {"entity_id": "sensor.electricity_price", "invert_sign": false}, "fixed_price": null}, + "export": {"kind": "tiered", "fixed_price": null, "tiers": [{"up_to_kwp": 10.0, "price": 0.0786}, {"up_to_kwp": null, "price": 0.068}]} + } + }, + "forecast": { + "schema_version": 1, + "body": { + "schema_version": 1, + "forecast_template_ids": ["solcast.pv"], + "forecast_entities": { + "forecast_today": {"entity_id": "sensor.solcast_pv_forecast_forecast_today", "invert_sign": false}, + "forecast_tomorrow": {"entity_id": "sensor.solcast_pv_forecast_forecast_tomorrow", "invert_sign": false} + } + } + }, + "goals": { + "schema_version": 1, + "body": { + "schema_version": 1, + "battery_target": {"soc_pct": 80.0, "time": "18:00"}, + "climate_before_export": true, + "grid_charge_night_only": true + } + } + } +} diff --git a/tests/storage/test_config_sections.py b/tests/storage/test_config_sections.py index 95a4c79..6563fa5 100644 --- a/tests/storage/test_config_sections.py +++ b/tests/storage/test_config_sections.py @@ -12,6 +12,13 @@ from hypothesis import strategies as st from pydantic import BaseModel +from suncore.core.battery import ( + BatteryModel, + BatterySettings, + BatteryStrategy, + GridChargePolicy, + RiskLevel, +) from suncore.core.config import ( SECTION_MODELS, BatteryTarget, @@ -19,6 +26,7 @@ ExportTariff, ExportTier, ForecastConfig, + ForecastSourceConfig, GoalsConfig, ImportTariff, OperationConfig, @@ -35,6 +43,7 @@ PowerModelKind, ) from suncore.core.site import EntityRef, GridConfig, PvArray, SiteConfig +from suncore.core.tariff import NIGHT_WINDOW from suncore.storage import ( ConfigCorruptError, ConfigStore, @@ -219,8 +228,14 @@ def _devices(draw: st.DrawFn) -> DevicesConfig: ), "forecast": st.builds( ForecastConfig, - forecast_template_ids=st.just(["solcast.pv"]), - forecast_entities=st.just({"today": EntityRef(entity_id="sensor.fc_today")}), + sources=st.just( + [ + ForecastSourceConfig( + template_id="solcast.pv", + entities={"today": EntityRef(entity_id="sensor.fc_today")}, + ) + ] + ), ), "devices": _devices(), "goals": st.builds( @@ -228,13 +243,29 @@ def _devices(draw: st.DrawFn) -> DevicesConfig: battery_target=st.none() | st.builds(BatteryTarget, soc_pct=_pct, time=st.sampled_from(["06:30", "23:59", "00:00"])), climate_before_export=st.booleans(), - grid_charge_night_only=st.booleans(), + grid_charge_schedule=st.sampled_from([None, NIGHT_WINDOW]), + conservative_planning=st.booleans(), ), "operation": st.builds( OperationConfig, mode=_modes, device_modes=st.dictionaries(_ids.map(lambda i: f"ghost_{i}"), _modes, max_size=3), ), + "battery": st.builds( + BatterySettings, + model=st.none() + | st.builds( + BatteryModel, + capacity_wh=st.floats(1000, 50_000), + max_charge_w=st.floats(100, 20_000), + max_discharge_w=st.floats(100, 20_000), + ), + strategy=st.sampled_from(list(BatteryStrategy)), + risk=st.sampled_from(list(RiskLevel)), + grid_charge=st.sampled_from( + [GridChargePolicy(), GridChargePolicy(enabled=True, max_price=0.12)] + ), + ), "wizard": st.builds( WizardState, completed=st.booleans(), @@ -483,17 +514,21 @@ async def test_tariff_v1_forecast_fields_move_to_the_forecast_section(db: Databa # A plain read already drops the moved fields (document migration)... tariff = await store.get("tariff") assert isinstance(tariff, TariffConfig) - assert tariff.schema_version == 2 - # ...and the start-up migration keeps them in their new section. + assert tariff.schema_version == 3 + # ...and the start-up migration keeps them in their new section (forecast v2). assert await store.migrate_legacy_forecast() is True forecast = await store.get("forecast") assert forecast == ForecastConfig( - forecast_template_ids=["solcast.pv"], - forecast_entities={"today": EntityRef(entity_id="sensor.fc_today")}, + sources=[ + ForecastSourceConfig( + template_id="solcast.pv", + entities={"today": EntityRef(entity_id="sensor.fc_today")}, + ) + ] ) row = await db.fetchone("SELECT schema_version, body FROM config_doc WHERE section='tariff'") assert row is not None - assert int(str(row[0])) == 2 + assert int(str(row[0])) == 3 assert "forecast_entities" not in json.loads(str(row[1])) assert [h.schema_version for h in await store.history("tariff")] == [1] assert await store.migrate_legacy_forecast() is False # idempotent @@ -501,7 +536,7 @@ async def test_tariff_v1_forecast_fields_move_to_the_forecast_section(db: Databa async def test_legacy_forecast_migration_keeps_an_existing_forecast_section(db: Database) -> None: store = ConfigStore(db) - mine = ForecastConfig(forecast_template_ids=["other.pv"]) + mine = ForecastConfig(sources=[ForecastSourceConfig(template_id="other.pv")]) await store.put("forecast", mine) await _store_v1_tariff(db, V1_TARIFF) assert await store.migrate_legacy_forecast() is True @@ -529,7 +564,7 @@ async def test_import_of_a_v1_tariff_export_carries_the_forecast_over(db: Databa assert await store.import_json(data) == ["forecast", "tariff"] forecast = await store.get("forecast") assert isinstance(forecast, ForecastConfig) - assert forecast.forecast_template_ids == ["solcast.pv"] + assert forecast.template_ids == ["solcast.pv"] def test_section_models_match_the_api_registry() -> None: diff --git a/tests/storage/test_phase4_migrations.py b/tests/storage/test_phase4_migrations.py new file mode 100644 index 0000000..32b45ea --- /dev/null +++ b/tests/storage/test_phase4_migrations.py @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Phase 4 document migrations from stored Phase 3 documents (tariff v2, forecast v1, goals v1).""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +import pytest + +from suncore.core.battery import BatterySettings, BatteryStrategy +from suncore.core.config import ForecastConfig, GoalsConfig, TariffConfig +from suncore.core.tariff import NIGHT_WINDOW +from suncore.storage import ConfigStore, Database +from suncore.storage.config_store import DEFAULT_SECTIONS + +EXPORT = json.loads((Path(__file__).parent / "fixtures" / "export_phase3.json").read_text("utf-8")) + + +async def _store_raw(db: Database, section: str, version: int, body: dict[str, Any]) -> None: + await db.execute( + "INSERT INTO config_doc (section, schema_version, body, updated_at) VALUES (?, ?, ?, 't')", + (section, version, json.dumps(body)), + ) + + +def test_current_versions() -> None: + versions = {name: spec.current_version for name, spec in DEFAULT_SECTIONS.items()} + assert versions["tariff"] == 3 + assert versions["forecast"] == 2 + assert versions["goals"] == 2 + assert versions["battery"] == 1 + + +@pytest.mark.parametrize("section", ["tariff", "forecast", "goals"]) +async def test_stored_phase3_documents_load(db: Database, section: str) -> None: + doc = EXPORT["sections"][section] + await _store_raw(db, section, doc["schema_version"], doc["body"]) + loaded = await ConfigStore(db).get(section) + assert loaded is not None + assert loaded.schema_version == DEFAULT_SECTIONS[section].current_version # type: ignore[attr-defined] + if isinstance(loaded, ForecastConfig): + assert loaded.template_ids == ["solcast.pv"] + assert set(loaded.sources[0].entities) == {"forecast_today", "forecast_tomorrow"} + if isinstance(loaded, GoalsConfig): + assert loaded.grid_charge_schedule == NIGHT_WINDOW + assert loaded.battery_target is not None + if isinstance(loaded, TariffConfig): + assert loaded.import_.template_id == "tibber.price" + assert loaded.import_.formula is None + + +async def test_phase3_export_imports_and_migration_never_enables_grid_charging( + db: Database, +) -> None: + store = ConfigStore(db) + assert await store.import_json(EXPORT) == ["forecast", "goals", "tariff"] + assert await store.get("battery") is None # no battery section until the user makes one + goals = await store.get("goals") + assert isinstance(goals, GoalsConfig) + assert goals.grid_charge_schedule == NIGHT_WINDOW + # Re-export is at the current versions and imports again unchanged. + exported = await store.export_json() + assert exported["sections"]["goals"]["schema_version"] == 2 + assert "grid_charge_night_only" not in exported["sections"]["goals"]["body"] + assert await ConfigStore(db).import_json(exported) == ["forecast", "goals", "tariff"] + + +async def test_battery_section_round_trips(db: Database) -> None: + store = ConfigStore(db) + settings = BatterySettings(strategy=BatteryStrategy.PREFER_FULL) + await store.put("battery", settings) + assert await store.get("battery") == settings + assert not settings.grid_charge.enabled diff --git a/tests/templates/test_battery_template.py b/tests/templates/test_battery_template.py new file mode 100644 index 0000000..6dfba17 --- /dev/null +++ b/tests/templates/test_battery_template.py @@ -0,0 +1,150 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Template kind ``battery`` and the forecast ``list_path`` (schema v1 additions, Phase 4).""" + +from __future__ import annotations + +import copy +from pathlib import Path +from typing import Any + +import pytest +from pydantic import ValidationError + +from suncore.templates import BatteryTemplate, load_templates +from suncore.templates.models import TEMPLATE_ADAPTER, TEMPLATE_KINDS, ForecastTemplate +from tests.templates.conftest import _header, forecast_doc, write_template + + +def _role(tid: str, name: str, domain: str, **matcher: object) -> dict[str, Any]: + return { + "matcher": {"domain": domain, **matcher}, + "description": f"template.{tid}.role.{name}", + } + + +def battery_doc(tid: str = "acme.battery") -> dict[str, Any]: + return { + **_header("battery", tid), + "capabilities": [ + "battery_soc", + "power_measured", + "min_soc_limit", + "charge_limit", + "discharge_limit", + "grid_charge", + "battery_modes", + ], + "roles": { + "soc": _role(tid, "soc", "sensor", device_class="battery", units=["%"]), + "power": _role(tid, "power", "sensor", device_class="power"), + "min_soc_number": _role(tid, "min_soc_number", "number"), + "max_charge_number": _role(tid, "max_charge_number", "number"), + "max_discharge_number": _role(tid, "max_discharge_number", "number"), + "mode_select": _role(tid, "mode_select", "select"), + }, + "controls": { + "min_soc_number": {"kind": "min_soc", "unit": "%", "min": 5, "max": 100, "step": 1}, + "max_charge_number": {"kind": "max_charge", "unit": "W", "min": 0, "max": 10_000}, + "max_discharge_number": {"kind": "max_discharge", "unit": "kW", "min": 0, "max": 10}, + "mode_select": { + "kind": "mode", + "options": [ + {"value": "Automatic", "meaning": "auto"}, + {"value": "Charge from grid", "meaning": "force_charge"}, + ], + }, + }, + "settle_s": 5, + } + + +def test_battery_kind_is_registered_and_parses() -> None: + assert "battery" in TEMPLATE_KINDS + template = TEMPLATE_ADAPTER.validate_python(battery_doc()) + assert isinstance(template, BatteryTemplate) + assert template.settle_s == 5 + assert "template.acme.battery.role.soc" in template.message_keys() + + +def test_loader_reads_the_battery_directory(tmp_path: Path) -> None: + write_template(tmp_path, battery_doc()) + catalog = load_templates(tmp_path) + assert [t.id for t in catalog.batteries()] == ["acme.battery"] + + +def _mutated(mutate: Any) -> dict[str, Any]: + doc = copy.deepcopy(battery_doc()) + mutate(doc) + return doc + + +@pytest.mark.parametrize( + ("mutate", "message"), + [ + (lambda d: d["controls"].update(ghost={"kind": "force_charge"}), "unknown role"), + ( + lambda d: d["controls"].update(power={"kind": "force_charge"}), + "needs a `switch` entity", + ), + ( + lambda d: d["controls"]["min_soc_number"].update(unit="W"), + "`min_soc` is set in %", + ), + (lambda d: d["controls"]["max_charge_number"].update(min=10_000), "below `max`"), + (lambda d: d["controls"]["min_soc_number"].update(max=120), "at most 100"), + ( + lambda d: d["controls"]["mode_select"]["options"][1].update(meaning="auto"), + "exactly one option", + ), + ( + lambda d: d["controls"]["mode_select"]["options"][1].update(value="Automatic"), + "unique", + ), + ( + lambda d: ( + d["roles"].update(extra=_role("acme.battery", "extra", "number")), + d["controls"].update(extra={"kind": "min_soc", "unit": "%", "min": 0, "max": 1}), + ), + "at most once", + ), + ( + lambda d: (d["controls"].pop("mode_select"), d["capabilities"].remove("grid_charge")), + "`battery_modes` declared", + ), + (lambda d: d["controls"].pop("min_soc_number"), "`min_soc_limit` declared"), + (lambda d: d["roles"]["soc"]["matcher"].pop("device_class"), "`battery_soc` declared"), + (lambda d: d.update(id="nodot", name="template.nodot.name"), "vendor"), + (lambda d: d["roles"].clear(), "at least one role"), + ], +) +def test_battery_template_rejects(mutate: Any, message: str) -> None: + with pytest.raises(ValidationError, match=message): + TEMPLATE_ADAPTER.validate_python(_mutated(mutate)) + + +def test_grid_charge_capability_accepts_a_switch_or_a_power_number() -> None: + def switch(d: dict[str, Any]) -> None: + d["controls"].pop("mode_select") + d["capabilities"].remove("battery_modes") + d["roles"]["force"] = _role("acme.battery", "force", "switch") + d["controls"]["force"] = {"kind": "force_charge"} + + assert TEMPLATE_ADAPTER.validate_python(_mutated(switch)) + + def none(d: dict[str, Any]) -> None: + d["controls"].pop("mode_select") + d["capabilities"].remove("battery_modes") + + with pytest.raises(ValidationError, match="no grid charge control"): + TEMPLATE_ADAPTER.validate_python(_mutated(none)) + + +def test_forecast_series_list_path() -> None: + doc = forecast_doc() + doc["series"]["list_path"] = "forecast.detailed" + template = TEMPLATE_ADAPTER.validate_python(doc) + assert isinstance(template, ForecastTemplate) + assert template.series.list_path == "forecast.detailed" # type: ignore[union-attr] + doc["series"]["list_path"] = "a..b" + with pytest.raises(ValidationError): + TEMPLATE_ADAPTER.validate_python(doc) diff --git a/tests/templates/test_loader.py b/tests/templates/test_loader.py index 895f8d7..84a7c98 100644 --- a/tests/templates/test_loader.py +++ b/tests/templates/test_loader.py @@ -133,10 +133,10 @@ def test_validation_message_names_the_field(tmp_path: Path) -> None: def test_unknown_kind_directory_is_an_error(builtin: Path) -> None: - battery = builtin / "battery" - battery.mkdir() - (battery / "kostal.plenticore.yaml").write_text("id: x\n", encoding="utf-8") - with pytest.raises(BuiltinTemplateError, match="unknown template kind directory `battery`"): + unknown = builtin / "inverter" + unknown.mkdir() + (unknown / "kostal.plenticore.yaml").write_text("id: x\n", encoding="utf-8") + with pytest.raises(BuiltinTemplateError, match="unknown template kind directory `inverter`"): load_templates(builtin) diff --git a/tests/templates/test_models.py b/tests/templates/test_models.py index ca5e1df..8ff87cc 100644 --- a/tests/templates/test_models.py +++ b/tests/templates/test_models.py @@ -36,7 +36,7 @@ def test_union_dispatches_on_kind(kind: str) -> None: def test_unknown_kind_rejected() -> None: doc = device_doc() - doc["kind"] = "battery" + doc["kind"] = "inverter" with pytest.raises(ValidationError, match="kind"): TEMPLATE_ADAPTER.validate_python(doc) diff --git a/tests/templates/test_schemas.py b/tests/templates/test_schemas.py index 2b86117..daba414 100644 --- a/tests/templates/test_schemas.py +++ b/tests/templates/test_schemas.py @@ -58,6 +58,7 @@ def test_schema_shape(gen: ModuleType) -> None: "tariff.schema.json", "forecast.schema.json", "region.schema.json", + "battery.schema.json", "template.schema.json", } union = json.loads(files["template.schema.json"]) From d4893c4abb34625e89a10df411d844d4462e1714 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 17 Oct 2026 02:04:51 +0000 Subject: [PATCH 06/12] docs: ADR-0011 addendum, proposed ADR-0013 and STATUS for Phase 4 PR 4-A Every new action kind is rejected by the gate until its invariants exist (ADR-0011 addendum). ADR-0013 'Battery as a controlled asset' is proposed and must be accepted before WP10. STATUS, roadmap and the plan show Phase 4 / 4-A. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/STATUS.md | 54 ++++- docs/adr/0011-two-control-layers.md | 33 +++ docs/adr/0013-battery-as-controlled-asset.md | 230 +++++++++++++++++++ docs/adr/README.md | 3 +- docs/plans/phase-4.md | 47 ++++ docs/roadmap.md | 5 +- 6 files changed, 363 insertions(+), 9 deletions(-) create mode 100644 docs/adr/0013-battery-as-controlled-asset.md diff --git a/docs/STATUS.md b/docs/STATUS.md index 8c6a4fb..2352748 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -3,12 +3,32 @@ Read this first in every session, then `CLAUDE.md` (once it exists) and the ADRs relevant to your task. Keep it short; history lives in git. -**Last update:** 2026-10-15 · **Phase:** 3 implemented (3-A `phase-3/a-safety`, 3-B -`phase-3/b-controller`, 3-C `phase-3/c-ui`), waiting for the maintainer's shadow-mode test · -**Branch:** `phase-3/c-ui` +**Last update:** 2026-10-16 · **Phase:** 4 in progress – PR 4-A contracts; Phase 3 (3-A +`phase-3/a-safety`, 3-B `phase-3/b-controller`, 3-C `phase-3/c-ui`) implemented, waiting for the +maintainer's shadow test · **Branch:** `claude/zealous-mayer-j5oda9` ## Done +- **Phase 4 – PR 4-A (plan, WP1, WP2; branch `claude/zealous-mayer-j5oda9`)**: + - Plan `docs/plans/phase-4.md` **revision 2 approved** (2026-10-16), incl. the merged concept + "Prognose, Puffer und Akkuplanung" (§0) and the split into stacked PRs 4-A…4-I. + - WP1 research: Phase 4 interfaces verified from source with pinned refs (HA core `dev` + `5cfe9aad`, Solcast `v4.6.1`, Open-Meteo Solar Forecast `0.1.33`, ENTSO-E `v0.7.5`, EPEX Spot + `4.1.0`, Octopus `v19.2.1`, Open-Meteo `01ceed5b`); summary table at the end of + `research/integrations.md`, Modbus/Kostal write path in `research/kostal-plenticore-modbus.md`. + - WP2 contracts (`core`, additive): `series.py` (`TimeSeries`, `QuantileSeries`, `PriceSeries`, + 15-min grid resampling, DST-safe), `forecast.py` (`ForecastSource`/`PriceSource`, + `AccuracyRecord`), `tariff.py` (`WeeklySchedule`, `PriceFormula` with VAT, HT/NT, blended + export tiers), `battery.py` (`BatteryModel`, `BatteryStrategy`, `RiskLevel`, + `GridChargePolicy`, `BatterySettings` = section `battery`), `BatteryAction` in `decisions.py` + (**rejected by the gate** with `safety.battery_not_supported`, ADR-0011 addendum), `Plan` v2 + (`IntervalFlows`, `IntervalPrices`, per-interval reasons, `source`, `SolverStats`, + `BatteryBand.priority_until_soc_pct`; v1 still loads); config `TariffConfig` v3, + `ForecastConfig` v2, `goals` v2; template schema v1 additions (kind `battery`, `series`, + HT/NT). + - ADR-0013 "Battery as a controlled asset" **proposed** (prerequisite of WP10). + - Review fixes: in progress (`fix(...)` commits on the branch). + - **Fix after the first live test (2026-10-16, branch `fix/device-freshness`)**: on the maintainer's HA the SunSpec battery SoC (84 % for 25 min) and the `smartthinq_sensors` power sensors (5 W for a day) never advance `last_reported` for an unchanged value, so SunCore marked them "stale" and the @@ -280,6 +300,18 @@ ADRs relevant to your task. Keep it short; history lives in git. - Beta version derivation in `build.yml` and the E2E/screenshot work were done in PR 2-D, see Open – Phase 1. +## Open – Phase 4 + +- PR order (stacked, each on the previous, `phase-4/-`): 4-B storage/time series + (WP3) → 4-C tariffs (WP4) and 4-D PV forecast (WP5) → 4-E load + risk (WP6, WP7) → 4-F planner + + runtime (WP8, WP9) → 4-G battery control (WP10) → 4-H frontend + docs (WP11, WP13); 4-I ML load + (WP12, optional, after 4-E). Merge only after the Phase 3 acceptance. +- **ADR-0013 must be accepted before WP10** (4-G). Its contract changes (`BatteryAction` `None` = + baseline, `BatteryBand.grid_charge_max_w`/`discharge_lock`, `effective_reserve_pct`, + `battery_identity`) land in a WP2 fix before 4-B. +- Maintainer questions from 4-A: plan §6 "Offene Fragen aus PR 4-A" (grid-charge schedule + precedence, VAT on negative spot prices, export tiers, grid charging on the Kostal). + ## Open – Phase 3 - **WP5/WP6 (branch `phase-3/b-controller`)** done: `suncore/devices/power/` (measured, @@ -454,7 +486,8 @@ ADRs relevant to your task. Keep it short; history lives in git. the first live writes (the climate unit's counter-only estimate and the cloud echo delay are the interesting parts). Then merge 3-A, 3-B, 3-C and release. 4. Release the next release-please PR for the stable app after acceptance. -5. Phase 4 (forecasts, tariffs, planner, battery) starts after the maintainer's acceptance. +5. Phase 4: finish the 4-A review fixes, open PR 4-A; maintainer decides on ADR-0013 and the 4-A + questions; continue with 4-B (storage/time series). ## Known problems and `UNVERIFIED` items @@ -462,15 +495,22 @@ ADRs relevant to your task. Keep it short; history lives in git. |---|---|---| | Kostal §3.4 set-point timeout and how external battery management is enabled (register map itself is verified from KOSTAL rev 2.9) | `research/kostal-plenticore-modbus.md` | Phase 4 measurement on the maintainer's inverter | | Hosted Weblate free tier for libre projects | ADR-0007 | maintainer confirms on weblate.org | -| Open-Meteo tilted-irradiance parameters; Energy-Charts and aWATTar APIs and terms | `research/integrations.md` | Phase 4, session with network access | -| Forecast.Solar / Open-Meteo Solar Forecast attribute names; Octopus attributes | `research/integrations.md` | Phase 4, read integration source | +| Energy-Charts and aWATTar APIs and terms (not shipped in Phase 4) | `research/integrations.md` | session with network access (Phase 6) | +| Open-Meteo azimuth convention in a live response (code: 0 = south, OpenAPI text contradicts); separate terms page (README only) | `research/integrations.md` | live request; maintainer reads the terms page | +| Kostal REST battery/grid power sign; `ActivePowerLimitation` semantics; live entity ids | `research/integrations.md` | maintainer's HA (enable the entities, compare with the inverter display) | +| Kostal Modbus external battery management not enabled on the maintainer's inverter; how it is enabled; §3.5 fallback timer behaviour | `research/kostal-plenticore-modbus.md`, ADR-0013 | maintainer enables it and measures; `kostal_modbus` capped at assisted until then | +| Creating a modbus hub from the HA UI (else YAML) | `research/ha-websocket-api.md` | maintainer's HA | +| Supervisor proxy passes `recorder/*` and `energy/*` commands | `research/ha-websocket-api.md` | first Phase 4 run on HA OS | +| Publication time of next-day prices (Tibber, Nord Pool, ENTSO-E, EPEX) | `research/integrations.md` | observe on the maintainer's HA | +| VAT applied to negative spot prices | `suncore/core/tariff.py` | maintainer's bill / supplier terms | +| KNN load model performance on aarch64 | plan WP12 | CI arm runner benchmark | | Energy colour variable names (`--energy-*`) | `frontend/src/app.css` | read HA frontend source | | HA UI menu paths in the user docs (Settings → Apps → App store, Repositories, Show in sidebar; German labels) | `addon/*/DOCS.md`, `README.md`, `docs/user/*/quick-start.md` | maintainer checks on HA OS during the beta install | | Supervisor Core proxy passes `supported_features` through | `suncore/ha/websocket.py` | first run on HA OS | | Registry and energy-prefs shapes on cores older than dev (single `config_entry_id` on devices, unified grid source); `unknown_command` code without the `energy` integration; Supervisor proxy passes registry commands | `research/ha-websocket-api.md` | first run on HA OS; parsers tolerate both shapes | | Section 51 EEG 2023 text could not be fetched; region `de` feed-in tiers are empty until the official BNetzA table is cited | `templates/region/de.yaml` | maintainer provides the table or a session with access to gesetze-im-internet.de | | Whether Tibber's `total` includes grid fees (country dependent) | `templates/tariff/tibber.price.yaml` | live check on the maintainer's Tibber account | -| LG translation keys (inferred from source and unique ids; LG entity ids, modes and units checked live on 2026-10-14), other LG models' modes, Tibber tomorrow-price publication time, Solcast default day-sensor count | `research/device-integrations.md` | maintainer's HA with LG ThinQ, Tibber, Solcast (Phase 2 wizard test) | +| LG translation keys (inferred from source and unique ids; LG entity ids, modes and units checked live on 2026-10-14), other LG models' modes (Solcast default day-sensor count verified from source in 4-A; Tibber publication time: see the price row above) | `research/device-integrations.md` | maintainer's HA with LG ThinQ, Tibber, Solcast (Phase 2 wizard test) | | Wording of the quick start steps and the HA menu names in the new docs (written from the UI catalogues, not clicked through on a real HA); the surplus control guide is written from the code and the demo, its walk-through has not been followed on a real system | `docs/user/*/quick-start.md`, `docs/user/*/surplus-control.md` | maintainer follows the quick start and the first-device walk-through on HA OS | | Which Home Assistant tool lets users place files in the app's `/data/templates` | `docs/user/*/device-templates.md` | maintainer checks on HA OS | | Water heater templates (`generic.water_heater_steps`, `_relays`), drivers for select/number/relays/mode, `ClimateDriver` paths other than `lg_thinq` source reading: simulator only, no hardware; `lg_thinq` `step is None` `TypeError` on `set_temperature`, push latency after a command, `climate.turn_on` fallback for modes not in `hvac_modes` | `templates/device/generic.water_heater_*.yaml`, `suncore/devices/drivers/`, `research/device-integrations.md` | first limited live write on the maintainer's AC; a user with a water heater | diff --git a/docs/adr/0011-two-control-layers.md b/docs/adr/0011-two-control-layers.md index 9362315..47c1b72 100644 --- a/docs/adr/0011-two-control-layers.md +++ b/docs/adr/0011-two-control-layers.md @@ -68,3 +68,36 @@ Option A. Accept. Note the sign convention (battery positive = charging) – it is the opposite of the Kostal HA integration, conversion happens in the adapter. + +## Addendum 2026-10-16: new action kinds are rejected until their invariants exist + +Context: the `Action` union in `core/decisions.py` is part of the stable +contract spine and grows additively. A new variant can appear in the +contract (so that planner, controller and storage can be built against it) +before the safety layer knows how to judge it. Without a rule, a new kind +could fall through the gate's dispatch and reach `apply()` without any +invariant having been checked. + +Decision: + +- Every new action kind is **rejected by the safety gate** with its own + reason key `safety._not_supported` (catalogued en/de like every + `safety.*` key) until its invariants and their tests exist. The rejection + happens before mode handling, so it also applies in shadow and assisted + mode; nothing is written and no proposal is created. +- The invariant functions in `safety/invariants.py` handle the new kind + explicitly (no catch-all that silently passes it). +- Removing the rejection requires: the kind's invariants in + `safety/invariants.py` with property tests, simulator scenarios for its + fail-safe paths, and a documented **`control-safety` sign-off** in the PR + (invariant table). Any ADR that defines the kind's semantics must be + accepted first. +- First case: `BatteryAction` (Phase 4, PR 4-A), rejected with + `safety.battery_not_supported`. Its semantics are the subject of + ADR-0013 ("Battery as a controlled asset"), which must be accepted before + Phase 4 WP10 lifts the rejection. + +Consequence: contracts can be published ahead of their write path without +weakening the single-writer guarantee; a boundary/unit test asserts that +every `ActionKind` is either handled by the invariants or rejected with its +`not_supported` reason. diff --git a/docs/adr/0013-battery-as-controlled-asset.md b/docs/adr/0013-battery-as-controlled-asset.md new file mode 100644 index 0000000..04b47e4 --- /dev/null +++ b/docs/adr/0013-battery-as-controlled-asset.md @@ -0,0 +1,230 @@ +# ADR-0013: Battery as a controlled asset + +- **Status:** proposed +- **Date:** 2026-10-16 +- **Deciders:** maintainer, lead engineer (`architect`), `control-safety` +- **Research:** [`integrations.md`](../research/integrations.md) (Kostal Plenticore, summary + table), [`kostal-plenticore-modbus.md`](../research/kostal-plenticore-modbus.md) (§3.4, §3.5, + "Phase 4 verification"), [`ha-websocket-api.md`](../research/ha-websocket-api.md) ("Modbus + write action"); plan [`phase-4.md`](../plans/phase-4.md) WP2, WP8, WP10, §6 questions 3, 6, 7 + +## Context + +Phase 4 makes the battery the first asset SunCore writes to that can **cause grid import on +purpose** (planned grid charging) and that **guards energy the user relies on** (the backup +reserve). PR 4-A published `BatteryAction` in `core/decisions.py` (fields `max_charge_w`, +`max_discharge_w`, `min_soc_pct`, `grid_charge_w`, magnitudes, at least one set) and the +`battery` config section in `core/battery.py` (`BatteryModel` with `reserve_pct` + +`bms_drift_margin_pct`, `GridChargePolicy`, strategies). The gate rejects every +`BatteryAction` with `safety.battery_not_supported` (ADR-0011 addendum 2026-10-16). Before WP10 +removes that rejection, the semantics below must be fixed, because they decide what the +invariants are. + +Facts that constrain the design (verified unless marked): + +- **Kostal Modbus §3.5** (G3 ≥ SW 03.05): limits only – 1280/1282 max charge/discharge (W + float), 1284/1286 fallback limits, 1288 time until fallback 30–10800 s. Once 1280/1282 are + written they must be re-written cyclically, otherwise the fallback limits take over: a + documented hardware watchdog. §3.4 (DC set-point 1034, negative = charge) has no documented + timeout and is out of scope for Phase 4 (plan §7). +- **Writing Modbus through HA:** `modbus.write_register` writes raw unsigned 16-bit words + (floats must be encoded into two words, word order from register 5), defaults `slave` to 1, + has no read action and **only logs a failed write** – read-back through modbus sensor + entities is mandatory. Creating a modbus hub from the HA UI is `UNVERIFIED` (YAML may be + required). External battery management via Modbus (register 1080 = `0x02`) is **not enabled** + on the maintainer's inverter; how it is enabled is `UNVERIFIED`. +- **`kostal_plenticore` (REST):** `number` `Battery:MinSoc` (5–100 %, step 5; live `min`/`max` + from the inverter), `Battery:MinHomeComsumption`, `Inverter:ActivePowerLimitation`; **all + disabled by default**; a failed write is **silent** (`async_write_data` swallows + `ApiException`, `number.set_value` succeeds) – read-back after the forced refresh is + mandatory. No charge/discharge power set-point over REST. Sign of `Battery Power`/`Grid + Power` and the meaning of `ActivePowerLimitation` are `UNVERIFIED`. +- **Sign conventions:** SunCore core: battery + = charging (ADR-0011). Kostal REST battery power + is believed to be + = discharge (`UNVERIFIED`); §3.4 1034 and MDC 1549 negative = charge. +- **Reserve today exists twice:** `site.battery.min_soc_pct` (Phase 1/3, read by the controller + and the gate through `GateInputs.reserve_soc_pct`) and `BatteryModel.reserve_pct + + bms_drift_margin_pct` (Phase 4 planning floor). Two values for one concept will drift. +- **Phase 3 invariant "no SunCore-caused import"** holds for devices. Grid charging violates it + by design, and so does a discharge lock (`max_discharge_w = 0` makes the house import what + the battery would have covered). +- **Identity:** the device identity rule (`core/identity.py`: id + template + entities) and the + mode store (`operation` section) cover `DeviceConfig`s only. The battery is configured in the + `battery` section and has no identity token, so re-mapping the battery would not reset its + mode to shadow today. +- Budgets: heartbeat and read-back are a few service calls per minute; no effect on the 300 MB + RSS or the planner's 10 s budget. + +## Options considered + +### 1. Action shape + +- **A – limits** (the 4-A `BatteryAction`): caps on the inverter's own self-consumption loop, + a discharge floor, and an explicit forced grid-charge power. Pros: maps 1:1 onto §3.5 + (1280/1282), `Battery:MinSoc` and generic number entities; when SunCore stops, the inverter's + own control is the natural fallback; PV charging needs no write at all. Cons: cannot + express "discharge exactly 2 kW for export"; forced grid charging needs an adapter capability + that §3.5 and REST do not offer. +- **B – power set-point** (§3.4 style, battery W per interval). Pros: full control, what the LP + computes. Cons: SunCore becomes the battery's real-time controller (must track house load + every second or cause import/export); no documented timeout on Kostal; failure mode is a + battery stuck at a set-point. +- **C – both, set-point as opt-in.** Pros: future-proof. Cons: two invariant sets now for a path + nobody can test on hardware in Phase 4. + +### 2. Missing fields in an action + +- **A – delta** (4-A docstring: "a missing field leaves that limit as it is"). Cons: the full + state SunCore holds is spread over several decisions; heartbeat, read-back and release cannot + be judged from one decision. +- **B – full desired state:** `None` = "no SunCore restriction on this field", i.e. the + adapter's baseline value. Pros: each accepted action is the complete state; the heartbeat + re-sends it idempotently; read-back compares one record. + +### 3. Reserve + +- **A – each layer keeps its own value.** Cons: the planner and the gate can disagree; the gate + could accept a floor the planner considers below reserve. +- **B – one function in `core`** used by planner, controller and gate. + +### 4. Grid import caused by the battery + +- **A – forbid** (Phase 3 rule unchanged). Cons: no grid charging and no discharge lock – both + are approved Phase 4 features (questions 7, 3). +- **B – a bounded, explicit exception:** allowed only when the current plan interval asks for + it and every live bound holds at execution time. +- **C – trust the planner.** Cons: the gate would no longer judge; a stale or wrong plan writes. + +### 5. Heartbeat and release + +- **A – the controller re-issues the intent every N s** through the gate. Cons: every heartbeat + is a decision row and fights "write only on change" and the switch/write-rate invariants. +- **B – lease held by the gate, heartbeat in the actuator:** an accepted `BatteryAction` creates + a lease; `gate.poll()` (every tick) renews it while the asset is still automatic (or the + assisted proposal is confirmed), inputs are fresh, no trip/emergency stop/shutdown is active + and the plan still covers the action; the actuator re-writes the accepted state at its + heartbeat cadence while the lease is valid and stops otherwise. Hardware fallback (§3.5) + catches a dead SunCore. +- **C – no heartbeat, rely on the hardware fallback only.** Cons: the limit drops out every + 1288 seconds; only works with §3.5. + +### 6. Identity and mode + +- **A – battery as a `DeviceConfig`.** Cons: battery fields do not fit the device model; the + template kind `battery` already exists separately. +- **B – reserved asset id `battery`** with its own identity token from the `battery` section, + stored and enforced through the existing `operation` / `mode_identities` machinery. + +## Decision + +Proposed: **1A, 2B, 3B, 4B, 5B, 6B.** + +1. **Action shape – limits.** `BatteryAction` stays a set of limits. Powers are magnitudes + ≥ 0; core sign convention (battery + = charging) applies to every *measured* value; each + adapter converts its integration's signs (Kostal REST battery/grid power, §3.4) and the + wizard's sign check confirms them live. §3.4 set-points are not decided here (later ADR). + Every adapter declares **capabilities** (`charge_limit`, `discharge_limit`, `min_soc`, + `grid_charge`, `hardware_fallback`); the gate rejects an action using a field the adapter + cannot execute (`safety.battery_capability_missing`). Phase 4 adapters: `kostal_modbus` + (§3.5: charge/discharge limit, hardware fallback; no `grid_charge`), `kostal_plenticore` + (`min_soc` only; step 5, so the adapter rounds **up**), `generic` (per template roles). +2. **Full desired state.** A `BatteryAction` is the complete state SunCore wants; `None` = + baseline value of that field (captured at takeover). Changes the 4-A docstring (contract + change covered by this ADR). +3. **One reserve.** `core.battery.effective_reserve_pct(site_battery, model) = + max(site.battery.min_soc_pct or 0, model.reserve_pct + model.bms_drift_margin_pct)`. The + planner's SoC floor, `GateInputs.reserve_soc_pct` and the controller's back-off use only this + function. Without a `BatteryModel` the battery is observe-only (gate rejects with + `safety.battery_model_missing`). +4. **Bounded exception to "no SunCore-caused import"**, for the battery only. A battery action + that can cause import – `grid_charge_w > 0`, or `max_discharge_w` below the current house + deficit (discharge lock), or `min_soc_pct` at/above the live SoC – is accepted only if **all** + hold at execution time: + - the plan interval covering *now* explicitly allows it (new additive `BatteryBand` fields + `grid_charge_max_w: float | None` and `discharge_lock: bool = False`; plan v2 contract + change covered by this ADR) and the plan is fresh (created ≤ 2 intervals ago); + `Plan.default()` allows neither; + - for grid charging: `effective_grid_charge_policy(...).allows(now, live import price)` with + the **live** price (not the planned one; an unknown price never satisfies a price level), + and the live SoC is below `min(band.max_soc_pct, model.max_soc_pct)`; + - `grid_charge_w ≤ min(model.max_charge_w, adapter/inverter limit, band.grid_charge_max_w, + import headroom)` where import headroom = `grid_import_limit_w` − current import without + the battery − `ControlSettings.reserve_w`; the site import limit holds **always**, planned + or not. + Devices keep the Phase 3 rule unchanged; a device may never use import created for the + battery (the free surplus excludes it). +5. **Lease and heartbeat (5B).** The actuator owns the heartbeat (e.g. §3.5: 1288 = 60 s, + re-write every 20 s, fallback registers 1284/1286 = the battery's own limits from 1076/1078, + written and read back **before** the first limit write, otherwise refuse). Heartbeat writes + are the accepted state, exempt from write-on-change but bounded by the heartbeat rate and the + per-(domain, service) rate limiter. **Release** = stop the heartbeat and write the baseline + (§3.5: own limits; `kostal_plenticore`: the `MinSoc` value captured at takeover), then + read back. Release paths are the Phase 3 ones (emergency stop, shadow, shutdown, site/device + trip, reconnect, crash recovery via `recover()`). An adapter without `hardware_fallback` + (`kostal_plenticore`, most `generic`) relies on `recover()` and the next start: documented + as a risk; such adapters may apply a discharge lock only for the planned interval, never open + ended. + **Read-back is mandatory** for every battery write (both Kostal paths fail silently); a + mismatch is `failed`, three failures trip the battery (Phase 3 rule). For `kostal_modbus` + the adapter also reads register 1080 through a modbus sensor and refuses to write unless it + is `0x02`. +6. **Identity and mode.** The battery is the asset id `battery` (one battery per site in + Phase 4). `core/identity.py` gains `battery_identity(settings, site)` = asset id + adapter + template + mapped role entities + the site's SoC and battery power entities. The `operation` + section stores its mode like a device's; every rule of the device identity applies (changed + identity → shadow; import → shadow; raising the mode needs the identity token). The battery + starts in shadow. The gate caps the effective mode at **assisted** for an adapter whose + hardware fallback is not marked verified (`kostal_modbus` until the maintainer's timer + measurement – plan §6 question 3). + +**Invariants to add in `safety/invariants.py`** (pure, property-tested; reason keys +`safety.battery_*`): + +| # | Invariant | +|---|---| +| B1 | `min_soc_pct` written (after rounding to the device step, rounded up) is never below `effective_reserve_pct` | +| B2 | `grid_charge_w ≤ min(model.max_charge_w, adapter limit, band.grid_charge_max_w, import headroom)`; 0 when any bound is unknown | +| B3 | `grid_charge_w > 0` only with a fresh plan interval allowing it, the live policy check, SoC below max | +| B4 | a discharge lock (`max_discharge_w` below the house deficit, or `min_soc_pct` ≥ live SoC) only when the plan interval has `discharge_lock`; otherwise `safety.battery_unplanned_import` | +| B5 | the site import limit holds for every battery action | +| B6 | every value inside the adapter's live range (e.g. `min`/`max` attributes of the Kostal numbers) | +| B7 | write rate, write-on-change; heartbeat only while the lease is valid | +| B8 | stale SoC/battery power while the grid meter moves, or a hung inverter → release and stop writing | +| B9 | capability check: no field the adapter cannot execute | + +**Not decided here:** §3.4 set-points and MDC; export limiting through +`ActivePowerLimitation` / register 533 (Phase 6); several batteries; whether forced grid +charging is possible on the maintainer's Kostal without §3.4 (see consequences). + +## Consequences + +- Positive: the safety gate can judge every battery write from one decision plus live inputs; + a crashed SunCore leaves a §3.5 inverter in its own control within 60 s; the reserve cannot + diverge between planner and gate; re-mapping the battery drops it to shadow. +- Negative / risks: + - **Grid charging is not executable on the maintainer's system in Phase 4:** §3.5 only + limits and REST has no charge set-point; the planner may still plan it and shadow/assisted + shows "would charge". Mitigation: a `generic` adapter with a force-charge role, or a later + §3.4 ADR. Needs the maintainer's input. + - `kostal_plenticore` has no hardware fallback; a discharge lock via `MinSoc` survives a + SunCore crash until the next start. Mitigation: plan-bounded locks only, `recover()` on + start, documented. + - Required entities disabled by default (Kostal numbers) or not creatable from the UI + (modbus hub, `UNVERIFIED`): the wizard must check and explain. +- Contract changes (additive, ADR-0011 spine): `BatteryAction` `None` semantics (docstring and + tests), `BatteryBand.grid_charge_max_w`, `BatteryBand.discharge_lock`, + `core.battery.effective_reserve_pct`, `core.identity.battery_identity`. +- Follow-ups: + - Phase 4 WP2 fix (`optimizer-engineer`): the contract changes above, before PR 4-B. + - WP8 (`optimizer-engineer`): planner floor = `effective_reserve_pct`; emits the new band + fields only where the policy allows. + - WP10 (`control-safety`): invariants B1–B9, lease/heartbeat in the gate, mode cap, identity; + removal of `safety.battery_not_supported` with sign-off (ADR-0011 addendum). + - WP10 (`device-adapters`): adapters with capabilities, sign conversion, read-back, + fallback registers, 1080 check. + - WP13 (`docs-writer`): enabling the Kostal numbers, modbus hub setup, the no-fallback risk + of the REST adapter. + +## Recommendation to the maintainer + +Accept as proposed; your input is needed on how grid charging should be executed on your +Kostal in Phase 4 (not at all, a generic force-charge entity, or a later §3.4 ADR). diff --git a/docs/adr/README.md b/docs/adr/README.md index 2a2b586..6e7cd90 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -16,5 +16,6 @@ the status line; a new ADR supersedes an old one. Template: | [0008](0008-documentation-tooling.md) | Documentation tooling | accepted | MkDocs Material + static-i18n, GitHub Pages | | [0009](0009-release-and-versioning.md) | Release and versioning | accepted | release-please, SemVer, stable + beta app, GHCR | | [0010](0010-target-platform.md) | Target platform | accepted | HA OS app on aarch64/amd64, HA connection abstracted | -| [0011](0011-two-control-layers.md) | Two control layers and safety boundary | accepted | Planner → Controller → Safety (single writer) → Actuators | +| [0011](0011-two-control-layers.md) | Two control layers and safety boundary | accepted (addendum 2026-10-16: new action kinds rejected until their invariants exist) | Planner → Controller → Safety (single writer) → Actuators | | [0012](0012-template-schema-v1.md) | Template schema v1 – review before freeze | accepted | Required changes (matcher regex/lists, same-device role, `ControlSpec`, series source, capability enum) before freeze; `battery` kind added later under the v1 compatibility policy | +| [0013](0013-battery-as-controlled-asset.md) | Battery as a controlled asset | proposed | `BatteryAction` as limits (not set-points); one reserve source of truth; planned grid charging as the only bounded exception to "no SunCore-caused import"; adapter heartbeat with hardware fallback; battery identity in the fingerprint; prerequisite of Phase 4 WP10 | diff --git a/docs/plans/phase-4.md b/docs/plans/phase-4.md index 9ab9f38..0fed4f4 100644 --- a/docs/plans/phase-4.md +++ b/docs/plans/phase-4.md @@ -211,6 +211,24 @@ profile), `GET /api/v1/prices`; live feed `plan`. Demo with simulator prices and ### WP10 – Battery control · `control-safety` (Opus), adapters by `device-adapters` +**Prerequisite:** [ADR-0013 "Battery as a controlled asset"](../adr/0013-battery-as-controlled-asset.md) +is **accepted before WP10 starts** (proposed in PR 4-A). Its scope: + +- action shape (limits vs set-points), sign convention and the adapters' conversion; +- one source of truth for the reserve, `max(site battery min_soc, BatteryModel reserve + + BMS-drift margin)`, used by planner and gate; +- the explicit exception to "no SunCore-caused import" for planned grid charging and its bounds + (import limit, price level/schedule, a plan interval allowing it); +- heartbeat/fallback-timer and release semantics; +- battery identity and mode rules (the battery is not covered by the device identity fingerprint + today); +- invariants to add: `min_soc` never below the reserve; `grid_charge_w` bounded by battery, + inverter and import limits; `max_discharge_w = 0` must not cause grid import against the + device rule unless planned. + +Until then the gate rejects every `BatteryAction` with `safety.battery_not_supported` (ADR-0011 +addendum 2026-10-16); removing it needs the `control-safety` sign-off. + - Controller: `BatteryBand` → `BatteryAction` intents (on change, hysteresis); evaluates `priority_until_soc_pct` against the live SoC; device bands from the plan. - Safety invariants (property-tested): SunCore never discharges below reserve + drift margin; @@ -357,6 +375,35 @@ shadow and assisted run is the acceptance test. write at all (the inverter's own self-consumption logic does it). - Self-sufficiency weight: explained to the maintainer, default 0.01 currency/kWh kept. +### Offene Fragen aus PR 4-A (an den Maintainer) + +1. **Doppelter Netzladeplan:** `goals.grid_charge_night_only` wird zu einem Zeitplan migriert, + und `battery.grid_charge` hat einen eigenen Zeitplan. Aufgelöst durch + `effective_grid_charge_policy`: die Akku-Richtlinie (`battery.grid_charge`) gewinnt, der + Zeitplan aus `goals` gilt nur, solange die Akku-Richtlinie keinen eigenen setzt. Einverstanden? +2. **Mehrwertsteuer auf negative Spotpreise:** Die Preisformel wendet die MwSt. auf die ganze + Summe an, auch wenn der Spotpreis negativ ist (Endpreis kann dadurch negativ werden). Das ist + eine Annahme (`UNVERIFIED`); bitte mit einer Tibber- oder Lieferantenrechnung bestätigen. +3. **Einspeise-Stufen:** Stufen, die die Anlagengröße nicht vollständig abdecken (z. B. nur + „bis 10 kWp“ bei 17 kWp), werden abgelehnt – SunCore rät keinen Satz. Bitte beide EEG-Sätze + inklusive der Stufe über 10 kWp eintragen (Frage 8). +4. **Netzladen auf deinem Kostal (ADR-0013):** §3.5 begrenzt nur, die REST-API hat keinen + Lade-Sollwert. Netzladen wäre in Phase 4 nur geplant und im Schatten-/Assistenzmodus sichtbar, + nicht ausführbar. Reicht das, oder soll eine generische „Zwangsladen“-Entität bzw. später ein + eigener §3.4-ADR kommen? + +## Deviations during implementation + +- Phase 4 is split into the stacked PRs 4-A…4-I (§3) instead of one PR. +- The built-in price fallbacks (Energy-Charts, aWATTar) are **not shipped in Phase 4**: their APIs + and terms could not be read from the session (question 5 default: only if verifiable, else + Phase 6). +- Open-Meteo API terms are known only from the repository README (free for open-source and + non-commercial use, fair use, 10 000 requests/day, CC BY 4.0 attribution); the separate terms + page was unreachable. The built-in PV fallback stays opt-in and off. +- `BatteryAction` is part of the 4-A contracts but rejected by the gate until WP10 + (ADR-0011 addendum); its semantics are proposed in ADR-0013. + ## 7. Out of scope for Phase 4 Full plan chart with device Gantt, zoom/pan, history, savings vs baselines, what-if and diff --git a/docs/roadmap.md b/docs/roadmap.md index 80c25d9..d451c7d 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -75,7 +75,10 @@ Exit: climate surplus control runs in shadow mode on the maintainer's system with plausible "would have" entries; maintainer switches one device to automatic. ⏸ waiting for the maintainer (shadow run on HA OS, then one device to automatic). -## Phase 4 – Forecasts, tariffs, planner, battery ⬜ +## Phase 4 – Forecasts, tariffs, planner, battery 🟨 + +In progress in stacked PRs 4-A…4-I ([`plans/phase-4.md`](plans/phase-4.md)); PR 4-A +(contracts) first. Battery control (WP10) waits for ADR-0013. | Milestone | Acceptance | |---|---| From 595edbab13de25a66d786d73095728f5d2c4c539 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 17 Oct 2026 02:04:51 +0000 Subject: [PATCH 07/12] fix(core): address the review of the Phase 4 contracts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Gaps in point series become None instead of invented values; the leftover P10 semantics give way to the risk quantile (conservative_planning removed); forecast form keeps entities per source; tariff templates accept currency/MWh and mapping series; effective_grid_charge_policy fixes the schedule precedence; §14a fee windows per quarter; season of a window crossing midnight follows its start day; tz is required; a formula on an all-in source is rejected; BatteryAction rejects grid charging above its charge cap. Safety tests: battery decisions round-trip, every driver refuses a BatteryAction, the gate rejects it for every origin, mode and emergency-stop state. E2E reset helper uses goals v2. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- frontend/e2e/helpers.ts | 2 +- frontend/src/lib/api/openapi.json | 77 ++++++++++----- frontend/src/lib/api/schema.d.ts | 59 ++++++++--- frontend/src/lib/i18n/locales/de.json | 4 + frontend/src/lib/i18n/locales/en.json | 4 + frontend/src/lib/views/steps/PvStep.svelte | 6 +- .../src/lib/views/steps/TariffStep.svelte | 3 + frontend/src/lib/wizard/forecastForm.ts | 51 +++++----- frontend/src/lib/wizard/forms.test.ts | 57 ++++++++++- frontend/src/lib/wizard/goalsForm.ts | 9 +- suncore/api/app.py | 31 +++++- suncore/api/validation.py | 3 + suncore/core/battery.py | 21 ++-- suncore/core/config.py | 52 +++++++++- suncore/core/control.py | 23 ++++- suncore/core/decisions.py | 28 +++++- suncore/core/forecast.py | 7 +- suncore/core/series.py | 90 ++++++++++++----- suncore/core/tariff.py | 77 ++++++++++----- suncore/safety/gate.py | 4 +- suncore/safety/invariants.py | 6 +- suncore/storage/config_store.py | 21 +++- suncore/templates/models.py | 98 +++++++++++++++---- templates/README.md | 2 +- templates/schema/forecast.schema.json | 47 ++++++--- templates/schema/tariff.schema.json | 49 +++++++--- templates/schema/template.schema.json | 49 +++++++--- tests/api/test_config_api.py | 39 ++++++++ tests/core/test_battery.py | 27 +++-- tests/core/test_config_phase4.py | 85 +++++++++++++++- tests/core/test_config_sections.py | 1 - tests/core/test_decisions.py | 54 ++++++++++ tests/core/test_plan_v2.py | 10 +- tests/core/test_series.py | 84 ++++++++++++++-- tests/core/test_tariff.py | 96 ++++++++++++------ tests/devices/drivers/test_battery_action.py | 57 +++++++++++ tests/safety/test_gate.py | 63 +++++++++++- tests/storage/test_config_sections.py | 1 - tests/storage/test_phase4_migrations.py | 43 +++++++- tests/templates/test_battery_template.py | 18 +++- tests/templates/test_models.py | 41 ++++++++ 41 files changed, 1232 insertions(+), 267 deletions(-) create mode 100644 tests/devices/drivers/test_battery_action.py diff --git a/frontend/e2e/helpers.ts b/frontend/e2e/helpers.ts index 4952a11..d5ae5a4 100644 --- a/frontend/e2e/helpers.ts +++ b/frontend/e2e/helpers.ts @@ -31,7 +31,7 @@ export async function resetDemo( expect(res.ok(), `import: ${res.status()}`).toBe(true); if (opts.wizardDone) return; await put(request, 'devices', { schema_version: 1, devices: [] }); - await put(request, 'goals', { schema_version: 1 }); + await put(request, 'goals', { schema_version: 2 }); await put(request, 'operation', { schema_version: 1, mode: 'shadow', device_modes: {} }); await put(request, 'wizard', { schema_version: 1, diff --git a/frontend/src/lib/api/openapi.json b/frontend/src/lib/api/openapi.json index 98704f4..ae5e932 100644 --- a/frontend/src/lib/api/openapi.json +++ b/frontend/src/lib/api/openapi.json @@ -77,7 +77,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the points when the attribute is a mapping (``None`` = the attribute itself\nholds the points). ``shape`` says what the points are:\n\n* ``list`` (default): a list of objects named by ``fields`` (Solcast\n ``detailedForecast``);\n* ``mapping``: an object ``{iso_timestamp: value}`` (Forecast.Solar ``wh_hours``,\n Open-Meteo Solar Forecast ``wh_period_15m``); ``fields`` then gives only\n ``value_unit`` - no point field names and no quantiles.\n\nThe period length of a point is the template's ``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -111,6 +111,15 @@ "title": "Roles", "type": "array" }, + "shape": { + "default": "list", + "enum": [ + "list", + "mapping" + ], + "title": "Shape", + "type": "string" + }, "source": { "const": "attribute", "title": "Source", @@ -128,7 +137,7 @@ }, "BatteryAction": { "additionalProperties": false, - "description": "Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP9).\n\nEvery field is optional, at least one must be set; a missing field leaves that limit\nas it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a\ncharging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the\ninverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at\nor above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the\ngrid. Until the battery path exists the gate rejects this action\n(``safety.battery_not_supported``); nothing is written.", + "description": "Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP10).\n\nEvery field is optional, at least one must be set; a missing field leaves that limit\nas it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a\ncharging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the\ninverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at\nor above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the\ngrid. ``grid_charge_w`` never exceeds ``max_charge_w`` when both are set (forced\ncharging cannot be above the charge cap, and a cap of 0 forbids it). Until the battery\npath exists the gate rejects this action (``safety.battery_not_supported``); nothing is\nwritten.", "properties": { "grid_charge_w": { "anyOf": [ @@ -2775,7 +2784,7 @@ }, "GoalsConfig": { "additionalProperties": false, - "description": "User goals (schema v2).\n\nv2 drops ``grid_charge_night_only`` (answers 2026-10-16, question 7): grid charging is\nallowed by a price level and/or a schedule in the ``battery`` section's\n:class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the\nwindow a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not\nlost; the planner uses it as the policy's schedule when the policy has none. **It never\nenables grid charging**: that stays off until the user enables the policy with a level\nor a schedule. ``conservative_planning`` = targets are planned on P10 PV (default on).", + "description": "User goals (schema v2).\n\nv2 drops ``grid_charge_night_only`` (answers 2026-10-16, question 7): grid charging is\nallowed by a price level and/or a schedule in the ``battery`` section's\n:class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the\nwindow a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not\nlost; :func:`effective_grid_charge_policy` uses it as the policy's schedule when the\npolicy has none. **It never enables grid charging**: that stays off until the user\nenables the policy with a level or a schedule. How cautiously targets are planned is\nthe ``battery`` section's ``risk`` level, not a goal.", "properties": { "battery_target": { "anyOf": [ @@ -2792,11 +2801,6 @@ "title": "Climate Before Export", "type": "boolean" }, - "conservative_planning": { - "default": true, - "title": "Conservative Planning", - "type": "boolean" - }, "grid_charge_schedule": { "anyOf": [ { @@ -2818,7 +2822,7 @@ }, "GridChargePolicy": { "additionalProperties": false, - "description": "When the battery may be charged from the grid (answers 2026-10-16, question 7).\n\nOff by default. When enabled, a ``max_price`` (charge only at or below this import\nprice) and/or a ``schedule`` (only inside these weekly windows) must be set; both given\nmeans both must hold. On a dynamic tariff the planner's spread rule\n(:meth:`BatteryModel.grid_charge_pays`) applies in addition.", + "description": "When the battery may be charged from the grid (answers 2026-10-16, question 7).\n\nOff by default. When enabled, a ``max_price`` (charge only at or below this import\nprice) and/or a ``schedule`` (only inside these weekly windows) must be set; both given\nmeans both must hold. On a dynamic tariff the planner's spread rule\n(:meth:`BatteryModel.grid_charge_pays`) applies in addition.\n\nThe planner applies :func:`suncore.core.config.effective_grid_charge_policy`: this\npolicy wins over ``goals.grid_charge_schedule``, which only fills in a missing schedule\nof an enabled policy and never enables charging. Hence an enabled policy must be\ncomplete on its own (the validator below), and the goals schedule can only narrow it.", "properties": { "enabled": { "default": false, @@ -2966,7 +2970,7 @@ }, "GridFeeWindow": { "additionalProperties": false, - "description": "A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3).", + "description": "A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3).\n\n``months`` restricts it to some months (module 3 fees are set per quarter, e.g. only\nQ1 and Q4); ``None`` = all year. The month is that of the day the occurrence started\n(see :meth:`TouWindow.occurrence_date`).", "properties": { "fee": { "maximum": 100.0, @@ -2974,6 +2978,23 @@ "title": "Fee", "type": "number" }, + "months": { + "anyOf": [ + { + "items": { + "maximum": 12.0, + "minimum": 1.0, + "type": "integer" + }, + "type": "array", + "uniqueItems": true + }, + { + "type": "null" + } + ], + "title": "Months" + }, "window": { "$ref": "#/components/schemas/TouWindow" } @@ -4463,7 +4484,7 @@ }, "SeriesFields": { "additionalProperties": false, - "description": "Names of the fields of one series point and the unit of its values.", + "description": "Names of the fields of one series point and the unit of its values.\n\n``timestamp_field`` and ``value_field`` are required for a list of point objects\n(``action`` series, ``attribute`` series of shape ``list``) and absent for an\n``attribute`` series of shape ``mapping``, whose keys are the timestamps and whose\nvalues are the values.", "properties": { "p10_field": { "anyOf": [ @@ -4492,16 +4513,30 @@ "title": "P90 Field" }, "timestamp_field": { - "maxLength": 200, - "minLength": 1, - "title": "Timestamp Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Timestamp Field" }, "value_field": { - "maxLength": 200, - "minLength": 1, - "title": "Value Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Value Field" }, "value_unit": { "maxLength": 200, @@ -4511,8 +4546,6 @@ } }, "required": [ - "timestamp_field", - "value_field", "value_unit" ], "title": "SeriesFields", @@ -5045,7 +5078,7 @@ "title": "Price Series" }, "price_unit": { - "pattern": "^(\\{currency\\}|[A-Z]{3})/kWh$", + "pattern": "^(\\{currency\\}|[A-Z]{3})/(kWh|MWh)$", "title": "Price Unit", "type": "string" }, diff --git a/frontend/src/lib/api/schema.d.ts b/frontend/src/lib/api/schema.d.ts index a2cf786..e69c60c 100644 --- a/frontend/src/lib/api/schema.d.ts +++ b/frontend/src/lib/api/schema.d.ts @@ -633,9 +633,16 @@ export interface components { * @description Series held in an attribute of the entities of ``roles`` (concatenated in order). * * ``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's - * value to the list of points when the attribute is a mapping (``None`` = the attribute - * itself is the list). The period length of a point is the template's - * ``resolution_min``. + * value to the points when the attribute is a mapping (``None`` = the attribute itself + * holds the points). ``shape`` says what the points are: + * + * * ``list`` (default): a list of objects named by ``fields`` (Solcast + * ``detailedForecast``); + * * ``mapping``: an object ``{iso_timestamp: value}`` (Forecast.Solar ``wh_hours``, + * Open-Meteo Solar Forecast ``wh_period_15m``); ``fields`` then gives only + * ``value_unit`` - no point field names and no quantiles. + * + * The period length of a point is the template's ``resolution_min``. */ AttributeSeries: { /** Attribute */ @@ -645,6 +652,12 @@ export interface components { list_path?: string | null; /** Roles */ roles: string[]; + /** + * Shape + * @default list + * @enum {string} + */ + shape: "list" | "mapping"; /** * @description discriminator enum property added by openapi-typescript * @enum {string} @@ -653,15 +666,17 @@ export interface components { }; /** * BatteryAction - * @description Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP9). + * @description Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP10). * * Every field is optional, at least one must be set; a missing field leaves that limit * as it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a * charging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the * inverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at * or above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the - * grid. Until the battery path exists the gate rejects this action - * (``safety.battery_not_supported``); nothing is written. + * grid. ``grid_charge_w`` never exceeds ``max_charge_w`` when both are set (forced + * charging cannot be above the charge cap, and a cap of 0 forbids it). Until the battery + * path exists the gate rejects this action (``safety.battery_not_supported``); nothing is + * written. */ BatteryAction: { /** Grid Charge W */ @@ -1627,9 +1642,10 @@ export interface components { * allowed by a price level and/or a schedule in the ``battery`` section's * :class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the * window a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not - * lost; the planner uses it as the policy's schedule when the policy has none. **It never - * enables grid charging**: that stays off until the user enables the policy with a level - * or a schedule. ``conservative_planning`` = targets are planned on P10 PV (default on). + * lost; :func:`effective_grid_charge_policy` uses it as the policy's schedule when the + * policy has none. **It never enables grid charging**: that stays off until the user + * enables the policy with a level or a schedule. How cautiously targets are planned is + * the ``battery`` section's ``risk`` level, not a goal. */ GoalsConfig: { battery_target?: components["schemas"]["BatteryTarget"] | null; @@ -1638,11 +1654,6 @@ export interface components { * @default false */ climate_before_export: boolean; - /** - * Conservative Planning - * @default true - */ - conservative_planning: boolean; grid_charge_schedule?: components["schemas"]["WeeklySchedule"] | null; /** * Schema Version @@ -1658,6 +1669,11 @@ export interface components { * price) and/or a ``schedule`` (only inside these weekly windows) must be set; both given * means both must hold. On a dynamic tariff the planner's spread rule * (:meth:`BatteryModel.grid_charge_pays`) applies in addition. + * + * The planner applies :func:`suncore.core.config.effective_grid_charge_policy`: this + * policy wins over ``goals.grid_charge_schedule``, which only fills in a missing schedule + * of an enabled policy and never enables charging. Hence an enabled policy must be + * complete on its own (the validator below), and the goals schedule can only narrow it. */ GridChargePolicy: { /** @@ -1695,10 +1711,16 @@ export interface components { /** * GridFeeWindow * @description A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3). + * + * ``months`` restricts it to some months (module 3 fees are set per quarter, e.g. only + * Q1 and Q4); ``None`` = all year. The month is that of the day the occurrence started + * (see :meth:`TouWindow.occurrence_date`). */ GridFeeWindow: { /** Fee */ fee: number; + /** Months */ + months?: number[] | null; window: components["schemas"]["TouWindow"]; }; /** HealthMessage */ @@ -2327,6 +2349,11 @@ export interface components { /** * SeriesFields * @description Names of the fields of one series point and the unit of its values. + * + * ``timestamp_field`` and ``value_field`` are required for a list of point objects + * (``action`` series, ``attribute`` series of shape ``list``) and absent for an + * ``attribute`` series of shape ``mapping``, whose keys are the timestamps and whose + * values are the values. */ SeriesFields: { /** P10 Field */ @@ -2334,9 +2361,9 @@ export interface components { /** P90 Field */ p90_field?: string | null; /** Timestamp Field */ - timestamp_field: string; + timestamp_field?: string | null; /** Value Field */ - value_field: string; + value_field?: string | null; /** Value Unit */ value_unit: string; }; diff --git a/frontend/src/lib/i18n/locales/de.json b/frontend/src/lib/i18n/locales/de.json index a31a06a..9f24c8f 100644 --- a/frontend/src/lib/i18n/locales/de.json +++ b/frontend/src/lib/i18n/locales/de.json @@ -844,6 +844,10 @@ "api.validation.quantile_unit": "Die Quantile der Prognose brauchen dieselbe Einheit.", "api.validation.range_order": "Der Beginn muss vor dem Ende liegen.", "api.validation.series_step": "Die Intervalllänge muss zwischen 0 und einem Tag liegen.", + "api.validation.net_risk_pair": "Netto-Risikowert und sein Quantil müssen zusammen angegeben werden.", + "api.validation.grid_charge_above_cap": "Laden aus dem Netz ({grid_charge_w} W) darf die Ladegrenze ({max_charge_w} W) nicht überschreiten.", + "api.validation.import_formula_all_in": "Der Tarif {template_id} liefert bereits Endpreise inklusive Gebühren und Steuern; eine Preisformel würde sie doppelt zählen.", + "tariff.import.tou_stored": "Ein Zeittarif (HT/NT) ist gespeichert. Er lässt sich in einer späteren Version hier bearbeiten; Speichern lässt ihn unverändert, solange du keine andere Art wählst.", "safety.battery_not_supported": "Batteriesteuerung ist noch nicht verfügbar; es wurde nichts geschrieben.", "action.battery": "Batterie-Limits setzen" } diff --git a/frontend/src/lib/i18n/locales/en.json b/frontend/src/lib/i18n/locales/en.json index 6d375cf..bbc7daf 100644 --- a/frontend/src/lib/i18n/locales/en.json +++ b/frontend/src/lib/i18n/locales/en.json @@ -844,6 +844,10 @@ "api.validation.quantile_unit": "The forecast quantiles need the same unit.", "api.validation.range_order": "The start must be before the end.", "api.validation.series_step": "The interval length must be between 0 and one day.", + "api.validation.net_risk_pair": "The net risk value and its quantile must be given together.", + "api.validation.grid_charge_above_cap": "Grid charging ({grid_charge_w} W) must not exceed the charge limit ({max_charge_w} W).", + "api.validation.import_formula_all_in": "The tariff {template_id} already delivers end prices including fees and taxes; a price formula would count them twice.", + "tariff.import.tou_stored": "A time-of-use tariff (HT/NT) is stored. It can be edited here in a later version; saving keeps it unchanged unless you choose another kind.", "safety.battery_not_supported": "Battery control is not available yet; nothing was written.", "action.battery": "Set battery limits" } diff --git a/frontend/src/lib/views/steps/PvStep.svelte b/frontend/src/lib/views/steps/PvStep.svelte index 06eed43..9fb3a30 100644 --- a/frontend/src/lib/views/steps/PvStep.svelte +++ b/frontend/src/lib/views/steps/PvStep.svelte @@ -49,7 +49,7 @@ // Every role of a chosen forecast template gets a field, also when it was stored without one. $effect(() => { for (const id of wizard.forecast.templateIds) { - ensureRoleFields(wizard.forecast, Object.keys(details[id]?.roles ?? {})); + ensureRoleFields(wizard.forecast, id, Object.keys(details[id]?.roles ?? {})); } }); @@ -165,12 +165,12 @@ {/if} {#if on} {#each Object.entries(details[tpl.id]?.roles ?? {}) as [role, spec] (role)} - {#if wizard.forecast.entities[role]} + {#if wizard.forecast.entities[tpl.id]?.[role]} {/if} diff --git a/frontend/src/lib/views/steps/TariffStep.svelte b/frontend/src/lib/views/steps/TariffStep.svelte index db0e610..c783956 100644 --- a/frontend/src/lib/views/steps/TariffStep.svelte +++ b/frontend/src/lib/views/steps/TariffStep.svelte @@ -90,6 +90,9 @@ {/each} + {#if f.importKind === 'tou'} +

{t('tariff.import.tou_stored')}

+ {/if} {#if f.importKind === 'fixed'} [number]; export interface ForecastForm { templateIds: string[]; - /** Forecast role -> entity; rows without an entity are not sent. */ - entities: Record; - /** Roles per template id, when known; a template without an entry gets every entity. */ - roles: Record; + /** Per source (template id): role -> entity. Keyed per source, so two templates that use the + * same role name (e.g. `today`) keep their own entities. Rows without an entity are not sent. */ + entities: Record>; /** Stored weighting per template id (carried through). */ weights: Record>; learning: ForecastConfig['learning'] | null; } export function emptyForecastForm(): ForecastForm { - return { templateIds: [], entities: {}, roles: {}, weights: {}, learning: null }; + return { templateIds: [], entities: {}, weights: {}, learning: null }; } export function forecastFormFromConfig(cfg: ForecastConfig | null): ForecastForm { @@ -27,26 +26,23 @@ export function forecastFormFromConfig(cfg: ForecastConfig | null): ForecastForm const form = emptyForecastForm(); form.learning = cfg.learning; for (const source of cfg.sources ?? []) { - form.templateIds.push(source.template_id); - form.weights[source.template_id] = { weight_mode: source.weight_mode, weight: source.weight }; - for (const [role, ref] of Object.entries(source.entities ?? {})) { - form.entities[role] ??= fromRef(ref); - } + const id = source.template_id; + form.templateIds.push(id); + form.weights[id] = { weight_mode: source.weight_mode, weight: source.weight }; + form.entities[id] = Object.fromEntries( + Object.entries(source.entities ?? {}).map(([role, ref]) => [role, fromRef(ref)]), + ); } return form; } export function forecastFromForm(form: ForecastForm): ForecastConfig { - const entities: Record>> = {}; - for (const [role, field] of Object.entries(form.entities)) { - const ref = toRef(field); - if (ref) entities[role] = ref; - } const sources: Source[] = form.templateIds.map((id) => { - const roles = form.roles[id]; - const own = roles - ? Object.fromEntries(Object.entries(entities).filter(([role]) => roles.includes(role))) - : { ...entities }; + const own: Record>> = {}; + for (const [role, field] of Object.entries(form.entities[id] ?? {})) { + const ref = toRef(field); + if (ref) own[role] = ref; + } const weight = form.weights[id]; return { template_id: id, @@ -62,7 +58,7 @@ export function forecastFromForm(form: ForecastForm): ForecastConfig { }; } -/** Toggle a forecast template; entities of its roles are dropped when it is switched off. */ +/** Toggle a forecast template; its entities are dropped when it is switched off. */ export function setForecastTemplate( form: ForecastForm, id: string, @@ -70,17 +66,18 @@ export function setForecastTemplate( roles: readonly string[], ): void { const has = form.templateIds.includes(id); - if (on) form.roles[id] = [...roles]; - if (on && !has) form.templateIds.push(id); - if (!on && has) { + if (on) { + if (!has) form.templateIds.push(id); + ensureRoleFields(form, id, roles); + } else if (has) { form.templateIds = form.templateIds.filter((x) => x !== id); - for (const role of roles) delete form.entities[role]; - delete form.roles[id]; + delete form.entities[id]; delete form.weights[id]; } } /** Give every role of a chosen template a (possibly empty) field to render. */ -export function ensureRoleFields(form: ForecastForm, roles: readonly string[]): void { - for (const role of roles) form.entities[role] ??= emptyRef(); +export function ensureRoleFields(form: ForecastForm, id: string, roles: readonly string[]): void { + const own = (form.entities[id] ??= {}); + for (const role of roles) own[role] ??= emptyRef(); } diff --git a/frontend/src/lib/wizard/forms.test.ts b/frontend/src/lib/wizard/forms.test.ts index 1b76990..c42f4c0 100644 --- a/frontend/src/lib/wizard/forms.test.ts +++ b/frontend/src/lib/wizard/forms.test.ts @@ -154,16 +154,48 @@ describe('forecast form', () => { const form = emptyForecastForm(); setForecastTemplate(form, 'a.pv', true, ['today']); setForecastTemplate(form, 'b.pv', true, ['b_today']); - form.entities.today = ref('sensor.a'); - form.entities.b_today = ref('sensor.b'); + form.entities['a.pv']!.today = ref('sensor.a'); + form.entities['b.pv']!.b_today = ref('sensor.b'); const sources = forecastFromForm(form).sources!; expect(sources.map((s) => Object.keys(s.entities ?? {}))).toEqual([['today'], ['b_today']]); }); + it('keeps the entities of two sources that share a role name apart', () => { + const cfg: ForecastConfig = { + schema_version: 2, + sources: [ + { + template_id: 'solcast.pv', + entities: { today: ref('sensor.solcast_today') }, + weight_mode: 'auto', + weight: null, + }, + { + template_id: 'forecast_solar.pv', + entities: { today: ref('sensor.fs_today') }, + weight_mode: 'auto', + weight: null, + }, + ], + learning, + }; + const form = forecastFormFromConfig(cfg); + expect(form.entities['solcast.pv']!.today).not.toBe(form.entities['forecast_solar.pv']!.today); + form.entities['forecast_solar.pv']!.today = ref('sensor.fs_today_new'); + const sources = forecastFromForm(form).sources!; + expect(sources.map((s) => s.entities?.today?.entity_id)).toEqual([ + 'sensor.solcast_today', + 'sensor.fs_today_new', + ]); + setForecastTemplate(form, 'solcast.pv', false, ['today']); + expect(forecastFromForm(form).sources!.map((s) => s.entities?.today?.entity_id)).toEqual([ + 'sensor.fs_today_new', + ]); + }); it('drops forecast entities of a template that is switched off', () => { const form = emptyForecastForm(); setForecastTemplate(form, 'solcast.pv', true, ['forecast_today']); - ensureRoleFields(form, ['forecast_today', 'forecast_tomorrow']); - form.entities.forecast_today = ref('sensor.today'); + ensureRoleFields(form, 'solcast.pv', ['forecast_today', 'forecast_tomorrow']); + form.entities['solcast.pv']!.forecast_today = ref('sensor.today'); expect(Object.keys(forecastFromForm(form).sources![0]!.entities!)).toEqual(['forecast_today']); setForecastTemplate(form, 'solcast.pv', false, ['forecast_today', 'forecast_tomorrow']); expect(form.templateIds).toEqual([]); @@ -824,6 +856,21 @@ describe('priority reducer', () => { }); describe('goals and operation', () => { + it('passes a stored custom grid-charge schedule through until the checkbox toggles', () => { + const custom = { windows: [{ days: [5, 6], start: '00:00', end: '00:00' }] }; + const form = goalsFormFromConfig({ + schema_version: 2, + climate_before_export: false, + grid_charge_schedule: custom, + }); + expect(form.gridChargeNightOnly).toBe(true); + expect(goalsFromForm(form).grid_charge_schedule).toEqual(custom); + form.gridChargeNightOnly = false; + expect(goalsFromForm(form).grid_charge_schedule).toBeNull(); + const fresh = goalsFormFromConfig(goalsFromForm(form)); + fresh.gridChargeNightOnly = true; + expect(goalsFromForm(fresh).grid_charge_schedule).toEqual(NIGHT_SCHEDULE); + }); it('maps goals both ways', () => { const form = goalsFormFromConfig(null); expect(form).toMatchObject({ @@ -833,7 +880,7 @@ describe('goals and operation', () => { }); expect(goalsFromForm(form).battery_target).toBeNull(); expect(goalsFromForm(form).grid_charge_schedule).toEqual(NIGHT_SCHEDULE); - expect(goalsFromForm(form).conservative_planning).toBe(true); + expect(goalsFromForm(form)).not.toHaveProperty('conservative_planning'); form.gridChargeNightOnly = false; expect(goalsFromForm(form).grid_charge_schedule).toBeNull(); expect(goalsFormFromConfig(goalsFromForm(form)).gridChargeNightOnly).toBe(false); diff --git a/frontend/src/lib/wizard/goalsForm.ts b/frontend/src/lib/wizard/goalsForm.ts index 4e8cf1d..4e80474 100644 --- a/frontend/src/lib/wizard/goalsForm.ts +++ b/frontend/src/lib/wizard/goalsForm.ts @@ -10,7 +10,9 @@ export interface GoalsForm { /** Goals v2: `true` = the grid-charge schedule is the night window 22:00-06:00. It never * enables grid charging (that is the battery section's policy). */ gridChargeNightOnly: boolean; - conservativePlanning: boolean; + /** The stored schedule (may be a custom one); sent back unchanged while the checkbox stays + * on. Only switching the checkbox off replaces it (with `null`). */ + storedSchedule: GoalsConfig['grid_charge_schedule'] | null; } /** The night window goals v1's `grid_charge_night_only` migrates to (22:00-06:00 daily). */ @@ -27,7 +29,7 @@ export function goalsFormFromConfig(cfg: GoalsConfig | null): GoalsForm { targetTime: cfg?.battery_target?.time ?? DEFAULT_TARGET_TIME, climateBeforeExport: cfg?.climate_before_export ?? false, gridChargeNightOnly: cfg ? cfg.grid_charge_schedule != null : true, - conservativePlanning: cfg?.conservative_planning ?? true, + storedSchedule: cfg?.grid_charge_schedule ?? null, }; } @@ -39,8 +41,7 @@ export function goalsFromForm(form: GoalsForm): GoalsConfig { ? { soc_pct: form.targetSoc, time: form.targetTime } : null, climate_before_export: form.climateBeforeExport, - grid_charge_schedule: form.gridChargeNightOnly ? NIGHT_SCHEDULE : null, - conservative_planning: form.conservativePlanning, + grid_charge_schedule: form.gridChargeNightOnly ? (form.storedSchedule ?? NIGHT_SCHEDULE) : null, }; } diff --git a/suncore/api/app.py b/suncore/api/app.py index 32b1c59..41f46ca 100644 --- a/suncore/api/app.py +++ b/suncore/api/app.py @@ -66,7 +66,12 @@ UnknownTemplateError, ) from suncore.api.validation import validation_exception_handler -from suncore.core.config import SECTION_MODELS, OperationConfig +from suncore.core.config import ( + SECTION_MODELS, + OperationConfig, + TariffConfig, + formula_on_all_in_source, +) from suncore.core.controlstate import ( ControlState, DeviceControlState, @@ -87,7 +92,8 @@ ImportRejectedError, UnknownSectionError, ) -from suncore.templates.models import TemplateKind +from suncore.templates import TemplateCatalog +from suncore.templates.models import TariffTemplate, TemplateKind log = logging.getLogger(__name__) @@ -383,6 +389,8 @@ async def put_section(body: BaseModel) -> BaseModel: assert isinstance(body, SiteConfig) await services.put_site(body) return body + if isinstance(body, TariffConfig): + _check_tariff_against_catalog(body, services.templates()) try: return await services.put_section(name, body) except ModeRaiseRejectedError as exc: @@ -416,6 +424,25 @@ async def put_section(body: BaseModel) -> BaseModel: ) +def _check_tariff_against_catalog(tariff: TariffConfig, catalog: TemplateCatalog) -> None: + """Checks a tariff needs the template catalogue for (same 422 body as model validation).""" + template_id = tariff.import_.template_id + template = catalog.get(template_id) if template_id is not None else None + includes = template.price_includes if isinstance(template, TariffTemplate) else None + error = formula_on_all_in_source(tariff, includes) + if error is not None: + raise RequestValidationError( + [ + { + "type": error.type, + "loc": ("body", "import", "formula"), + "msg": error.message(), + "ctx": error.context or {}, + } + ] + ) + + def _history_error(exc: Exception) -> HTTPException: if isinstance(exc, UnknownSectionError): return HTTPException(status_code=404, detail=ERR_UNKNOWN_SECTION) diff --git a/suncore/api/validation.py b/suncore/api/validation.py index a3de64f..0992c43 100644 --- a/suncore/api/validation.py +++ b/suncore/api/validation.py @@ -126,6 +126,9 @@ "suncore.quantile_unit": "api.validation.quantile_unit", "suncore.range_order": "api.validation.range_order", "suncore.series_step": "api.validation.series_step", + "suncore.import_formula_all_in": "api.validation.import_formula_all_in", + "suncore.grid_charge_above_cap": "api.validation.grid_charge_above_cap", + "suncore.net_risk_pair": "api.validation.net_risk_pair", "suncore.power_model_counter": "api.validation.power_model_counter", "suncore.power_model_measured": "api.validation.power_model_measured", "suncore.power_model_nominal": "api.validation.power_model_nominal", diff --git a/suncore/core/battery.py b/suncore/core/battery.py index 616f99c..9a04036 100644 --- a/suncore/core/battery.py +++ b/suncore/core/battery.py @@ -115,11 +115,14 @@ class BatteryStrategy(StrEnum): SELF_CONSUMPTION = "self_consumption" """Default: store surplus, discharge for the house, no grid charging unless allowed.""" PREFER_FULL = "prefer_full" - """As full as possible: target 100 % (capped by ``max_soc_pct``) as early as the P10 - forecast allows, PV surplus to the battery before consumers until full, discharge - only for the house (never for export).""" + """As full as possible: target 100 % (capped by ``max_soc_pct``) as early as the net + energy forecast at the risk quantile (``BatterySettings.risk``) allows, PV surplus to + the battery before consumers until full, discharge only for the house (never for + export).""" TARGET_BY_TIME = "target_by_time" - """Reach ``goals.battery_target`` by its time, planned on P10 PV.""" + """Reach ``goals.battery_target`` by its time, planned on the risk quantile + (``BatterySettings.risk``) of the net energy forecast (PV minus load), so the target + is met with the risk level's probability.""" @unique @@ -162,6 +165,11 @@ class GridChargePolicy(_Frozen): price) and/or a ``schedule`` (only inside these weekly windows) must be set; both given means both must hold. On a dynamic tariff the planner's spread rule (:meth:`BatteryModel.grid_charge_pays`) applies in addition. + + The planner applies :func:`suncore.core.config.effective_grid_charge_policy`: this + policy wins over ``goals.grid_charge_schedule``, which only fills in a missing schedule + of an enabled policy and never enables charging. Hence an enabled policy must be + complete on its own (the validator below), and the goals schedule can only narrow it. """ enabled: bool = False @@ -178,10 +186,11 @@ def _limited(self) -> Self: ) return self - def allows(self, at: datetime, import_price: float | None, tz: tzinfo | None = None) -> bool: + def allows(self, at: datetime, import_price: float | None, tz: tzinfo) -> bool: """Whether this policy admits grid charging at ``at`` with ``import_price``. - An unknown price never satisfies a price level. + ``tz`` is the region's zone the schedule is judged in. An unknown price never + satisfies a price level. """ if not self.enabled: return False diff --git a/suncore/core/config.py b/suncore/core/config.py index e2a2631..e9d7cfb 100644 --- a/suncore/core/config.py +++ b/suncore/core/config.py @@ -20,10 +20,12 @@ field_validator, model_validator, ) +from pydantic_core import PydanticCustomError -from suncore.core.battery import BatterySettings +from suncore.core.battery import BatterySettings, GridChargePolicy from suncore.core.decisions import OperatingMode from suncore.core.devices import DeviceConfig, DeviceId, RoleName, TemplateId +from suncore.core.series import PriceIncludes from suncore.core.site import EntityRef, SiteConfig from suncore.core.tariff import HHMM_PATTERN, NIGHT_WINDOW, PriceFormula, TouTariff, WeeklySchedule from suncore.core.validation import invalid @@ -287,6 +289,26 @@ def migrate_forecast_v1_to_v2(body: dict[str, Any]) -> dict[str, Any]: return body +def formula_on_all_in_source( + tariff: TariffConfig, price_includes: PriceIncludes | None +) -> PydanticCustomError | None: + """The error for a price formula on top of an all-in price source, else ``None``. + + ``price_includes`` is that of the import tariff's template (``None`` = unknown, e.g. a + bare ``price_entity``). An all-in price (Tibber) already contains grid fees, levies and + VAT; the formula would add them a second time. :class:`ImportTariff` cannot check this + because it does not know the template catalogue - the API does (``PUT /config/tariff``). + """ + if tariff.import_.formula is None or price_includes != "all_in": + return None + return invalid( + "import_formula_all_in", + "tariff.import: template {template_id} delivers all-in prices, a formula would " + "count fees and taxes twice", + template_id=tariff.import_.template_id or "", + ) + + def migrate_tariff_v2_to_v3(body: dict[str, Any]) -> dict[str, Any]: """Tariff v3 only adds optional fields (``formula``, kind ``tou``): unchanged body.""" return body @@ -336,16 +358,36 @@ class GoalsConfig(_Frozen): allowed by a price level and/or a schedule in the ``battery`` section's :class:`~suncore.core.battery.GridChargePolicy`. ``grid_charge_schedule`` carries the window a v1 document implied (``True`` -> 22:00-06:00 daily) so the choice is not - lost; the planner uses it as the policy's schedule when the policy has none. **It never - enables grid charging**: that stays off until the user enables the policy with a level - or a schedule. ``conservative_planning`` = targets are planned on P10 PV (default on). + lost; :func:`effective_grid_charge_policy` uses it as the policy's schedule when the + policy has none. **It never enables grid charging**: that stays off until the user + enables the policy with a level or a schedule. How cautiously targets are planned is + the ``battery`` section's ``risk`` level, not a goal. """ schema_version: int = 2 battery_target: BatteryTarget | None = None climate_before_export: bool = False grid_charge_schedule: WeeklySchedule | None = None - conservative_planning: bool = True + + +def effective_grid_charge_policy(goals: GoalsConfig, battery: BatterySettings) -> GridChargePolicy: + """The grid-charge policy the planner applies (single source of the precedence rule). + + 1. The ``battery`` section's policy wins: its ``enabled``, ``max_price`` and + ``schedule`` are used as they are. + 2. ``goals.grid_charge_schedule`` only fills in the schedule when the policy is + enabled and has none of its own (the migrated v1 "night only" choice then narrows a + price-level policy to the night window). + 3. It never enables charging: a disabled policy is returned unchanged. + + :class:`GridChargePolicy` validation matches this: an enabled policy must carry its + own limit (price level or schedule), so it is complete without ``goals`` and the goals + schedule can only narrow it, never be its only limit. + """ + policy = battery.grid_charge + if not policy.enabled or policy.schedule is not None or goals.grid_charge_schedule is None: + return policy + return policy.model_copy(update={"schedule": goals.grid_charge_schedule}) def migrate_goals_v1_to_v2(body: dict[str, Any]) -> dict[str, Any]: diff --git a/suncore/core/control.py b/suncore/core/control.py index 38f392a..0f7bfcd 100644 --- a/suncore/core/control.py +++ b/suncore/core/control.py @@ -192,7 +192,7 @@ class BatteryBand(_Frozen): """Plan v2: battery first until the *live* SoC reaches this value, then surplus may go to consumers. Evaluated by the controller against the measured SoC every tick (robust against plan deviation); ``None`` = no such rule. Accepted and stored now, the - controller wiring follows in Phase 4 WP9 (until then the controller ignores it).""" + controller wiring follows in Phase 4 WP10 (until then the controller ignores it).""" @model_validator(mode="after") def _ordered(self) -> Self: @@ -209,9 +209,15 @@ class IntervalFlows(_Frozen): """Plan v2: the flows the planner expects in an interval (mean power, W). Signs per ``core/units.py``: import/export are magnitudes, battery + = charging. - ``pv_p50_w`` is what the objective used; ``pv_p10_w`` the conservative value the - target constraints used (``None`` = the source has no quantiles). ``soc_pct`` is the - planned state of charge at the *end* of the interval (``None`` = no battery). + ``pv_p10_w``/``pv_p50_w``/``pv_p90_w`` are the PV forecast's quantiles as forecast data + (shown in the UI; ``None`` = the source has no quantiles); the objective uses P50. + Battery targets are *not* planned on P10 PV (plan §0): they use the risk quantile of + the learnt **net** energy error distribution (PV minus load) per weather class. + ``net_risk_quantile`` is that quantile (``RiskLevel.quantile``, e.g. 0.1 = cautious) + and ``net_risk_w`` the net power PV - load at that quantile the target constraints + used (may be negative; ``None`` = no target constraint in this interval). + ``soc_pct`` is the planned state of charge at the *end* of the interval (``None`` = no + battery). """ pv_p10_w: float | None = Field(default=None, ge=0, allow_inf_nan=False) @@ -222,6 +228,8 @@ class IntervalFlows(_Frozen): export_w: float = Field(ge=0, allow_inf_nan=False) battery_w: float = Field(default=0.0, allow_inf_nan=False) soc_pct: Annotated[float, Field(ge=0, le=100)] | None = None + net_risk_quantile: Annotated[float, Field(gt=0, lt=1)] | None = None + net_risk_w: float | None = Field(default=None, allow_inf_nan=False) @model_validator(mode="after") def _quantiles(self) -> Self: @@ -229,6 +237,11 @@ def _quantiles(self) -> Self: hi = self.pv_p90_w if self.pv_p90_w is not None else self.pv_p50_w if not lo <= self.pv_p50_w <= hi: raise invalid("quantile_order", "quantiles: p10 <= p50 <= p90 is violated") + if (self.net_risk_w is None) != (self.net_risk_quantile is None): + raise invalid( + "net_risk_pair", + "expected flows: net_risk_w and net_risk_quantile are set together", + ) return self @@ -304,7 +317,7 @@ class Plan(_Frozen): ``schema_version`` 1, ``source = "default"``); :func:`plan_from_payload` stamps v2. """ - schema_version: Annotated[int, Field(ge=1)] = PLAN_SCHEMA_VERSION + schema_version: Annotated[int, Field(ge=1, le=PLAN_SCHEMA_VERSION)] = PLAN_SCHEMA_VERSION created_at: AwareDatetime | None = None intervals: tuple[PlanInterval, ...] = () reasons: tuple[Reason, ...] = () diff --git a/suncore/core/decisions.py b/suncore/core/decisions.py index 60a0ee7..2befb82 100644 --- a/suncore/core/decisions.py +++ b/suncore/core/decisions.py @@ -35,7 +35,13 @@ _REASON_KEY_RE: Final = re.compile(REASON_KEY_PATTERN) DECISION_SCHEMA_VERSION: Final = 2 -"""Version written into every new :class:`Decision`.""" +"""Version written into every new :class:`Decision`. + +Adding a new action kind (Phase 4: :class:`BatteryAction`) is an *additive, non-bumping* +change: every stored v2 row stays valid as it is, and an older build that meets a row with +an unknown ``kind`` fails to parse it and skips it safely (the decision log reader logs +``decision_log.row_skipped``). The version is bumped only when an existing field changes +meaning or a stored row would need an upgrade step.""" class _FrozenModel(BaseModel): @@ -175,15 +181,17 @@ class ReleaseAction(_FrozenModel): class BatteryAction(_FrozenModel): - """Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP9). + """Battery limits for the gate (Phase 4 WP2; reviewed by ``control-safety`` in WP10). Every field is optional, at least one must be set; a missing field leaves that limit as it is. Powers are magnitudes in W (battery + = charging, so ``grid_charge_w`` is a charging power drawn from the grid): ``max_charge_w`` / ``max_discharge_w`` cap the inverter's own control, ``min_soc_pct`` is the discharge floor (discharge lock when at or above the current SoC), ``grid_charge_w > 0`` asks for forced charging from the - grid. Until the battery path exists the gate rejects this action - (``safety.battery_not_supported``); nothing is written. + grid. ``grid_charge_w`` never exceeds ``max_charge_w`` when both are set (forced + charging cannot be above the charge cap, and a cap of 0 forbids it). Until the battery + path exists the gate rejects this action (``safety.battery_not_supported``); nothing is + written. """ kind: Literal[ActionKind.BATTERY] = ActionKind.BATTERY @@ -201,6 +209,18 @@ def _not_empty(self) -> Self: and self.grid_charge_w is None ): raise invalid("battery_action_empty", "battery action: at least one limit must be set") + if ( + self.grid_charge_w is not None + and self.max_charge_w is not None + and self.grid_charge_w > self.max_charge_w + ): + raise invalid( + "grid_charge_above_cap", + "battery action: grid_charge_w ({grid_charge_w}) above max_charge_w " + "({max_charge_w})", + grid_charge_w=self.grid_charge_w, + max_charge_w=self.max_charge_w, + ) return self diff --git a/suncore/core/forecast.py b/suncore/core/forecast.py index 089375a..291a9ae 100644 --- a/suncore/core/forecast.py +++ b/suncore/core/forecast.py @@ -70,8 +70,11 @@ def of(cls, lead: timedelta) -> HorizonBucket: class AccuracyRecord(BaseModel): """Measured error of one source in one horizon bucket (WP5 bookkeeping). - ``mae_w`` is the mean absolute error and ``bias_w`` the mean signed error - (forecast - actual, so + = over-forecast) of P50 over ``n`` intervals. + ``mae_w`` is the mean absolute error and ``bias_w`` the mean signed error of P50 over + ``n`` intervals. **Sign convention (one for all of SunCore's forecast statistics): + error = actual - forecast**, so ``bias_w > 0`` means the source under-forecasts and + ``forecast + bias_w`` is the bias-corrected value; the risk model's net error + (PV error - load error) uses the same convention. """ model_config = ConfigDict(frozen=True, extra="forbid") diff --git a/suncore/core/series.py b/suncore/core/series.py index a0624dc..e08fbfb 100644 --- a/suncore/core/series.py +++ b/suncore/core/series.py @@ -28,7 +28,7 @@ from __future__ import annotations import itertools -from collections.abc import Iterable, Iterator, Sequence +from collections.abc import Collection, Iterable, Iterator, Sequence from datetime import UTC, datetime, timedelta from enum import StrEnum, unique from typing import Annotated, Final, Literal, Self @@ -393,7 +393,15 @@ def energy_to_power(series: TimeSeries) -> TimeSeries: def resample_quantiles( series: QuantileSeries | Sequence[QuantileSeries], grid: Grid ) -> QuantileSeries: - """Resample P10/P50/P90 alike; the order of the quantiles is kept (same weights).""" + """Resample P10/P50/P90 alike; the order of the quantiles is kept (same weights). + + Coarsening applies the same linear operation to every quantile, which is not how + quantiles of a sum behave: the P10 of a 60-minute total is *higher* than the mean (or + sum) of four 15-minute P10 values, because errors partly cancel. The resampled P10 is + therefore conservative (too low), the P90 too high - the safe direction for a target + constraint. The planner's risk model works on the 15-minute grid, where refining + (step-hold of a mean power) is exact. + """ items = [series] if isinstance(series, QuantileSeries) else list(series) unit = items[0].unit if items else SeriesUnit.W resample = resample_energy if unit is SeriesUnit.WH else resample_power @@ -444,42 +452,74 @@ def segments_from_points( points: Sequence[tuple[datetime, float | None]], unit: SeriesUnit, *, + expected_step: timedelta, + other_steps: Collection[timedelta] = (), last_step: timedelta | None = None, ) -> tuple[TimeSeries, ...]: """Regular segments from timestamped points (e.g. an integration's price list). - A new segment starts wherever the distance between two points changes (15 -> 60 - minutes) - this is how mixed resolutions enter the resamplers. The last point lasts as - long as the distance before it, or ``last_step`` for a single point. Points must be - strictly increasing in time; duplicates must be removed by the adapter. + ``expected_step`` is the resolution the source publishes at; ``other_steps`` are the + further resolutions the caller accepts (e.g. ``{60 min}`` for a 15-minute price source + whose next day is still hourly). Each distance ``d`` between two neighbouring points, + judged against the step ``cur`` of the running segment: + + * ``d == cur``: the segment continues; + * ``d`` is an accepted step (expected or other) and either not a multiple of ``cur`` + or the next distance is ``d`` again (or the point is the last but one): a + **resolution change** - a new segment with step ``d`` starts at this point; + * ``d`` is a multiple of ``cur``: a **gap** - the point lasts ``cur`` and the missing + slots become ``None`` (a gap is never filled with a neighbour's value); + * anything else raises ``ValueError`` (irregular timestamps; SunCore never guesses). + + So 15-minute prices at 10:00, 10:15, 10:45 give one segment with ``None`` at 10:30, + and a 60-minute distance inside 15-minute data is a gap unless the caller accepts + 60 minutes *and* the hourly rhythm continues. The last point lasts ``last_step`` + (default: the step of its segment). Points must be strictly increasing in time; + duplicates must be removed by the adapter. """ if not points: return () + if expected_step <= timedelta(0) or any(s <= timedelta(0) for s in other_steps): + msg = "segments_from_points: steps must be positive" + raise ValueError(msg) stamps = [_utc(t) for t, _ in points] if any(b <= a for a, b in itertools.pairwise(stamps)): msg = "segments_from_points: timestamps must be strictly increasing" raise ValueError(msg) - steps = [b - a for a, b in itertools.pairwise(stamps)] - if not steps: - if last_step is None: - msg = "segments_from_points: a single point needs last_step" - raise ValueError(msg) - steps = [last_step] - else: - steps.append(last_step if last_step is not None else steps[-1]) + accepted = {expected_step, *other_steps} + distances = [b - a for a, b in itertools.pairwise(stamps)] segments: list[TimeSeries] = [] - begin = 0 - for i in range(1, len(points) + 1): - if i == len(points) or steps[i] != steps[begin]: - segments.append( - TimeSeries( - start=stamps[begin], - step=steps[begin], - values=tuple(v for _, v in points[begin:i]), - unit=unit, - ) + cur = expected_step + begin = stamps[0] + values: list[float | None] = [] + + def close() -> None: + if values: + segments.append(TimeSeries(start=begin, step=cur, values=tuple(values), unit=unit)) + + for i, d in enumerate(distances): + value = points[i][1] + if d == cur: + values.append(value) + continue + steady = i + 1 >= len(distances) or distances[i + 1] == d + if d in accepted and (d % cur != timedelta(0) or steady): + close() + cur, begin, values = d, stamps[i], [value] + elif d % cur == timedelta(0): + values.append(value) + values.extend([None] * (d // cur - 1)) + else: + msg = ( + f"segments_from_points: distance {d} at {stamps[i].isoformat()} is neither " + f"an accepted step nor a gap of whole {cur} slots" ) - begin = i + raise ValueError(msg) + if last_step is not None and last_step != cur: + close() + cur, begin, values = last_step, stamps[-1], [] + values.append(points[-1][1]) + close() return tuple(segments) diff --git a/suncore/core/tariff.py b/suncore/core/tariff.py index 472901b..3605ad7 100644 --- a/suncore/core/tariff.py +++ b/suncore/core/tariff.py @@ -2,15 +2,17 @@ """Tariff mathematics (Phase 4 WP2): weekly time windows, price formula, HT/NT, export tiers. Pure and frozen. Prices are currency/kWh in the region's currency; spot prices may be -negative. Time windows are *local* wall-clock windows; callers pass the region's time zone, -the conversion happens here (``datetime.astimezone``), so DST days need no special case. +negative. Time windows are *local* wall-clock windows; callers must pass the region's time +zone (``tz`` is required everywhere - an aware ``at`` in UTC would otherwise silently be +judged in UTC), the conversion happens here (``datetime.astimezone``), so DST days need no +special case. """ from __future__ import annotations import itertools from collections.abc import Sequence -from datetime import datetime, time, timedelta, tzinfo +from datetime import date, datetime, time, timedelta, tzinfo from typing import Annotated, Final, Protocol, Self from pydantic import BaseModel, ConfigDict, Field, StringConstraints, model_validator @@ -40,11 +42,11 @@ def _minutes(hhmm: str) -> int: return int(hours) * 60 + int(minutes) -def _local(at: datetime, tz: tzinfo | None) -> datetime: +def _local(at: datetime, tz: tzinfo) -> datetime: if at.tzinfo is None: msg = "time windows need an aware datetime" raise ValueError(msg) - return at.astimezone(tz) if tz is not None else at + return at.astimezone(tz) # --- weekly windows -------------------------------------------------------------------- @@ -74,18 +76,32 @@ def _days(self) -> Self: def crosses_midnight(self) -> bool: return _minutes(self.end) < _minutes(self.start) - def contains(self, at: datetime, tz: tzinfo | None = None) -> bool: - """Whether ``at`` (aware) lies in the window, judged in ``tz`` (default: ``at``'s).""" + def contains(self, at: datetime, tz: tzinfo) -> bool: + """Whether ``at`` (aware) lies in the window, judged in the local zone ``tz``.""" + return self.occurrence_date(at, tz) is not None + + def occurrence_date(self, at: datetime, tz: tzinfo) -> date | None: + """Local date the occurrence containing ``at`` *started* on; ``None`` = outside. + + For a window crossing midnight the part after midnight belongs to the previous day + (Fri 22:00-06:00 at Sat 03:00 -> Friday's date); seasonal rules (months) use this + date, so a window starting on 31 October counts as October until it ends. + """ local = _local(at, tz) minute = local.hour * 60 + local.minute start, end = _minutes(self.start), _minutes(self.end) - today = local.weekday() - yesterday = (today - 1) % 7 + today = local.date() if start == end: - return today in self.days + return today if today.weekday() in self.days else None if start < end: - return today in self.days and start <= minute < end - return (today in self.days and minute >= start) or (yesterday in self.days and minute < end) + inside = today.weekday() in self.days and start <= minute < end + return today if inside else None + if today.weekday() in self.days and minute >= start: + return today + yesterday = today - timedelta(days=1) + if yesterday.weekday() in self.days and minute < end: + return yesterday + return None class WeeklySchedule(_Frozen): @@ -93,7 +109,7 @@ class WeeklySchedule(_Frozen): windows: Annotated[tuple[TouWindow, ...], Field(min_length=1, max_length=28)] - def contains(self, at: datetime, tz: tzinfo | None = None) -> bool: + def contains(self, at: datetime, tz: tzinfo) -> bool: return any(w.contains(at, tz) for w in self.windows) @classmethod @@ -109,11 +125,26 @@ def daily(cls, start: str, end: str) -> WeeklySchedule: # --- price formula --------------------------------------------------------------------- +def _in_season(window: TouWindow, months: frozenset[int] | None, at: datetime, tz: tzinfo) -> bool: + """``at`` lies in ``window`` and the occurrence *started* in one of ``months``.""" + started = window.occurrence_date(at, tz) + return started is not None and (months is None or started.month in months) + + class GridFeeWindow(_Frozen): - """A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3).""" + """A grid fee that applies inside ``window`` (time-variable grid fee, §14a module 3). + + ``months`` restricts it to some months (module 3 fees are set per quarter, e.g. only + Q1 and Q4); ``None`` = all year. The month is that of the day the occurrence started + (see :meth:`TouWindow.occurrence_date`). + """ window: TouWindow fee: UnitPrice + months: frozenset[Month] | None = None + + def applies(self, at: datetime, tz: tzinfo) -> bool: + return _in_season(self.window, self.months, at, tz) class PriceFormula(_Frozen): @@ -137,14 +168,14 @@ class PriceFormula(_Frozen): """Supplier margin per kWh, before VAT.""" vat_pct: float = Field(default=0.0, ge=0, le=100, allow_inf_nan=False) - def grid_fee_at(self, at: datetime | None, tz: tzinfo | None = None) -> float: + def grid_fee_at(self, at: datetime | None, tz: tzinfo) -> float: if at is not None: for entry in self.grid_fee_windows: - if entry.window.contains(at, tz): + if entry.applies(at, tz): return entry.fee return self.grid_fee - def apply(self, spot: float, at: datetime | None = None, tz: tzinfo | None = None) -> float: + def apply(self, spot: float, at: datetime | None, tz: tzinfo) -> float: """End price for ``spot`` at ``at`` (``None``: the base grid fee is used).""" net = spot * self.spot_multiplier + self.grid_fee_at(at, tz) + self.levies + self.margin return net * (1 + self.vat_pct / 100) @@ -159,13 +190,11 @@ class TouPeriod(_Frozen): window: TouWindow price: UnitPrice months: frozenset[Month] | None = None - """``None`` = all year (seasonal tariffs list the months).""" + """``None`` = all year (seasonal tariffs list the months). The month of a window that + crosses midnight is that of the day it started (:meth:`TouWindow.occurrence_date`).""" - def applies(self, at: datetime, tz: tzinfo | None = None) -> bool: - local = _local(at, tz) - if self.months is not None and local.month not in self.months: - return False - return self.window.contains(local) + def applies(self, at: datetime, tz: tzinfo) -> bool: + return _in_season(self.window, self.months, at, tz) class TouTariff(_Frozen): @@ -179,7 +208,7 @@ class TouTariff(_Frozen): base_price: UnitPrice periods: Annotated[tuple[TouPeriod, ...], Field(min_length=1, max_length=24)] - def price_at(self, at: datetime, tz: tzinfo | None = None) -> float: + def price_at(self, at: datetime, tz: tzinfo) -> float: for period in self.periods: if period.applies(at, tz): return period.price diff --git a/suncore/safety/gate.py b/suncore/safety/gate.py index dc98c52..654bb68 100644 --- a/suncore/safety/gate.py +++ b/suncore/safety/gate.py @@ -139,7 +139,7 @@ REASON_ORPHAN_DROPPED: Final = Reason(key="safety.orphan_dropped") """An orphaned runtime without the old actuator: reported, then dropped (manual check needed).""" REASON_BATTERY_NOT_SUPPORTED: Final = Reason(key="safety.battery_not_supported") -"""A battery action before the battery path exists (Phase 4 WP9): rejected, never written.""" +"""A battery action before the battery path exists (Phase 4 WP10): rejected, never written.""" MAX_CONSECUTIVE_FAILURES: Final = 3 """After this many failed writes in a row the device scope trips (watchdog).""" @@ -533,7 +533,7 @@ async def submit(self, intent: Intent) -> Decision: # noqa: PLR0911, PLR0912 - intent, DecisionKind.REJECTED, now, (REASON_EMERGENCY_STOP,), mode ) if isinstance(intent.action, BatteryAction): - # Explicitly unsupported until WP9 (control-safety): no invariants exist yet. + # Explicitly unsupported until WP10 (control-safety): no invariants exist yet. return await self._decide( intent, DecisionKind.REJECTED, now, (REASON_BATTERY_NOT_SUPPORTED,), mode ) diff --git a/suncore/safety/invariants.py b/suncore/safety/invariants.py index 353db9a..2d8eef7 100644 --- a/suncore/safety/invariants.py +++ b/suncore/safety/invariants.py @@ -108,7 +108,7 @@ def action_turns_on(action: Action, device: DeviceConfig) -> bool | None: # noq return power > 0 case SetpointAction(): return True - case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP9) + case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP10) return None return None # pragma: no cover - exhaustive @@ -150,7 +150,7 @@ def expected_power_w( # noqa: PLR0911 - one branch per action kind return max(0.0, power) case SetpointAction(): return nominal if nominal is not None else _upper(estimate) - case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP9) + case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP10) return None return None # pragma: no cover - exhaustive @@ -431,7 +431,7 @@ def action_matches_state( # noqa: PLR0911 if device is not None and isinstance(device.control, ContinuousControl): current = _continuous_power_w(current, device.control, device.power_model.nominal_w) return abs(current - power) <= POWER_TOLERANCE_W - case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP9) + case ReleaseAction() | BatteryAction(): # battery: rejected by the gate before (WP10) return False return False # pragma: no cover - exhaustive diff --git a/suncore/storage/config_store.py b/suncore/storage/config_store.py index d9190d9..d4029ae 100644 --- a/suncore/storage/config_store.py +++ b/suncore/storage/config_store.py @@ -9,6 +9,7 @@ from datetime import UTC, datetime from typing import Any, Final +import structlog from pydantic import BaseModel, ValidationError from suncore import __version__ @@ -39,6 +40,8 @@ from suncore.storage.migrations import latest_version from suncore.storage.runtime import KEY_MODE_IDENTITIES +log = structlog.get_logger(__name__) + EXPORT_FORMAT: Final = 1 """Value of the ``suncore_export`` header field.""" @@ -97,11 +100,27 @@ def legacy_forecast(body: Body) -> Body | None: return {"forecast_template_ids": ids, "forecast_entities": entities} +def migrate_forecast_v1_to_v2_logged(body: Body) -> Body: + """:func:`migrate_forecast_v1_to_v2` that warns when v1 entities have no template. + + v1 entities without a template id cannot be assigned to a source; the migration drops + them (the user picks the source again in the wizard), which is logged, never silent. + """ + ids = body.get("forecast_template_ids") or [] + entities = body.get("forecast_entities") or {} + if not ids and isinstance(entities, dict) and entities: + log.warning( + "config.forecast_v1_entities_without_template", + roles=sorted(str(role) for role in entities), + ) + return migrate_forecast_v1_to_v2(body) + + DEFAULT_SECTIONS: Final[Mapping[str, SectionSpec]] = { "site": SectionSpec(SiteConfig, (migrate_site_v1_to_v2,)), "region": SectionSpec(RegionConfig), "tariff": SectionSpec(TariffConfig, (migrate_tariff_v1_to_v2, migrate_tariff_v2_to_v3)), - "forecast": SectionSpec(ForecastConfig, (migrate_forecast_v1_to_v2,)), + "forecast": SectionSpec(ForecastConfig, (migrate_forecast_v1_to_v2_logged,)), "devices": SectionSpec(DevicesConfig), "goals": SectionSpec(GoalsConfig, (migrate_goals_v1_to_v2,)), "operation": SectionSpec(OperationConfig), diff --git a/suncore/templates/models.py b/suncore/templates/models.py index 66e6920..63c92c6 100644 --- a/suncore/templates/models.py +++ b/suncore/templates/models.py @@ -215,40 +215,62 @@ def _need(caps: list[Capability], cap: Capability, ok: bool, why: str) -> None: class SeriesFields(_Frozen): - """Names of the fields of one series point and the unit of its values.""" + """Names of the fields of one series point and the unit of its values. - timestamp_field: Text - value_field: Text + ``timestamp_field`` and ``value_field`` are required for a list of point objects + (``action`` series, ``attribute`` series of shape ``list``) and absent for an + ``attribute`` series of shape ``mapping``, whose keys are the timestamps and whose + values are the values. + """ + + timestamp_field: Text | None = None + value_field: Text | None = None p10_field: Text | None = None p90_field: Text | None = None value_unit: Text + def require_point_fields(self, where: str) -> None: + if self.timestamp_field is None or self.value_field is None: + msg = f"{where}: `timestamp_field` and `value_field` are required" + raise ValueError(msg) + class AttributeSeries(_Frozen): """Series held in an attribute of the entities of ``roles`` (concatenated in order). ``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's - value to the list of points when the attribute is a mapping (``None`` = the attribute - itself is the list). The period length of a point is the template's - ``resolution_min``. + value to the points when the attribute is a mapping (``None`` = the attribute itself + holds the points). ``shape`` says what the points are: + + * ``list`` (default): a list of objects named by ``fields`` (Solcast + ``detailedForecast``); + * ``mapping``: an object ``{iso_timestamp: value}`` (Forecast.Solar ``wh_hours``, + Open-Meteo Solar Forecast ``wh_period_15m``); ``fields`` then gives only + ``value_unit`` - no point field names and no quantiles. + + The period length of a point is the template's ``resolution_min``. """ source: Literal["attribute"] roles: Annotated[tuple[RoleName, ...], Field(min_length=1, max_length=16)] attribute: Text fields: SeriesFields - list_path: ( - Annotated[ - str, StringConstraints(pattern=r"^[A-Za-z0-9_]+(\.[A-Za-z0-9_]+)*$", max_length=200) - ] - | None - ) = None + list_path: DottedPath | None = None + shape: Literal["list", "mapping"] = "list" @model_validator(mode="after") def _unique(self) -> Self: if len(set(self.roles)) != len(self.roles): msg = "series.roles: entries must be unique" raise ValueError(msg) + f = self.fields + if self.shape == "list": + f.require_point_fields("series.fields") + elif any( + x is not None for x in (f.timestamp_field, f.value_field, f.p10_field, f.p90_field) + ): + msg = "series.fields: shape `mapping` takes only `value_unit` (keys are the timestamps)" + raise ValueError(msg) return self @@ -269,6 +291,7 @@ class ActionSeries(_Frozen): @model_validator(mode="after") def _allowed(self) -> Self: + self.fields.require_point_fields("series.fields") if not is_read_only_action(self.domain, self.service): msg = ( f"{ERR_ACTION_NOT_ALLOWED}: `{self.domain}.{self.service}` is not on the " @@ -392,14 +415,44 @@ def message_keys(self) -> list[str]: # --- tariff ------------------------------------------------------------------ +PRICE_UNIT_PATTERN: Final = r"^(\{currency\}|[A-Z]{3})/(kWh|MWh)$" +"""Unit of a tariff price: a currency (``{currency}`` = the region's) per kWh or MWh.""" + +_PRICE_UNIT_RE: Final = re.compile(PRICE_UNIT_PATTERN) + +PRICE_ENERGY_FACTORS: Final[dict[str, float]] = {"kWh": 1.0, "MWh": 1 / 1000} +"""Factor that turns a price per into a price per kWh (adapter contract).""" + + +def price_unit_factor(unit: str) -> float: + """Multiply a price in ``unit`` by this to get currency/kWh (``EUR/MWh`` -> 0.001). + + Raises ``ValueError`` for a unit that is not ``/kWh`` or ``/MWh``. + """ + match = _PRICE_UNIT_RE.match(unit) + if match is None: + msg = f"`{unit}` is not a price unit (/kWh or /MWh)" + raise ValueError(msg) + return PRICE_ENERGY_FACTORS[match.group(2)] + + +def _same_currency(unit: str, price_unit: str) -> bool: + """``unit`` is a price unit whose currency is ``price_unit``'s or ``{currency}``.""" + match = _PRICE_UNIT_RE.match(unit) + reference = _PRICE_UNIT_RE.match(price_unit) + if match is None or reference is None: + return False + return match.group(1) in (reference.group(1), "{currency}") + class TariffTemplate(TemplateHeader): kind: Literal["tariff"] direction: Literal["import", "export"] tariff_type: Literal["fixed", "dynamic"] roles: dict[RoleName, RoleSpec] = Field(default_factory=dict) - price_unit: Annotated[str, StringConstraints(pattern=r"^(\{currency\}|[A-Z]{3})/kWh$")] - """``/kWh``; ``{currency}`` takes the region's currency.""" + price_unit: Annotated[str, StringConstraints(pattern=PRICE_UNIT_PATTERN)] + """``/kWh`` or ``/MWh``; ``{currency}`` takes the region's currency. + Adapters convert to currency/kWh with :func:`price_unit_factor` (MWh: / 1000).""" resolution_min: Literal[15, 60] | None = None """``None`` = derive from the timestamps.""" price_series: SeriesSource | None = None @@ -429,10 +482,10 @@ def _tariff(self) -> Self: if self.price_series is not None: _check_series_roles(self.price_series, self.roles) unit = self.price_series.fields.value_unit - if unit not in (self.price_unit, "{currency}/kWh"): + if not _same_currency(unit, self.price_unit): msg = ( - f"price_series.fields.value_unit: `{unit}` does not match " - f"price_unit `{self.price_unit}`" + f"price_series.fields.value_unit: `{unit}` does not match the currency of " + f"price_unit `{self.price_unit}` (allowed: /kWh or /MWh)" ) raise ValueError(msg) caps = self.capabilities @@ -611,9 +664,13 @@ class BatteryModeMeaning(StrEnum): """Any other option; SunCore never selects it.""" +OptionValue = Annotated[str, StringConstraints(min_length=1, max_length=200)] +"""A select option exactly as the entity lists it (whitespace is significant).""" + + class BatteryModeOption(_Frozen): - value: Text - """The select option written to the entity.""" + value: OptionValue + """The select option written to the entity, verbatim (not stripped).""" meaning: BatteryModeMeaning @@ -752,6 +809,8 @@ def message_keys(self) -> list[str]: __all__ = [ "KIND_MODELS", + "PRICE_ENERGY_FACTORS", + "PRICE_UNIT_PATTERN", "REGION_ID_PATTERN", "SCHEMA_VERSION", "TEMPLATE_ADAPTER", @@ -785,4 +844,5 @@ def message_keys(self) -> list[str]: "TemplateKind", "TestedWith", "compile_pattern", + "price_unit_factor", ] diff --git a/templates/README.md b/templates/README.md index c4c4c28..dd1874c 100644 --- a/templates/README.md +++ b/templates/README.md @@ -126,4 +126,4 @@ Every addition is recorded below with the SunCore version that introduced it. | 0.3.0 (unreleased, Phase 2) | Initial schema v1 after the ADR-0012 review: matcher with `platforms`, `translation_key_pattern`, `unit_pattern`, digits in `domain`; `RoleSpec.same_device_as`; `PowerModelDefault` without required `nominal_w`, `counter_update_s`; `defaults.control` as `ControlSpec` (`on_off`, `steps`, `continuous`, `setpoint`, `mode`); `hysteresis_w`, `hysteresis_delay_s`, `settle_s`; `series` (`attribute` or `action`) for tariff and forecast, `price_includes`, optional `resolution_min` for tariffs; closed `capabilities` vocabulary; `min_app_version`; loader lint for unknown kind directories. | | 0.3.0 (unreleased, Phase 2) | `region.currency` is optional for `country: generic` (`None` = currency of the Home Assistant configuration). | | 0.4.0 (unreleased, Phase 3) | New `defaults.control` kind `relays` (additive union member, ADR-0012 policy): 1–3 switch roles (`roles`) and `steps`, each step naming the relays that are on (`relays`) and the power it draws (`power_w`, strictly increasing, relay sets distinct). `DeviceConfig` accepts the same control. Used by `generic.water_heater_relays`. | -| 0.5.0 (unreleased, Phase 4) | New template kind `battery` (`battery.schema.json`): `roles`, `controls` (union on `kind`: `min_soc`, `max_charge`, `max_discharge`, `grid_charge_power` numbers; `mode` select with option meanings; `force_charge` switch), optional `settle_s`. New capabilities `battery_soc`, `min_soc_limit`, `charge_limit`, `discharge_limit`, `grid_charge`, `battery_modes`. `series` of `source: attribute` takes an optional `list_path` for the generic attribute adapter. HT/NT needs no template change: time windows and prices are contract data in the `tariff` configuration section (v3). | +| 0.5.0 (unreleased, Phase 4) | New template kind `battery` (`battery.schema.json`): `roles`, `controls` (union on `kind`: `min_soc`, `max_charge`, `max_discharge`, `grid_charge_power` numbers; `mode` select with option meanings; `force_charge` switch), optional `settle_s`. New capabilities `battery_soc`, `min_soc_limit`, `charge_limit`, `discharge_limit`, `grid_charge`, `battery_modes`. `series` of `source: attribute` takes an optional `list_path` for the generic attribute adapter and an optional `shape` (`list` default, or `mapping` for `{iso_timestamp: value}` attributes such as Forecast.Solar `wh_hours` and Open-Meteo Solar Forecast `wh_period_15m`; `fields` then holds only `value_unit`, so `fields.timestamp_field`/`value_field` became optional and are required only for lists of point objects). Tariff `price_unit` and `price_series.fields.value_unit` may be per MWh (`{currency}/MWh`, `EUR/MWh`); the value unit's currency must match `price_unit`'s (or be `{currency}`), and adapters convert to currency/kWh (`price_unit_factor`: MWh / 1000). Battery mode option `value` is kept verbatim (no whitespace stripping). HT/NT needs no template change: time windows and prices are contract data in the `tariff` configuration section (v3). | diff --git a/templates/schema/forecast.schema.json b/templates/schema/forecast.schema.json index a80d5d5..267aee7 100644 --- a/templates/schema/forecast.schema.json +++ b/templates/schema/forecast.schema.json @@ -78,7 +78,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the points when the attribute is a mapping (``None`` = the attribute itself\nholds the points). ``shape`` says what the points are:\n\n* ``list`` (default): a list of objects named by ``fields`` (Solcast\n ``detailedForecast``);\n* ``mapping``: an object ``{iso_timestamp: value}`` (Forecast.Solar ``wh_hours``,\n Open-Meteo Solar Forecast ``wh_period_15m``); ``fields`` then gives only\n ``value_unit`` - no point field names and no quantiles.\n\nThe period length of a point is the template's ``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -113,6 +113,15 @@ "title": "Roles", "type": "array" }, + "shape": { + "default": "list", + "enum": [ + "list", + "mapping" + ], + "title": "Shape", + "type": "string" + }, "source": { "const": "attribute", "title": "Source", @@ -316,7 +325,7 @@ }, "SeriesFields": { "additionalProperties": false, - "description": "Names of the fields of one series point and the unit of its values.", + "description": "Names of the fields of one series point and the unit of its values.\n\n``timestamp_field`` and ``value_field`` are required for a list of point objects\n(``action`` series, ``attribute`` series of shape ``list``) and absent for an\n``attribute`` series of shape ``mapping``, whose keys are the timestamps and whose\nvalues are the values.", "properties": { "p10_field": { "anyOf": [ @@ -347,16 +356,32 @@ "title": "P90 Field" }, "timestamp_field": { - "maxLength": 200, - "minLength": 1, - "title": "Timestamp Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Timestamp Field" }, "value_field": { - "maxLength": 200, - "minLength": 1, - "title": "Value Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Value Field" }, "value_unit": { "maxLength": 200, @@ -366,8 +391,6 @@ } }, "required": [ - "timestamp_field", - "value_field", "value_unit" ], "title": "SeriesFields", diff --git a/templates/schema/tariff.schema.json b/templates/schema/tariff.schema.json index b791525..f3ce82a 100644 --- a/templates/schema/tariff.schema.json +++ b/templates/schema/tariff.schema.json @@ -78,7 +78,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the points when the attribute is a mapping (``None`` = the attribute itself\nholds the points). ``shape`` says what the points are:\n\n* ``list`` (default): a list of objects named by ``fields`` (Solcast\n ``detailedForecast``);\n* ``mapping``: an object ``{iso_timestamp: value}`` (Forecast.Solar ``wh_hours``,\n Open-Meteo Solar Forecast ``wh_period_15m``); ``fields`` then gives only\n ``value_unit`` - no point field names and no quantiles.\n\nThe period length of a point is the template's ``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -113,6 +113,15 @@ "title": "Roles", "type": "array" }, + "shape": { + "default": "list", + "enum": [ + "list", + "mapping" + ], + "title": "Shape", + "type": "string" + }, "source": { "const": "attribute", "title": "Source", @@ -316,7 +325,7 @@ }, "SeriesFields": { "additionalProperties": false, - "description": "Names of the fields of one series point and the unit of its values.", + "description": "Names of the fields of one series point and the unit of its values.\n\n``timestamp_field`` and ``value_field`` are required for a list of point objects\n(``action`` series, ``attribute`` series of shape ``list``) and absent for an\n``attribute`` series of shape ``mapping``, whose keys are the timestamps and whose\nvalues are the values.", "properties": { "p10_field": { "anyOf": [ @@ -347,16 +356,32 @@ "title": "P90 Field" }, "timestamp_field": { - "maxLength": 200, - "minLength": 1, - "title": "Timestamp Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Timestamp Field" }, "value_field": { - "maxLength": 200, - "minLength": 1, - "title": "Value Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Value Field" }, "value_unit": { "maxLength": 200, @@ -366,8 +391,6 @@ } }, "required": [ - "timestamp_field", - "value_field", "value_unit" ], "title": "SeriesFields", @@ -521,7 +544,7 @@ "title": "Price Series" }, "price_unit": { - "pattern": "^(\\{currency\\}|[A-Z]{3})/kWh$", + "pattern": "^(\\{currency\\}|[A-Z]{3})/(kWh|MWh)$", "title": "Price Unit", "type": "string" }, diff --git a/templates/schema/template.schema.json b/templates/schema/template.schema.json index e0da0b2..1c860c7 100644 --- a/templates/schema/template.schema.json +++ b/templates/schema/template.schema.json @@ -78,7 +78,7 @@ }, "AttributeSeries": { "additionalProperties": false, - "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the list of points when the attribute is a mapping (``None`` = the attribute\nitself is the list). The period length of a point is the template's\n``resolution_min``.", + "description": "Series held in an attribute of the entities of ``roles`` (concatenated in order).\n\n``list_path`` (Phase 4, generic attribute adapter): dotted path from the attribute's\nvalue to the points when the attribute is a mapping (``None`` = the attribute itself\nholds the points). ``shape`` says what the points are:\n\n* ``list`` (default): a list of objects named by ``fields`` (Solcast\n ``detailedForecast``);\n* ``mapping``: an object ``{iso_timestamp: value}`` (Forecast.Solar ``wh_hours``,\n Open-Meteo Solar Forecast ``wh_period_15m``); ``fields`` then gives only\n ``value_unit`` - no point field names and no quantiles.\n\nThe period length of a point is the template's ``resolution_min``.", "properties": { "attribute": { "maxLength": 200, @@ -113,6 +113,15 @@ "title": "Roles", "type": "array" }, + "shape": { + "default": "list", + "enum": [ + "list", + "mapping" + ], + "title": "Shape", + "type": "string" + }, "source": { "const": "attribute", "title": "Source", @@ -1476,7 +1485,7 @@ }, "SeriesFields": { "additionalProperties": false, - "description": "Names of the fields of one series point and the unit of its values.", + "description": "Names of the fields of one series point and the unit of its values.\n\n``timestamp_field`` and ``value_field`` are required for a list of point objects\n(``action`` series, ``attribute`` series of shape ``list``) and absent for an\n``attribute`` series of shape ``mapping``, whose keys are the timestamps and whose\nvalues are the values.", "properties": { "p10_field": { "anyOf": [ @@ -1507,16 +1516,32 @@ "title": "P90 Field" }, "timestamp_field": { - "maxLength": 200, - "minLength": 1, - "title": "Timestamp Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Timestamp Field" }, "value_field": { - "maxLength": 200, - "minLength": 1, - "title": "Value Field", - "type": "string" + "anyOf": [ + { + "maxLength": 200, + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Value Field" }, "value_unit": { "maxLength": 200, @@ -1526,8 +1551,6 @@ } }, "required": [ - "timestamp_field", - "value_field", "value_unit" ], "title": "SeriesFields", @@ -1755,7 +1778,7 @@ "title": "Price Series" }, "price_unit": { - "pattern": "^(\\{currency\\}|[A-Z]{3})/kWh$", + "pattern": "^(\\{currency\\}|[A-Z]{3})/(kWh|MWh)$", "title": "Price Unit", "type": "string" }, diff --git a/tests/api/test_config_api.py b/tests/api/test_config_api.py index 206d5c4..72a3930 100644 --- a/tests/api/test_config_api.py +++ b/tests/api/test_config_api.py @@ -151,3 +151,42 @@ async def boom(name: str, entry_id: int) -> Any: r = await client.post("/api/v1/config/goals/history/1/restore") assert r.status_code == 409 assert r.json() == {"detail": "api.error.history_invalid"} + + +async def test_tariff_formula_on_an_all_in_template_is_rejected(services: FakeServices) -> None: + """m6: Tibber's price is all-in; a formula on top would count fees and VAT twice.""" + dynamic = {"kind": "dynamic", "template_id": "tibber.price"} + formula = {"grid_fee": 0.08, "vat_pct": 19} + export = VALID["tariff"]["export"] + async with client_for(create_app(services)) as client: + bad = await client.put( + "/api/v1/config/tariff", + json={"import": {**dynamic, "formula": formula}, "export": export}, + ) + plain = await client.put( + "/api/v1/config/tariff", json={"import": dynamic, "export": export} + ) + entity = await client.put( # no template: price_includes unknown, the user decides + "/api/v1/config/tariff", + json={ + "import": { + "kind": "dynamic", + "price_entity": {"entity_id": "sensor.spot"}, + "formula": formula, + }, + "export": export, + }, + ) + assert bad.status_code == 422 + assert bad.json() == { + "detail": "api.error.validation", + "issues": [ + { + "loc": ["body", "import", "formula"], + "key": "api.validation.import_formula_all_in", + "args": {"template_id": "tibber.price"}, + } + ], + } + assert plain.status_code == 200 + assert entity.status_code == 200 diff --git a/tests/core/test_battery.py b/tests/core/test_battery.py index 5e29de9..b4ea845 100644 --- a/tests/core/test_battery.py +++ b/tests/core/test_battery.py @@ -92,19 +92,19 @@ def test_grid_charge_policy() -> None: off = GridChargePolicy() friday_night = datetime(2026, 10, 16, 23, tzinfo=BERLIN) noon = friday_night.replace(hour=12) - assert not off.allows(friday_night, 0.0) + assert not off.allows(friday_night, 0.0, BERLIN) with pytest.raises(ValidationError, match="price level or a schedule"): GridChargePolicy(enabled=True) level = GridChargePolicy(enabled=True, max_price=0.15) - assert level.allows(noon, 0.15) - assert not level.allows(noon, 0.16) - assert not level.allows(noon, None) # unknown price never satisfies a level + assert level.allows(noon, 0.15, BERLIN) + assert not level.allows(noon, 0.16, BERLIN) + assert not level.allows(noon, None, BERLIN) # unknown price never satisfies a level sched = GridChargePolicy(enabled=True, schedule=NIGHT_WINDOW) - assert sched.allows(friday_night, None) - assert not sched.allows(noon, 0.0) + assert sched.allows(friday_night, None, BERLIN) + assert not sched.allows(noon, 0.0, BERLIN) both = GridChargePolicy(enabled=True, max_price=-0.01, schedule=NIGHT_WINDOW) - assert both.allows(friday_night, -0.05) - assert not both.allows(friday_night, 0.0) + assert both.allows(friday_night, -0.05, BERLIN) + assert not both.allows(friday_night, 0.0, BERLIN) assert GridChargePolicy(max_price=0.1).enabled is False # a level alone enables nothing @@ -168,3 +168,14 @@ def test_battery_action_contract() -> None: ): with pytest.raises(ValidationError): BatteryAction.model_validate(bad) + + +def test_battery_action_grid_charge_never_above_the_charge_cap() -> None: + """m7: forced grid charging is bounded by ``max_charge_w``; a cap of 0 forbids it.""" + assert BatteryAction(max_charge_w=3000, grid_charge_w=3000).grid_charge_w == 3000 + assert BatteryAction(max_charge_w=0, grid_charge_w=0).grid_charge_w == 0 + assert BatteryAction(grid_charge_w=5000).max_charge_w is None # no cap given + with pytest.raises(ValidationError, match="above max_charge_w"): + BatteryAction(max_charge_w=2000, grid_charge_w=2500) + with pytest.raises(ValidationError, match="above max_charge_w"): + BatteryAction(max_charge_w=0, grid_charge_w=1) diff --git a/tests/core/test_config_phase4.py b/tests/core/test_config_phase4.py index e739551..ba72101 100644 --- a/tests/core/test_config_phase4.py +++ b/tests/core/test_config_phase4.py @@ -4,24 +4,35 @@ from __future__ import annotations import copy +from datetime import UTC, datetime from typing import Any import pytest +from hypothesis import given, settings +from hypothesis import strategies as st from pydantic import ValidationError -from suncore.core.battery import BatterySettings +from suncore.core.battery import BatterySettings, GridChargePolicy from suncore.core.config import ( SECTION_MODELS, ForecastConfig, GoalsConfig, ImportTariff, TariffConfig, + effective_grid_charge_policy, migrate_forecast_v1_to_v2, migrate_goals_v1_to_v2, migrate_tariff_v2_to_v3, ) from suncore.core.devices import DeviceConfig -from suncore.core.tariff import NIGHT_WINDOW, PriceFormula, TouPeriod, TouTariff, TouWindow +from suncore.core.tariff import ( + NIGHT_WINDOW, + PriceFormula, + TouPeriod, + TouTariff, + TouWindow, + WeeklySchedule, +) TOU = TouTariff( base_price=0.32, @@ -102,9 +113,77 @@ def test_goals_v1_to_v2(v1: dict[str, Any], schedule: object) -> None: g = GoalsConfig.model_validate({**body, "schema_version": 2}) assert g.grid_charge_schedule == schedule assert g.climate_before_export is True - assert g.conservative_planning is True with pytest.raises(ValidationError): # the v1 field is gone in v2 GoalsConfig.model_validate({"grid_charge_night_only": True}) + with pytest.raises(ValidationError): # P10 planning is the battery risk level (plan §0) + GoalsConfig.model_validate({"conservative_planning": True}) + + +DAY = WeeklySchedule.daily("10:00", "16:00") + + +@pytest.mark.parametrize( + ("policy", "goals_schedule", "expected"), + [ + # 3. never enables charging: a disabled policy stays as it is + (GridChargePolicy(), NIGHT_WINDOW, GridChargePolicy()), + # 2. the goals schedule fills in a missing schedule of an enabled policy + ( + GridChargePolicy(enabled=True, max_price=0.1), + NIGHT_WINDOW, + GridChargePolicy(enabled=True, max_price=0.1, schedule=NIGHT_WINDOW), + ), + # 1. the battery policy's own schedule wins + ( + GridChargePolicy(enabled=True, schedule=DAY), + NIGHT_WINDOW, + GridChargePolicy(enabled=True, schedule=DAY), + ), + # no goals schedule: the policy unchanged + ( + GridChargePolicy(enabled=True, max_price=0.1), + None, + GridChargePolicy(enabled=True, max_price=0.1), + ), + ], +) +def test_effective_grid_charge_policy( + policy: GridChargePolicy, goals_schedule: WeeklySchedule | None, expected: GridChargePolicy +) -> None: + goals = GoalsConfig(grid_charge_schedule=goals_schedule) + battery = BatterySettings(grid_charge=policy) + effective = effective_grid_charge_policy(goals, battery) + assert effective == expected + assert effective.enabled is policy.enabled # the goals never switch charging on + + +@settings(max_examples=60, deadline=None) +@given( + enabled=st.booleans(), + max_price=st.none() | st.floats(-1, 1, allow_nan=False), + own=st.sampled_from([None, DAY]), + goals_schedule=st.sampled_from([None, NIGHT_WINDOW]), + hour=st.integers(0, 23), + price=st.none() | st.floats(-1, 1, allow_nan=False), +) +def test_effective_policy_only_narrows( # noqa: PLR0917 - hypothesis arguments + enabled: bool, + max_price: float | None, + own: WeeklySchedule | None, + goals_schedule: WeeklySchedule | None, + hour: int, + price: float | None, +) -> None: + """Whatever the goals say, the effective policy never allows more than the battery's.""" + if enabled and max_price is None and own is None: + return # invalid policy (validator), covered elsewhere + policy = GridChargePolicy(enabled=enabled, max_price=max_price, schedule=own) + effective = effective_grid_charge_policy( + GoalsConfig(grid_charge_schedule=goals_schedule), BatterySettings(grid_charge=policy) + ) + at = datetime(2026, 1, 5, hour, tzinfo=UTC) + if effective.allows(at, price, UTC): + assert policy.allows(at, price, UTC) def test_battery_is_a_section() -> None: diff --git a/tests/core/test_config_sections.py b/tests/core/test_config_sections.py index 102a193..bf252e1 100644 --- a/tests/core/test_config_sections.py +++ b/tests/core/test_config_sections.py @@ -265,7 +265,6 @@ def test_goals() -> None: g = GoalsConfig() assert g.battery_target is None assert g.grid_charge_schedule is None - assert g.conservative_planning is True assert g.schema_version == 2 assert BatteryTarget(soc_pct=80, time="06:30").time == "06:30" for bad in ("6:30", "24:00", "06:60", "06:30:00", ""): diff --git a/tests/core/test_decisions.py b/tests/core/test_decisions.py index 208cd55..0ff8fc1 100644 --- a/tests/core/test_decisions.py +++ b/tests/core/test_decisions.py @@ -15,6 +15,7 @@ DECISION_SCHEMA_VERSION, Action, ActionKind, + BatteryAction, Decision, DecisionKind, Intent, @@ -221,6 +222,59 @@ def test_v2_payload_round_trips_through_the_loader() -> None: assert decision_from_json(decision.model_dump_json()) == decision +def test_battery_action_round_trips_as_intent_and_as_observed() -> None: + action = BatteryAction(max_charge_w=4000, grid_charge_w=2500, min_soc_pct=20) + rejected = Decision( + intent=make_intent(action=action, origin=Origin.PLANNER), + kind=DecisionKind.REJECTED, + ts=TS, + reasons=(Reason(key="safety.battery_not_supported"),), + mode=OperatingMode.SHADOW, + ) + observed = Decision( + intent=make_intent(action=BatteryAction(max_discharge_w=0)), + kind=DecisionKind.FAILED, + ts=TS, + mode=OperatingMode.AUTOMATIC, + observed=action, + ) + for decision in (rejected, observed): + loaded = decision_from_json(decision.model_dump_json()) + assert loaded == decision + assert loaded.schema_version == DECISION_SCHEMA_VERSION == 2 + assert isinstance(decision_from_json(observed.model_dump_json()).observed, BatteryAction) + + +V2_PHASE3_PAYLOAD = { + "schema_version": 2, + "intent": { + "id": "i-7", + "ts": "2026-10-12T12:00:00Z", + "asset_id": "boiler", + "action": {"kind": "step", "step": 2}, + "reasons": [{"key": "controller.surplus", "args": {"surplus_w": 2100}}], + "origin": "controller", + }, + "kind": "accepted", + "ts": "2026-10-12T12:00:02Z", + "reasons": [], + "mode": "automatic", + "expires_at": None, + "held_until": None, + "observed": {"kind": "step", "step": 2}, +} +"""A Phase 3 decision row (v2, before ``BatteryAction`` existed) as the log stored it.""" + + +def test_stored_phase3_v2_row_still_loads() -> None: + """Adding the battery kind did not bump the version: old v2 rows load unchanged.""" + decision = decision_from_payload(V2_PHASE3_PAYLOAD) + assert decision.schema_version == 2 + assert decision.intent.action == StepAction(step=2) + assert decision.observed == StepAction(step=2) + assert decision_from_json(json.dumps(V2_PHASE3_PAYLOAD)) == decision + + @pytest.mark.parametrize("version", [3, 99]) def test_newer_payloads_are_refused(version: int) -> None: with pytest.raises(ValueError, match="newer"): diff --git a/tests/core/test_plan_v2.py b/tests/core/test_plan_v2.py index 1431527..804fc17 100644 --- a/tests/core/test_plan_v2.py +++ b/tests/core/test_plan_v2.py @@ -82,6 +82,8 @@ def test_v2_interval_round_trip() -> None: export_w=100, battery_w=500, soc_pct=42.5, + net_risk_quantile=0.1, + net_risk_w=-150, ) interval = PlanInterval( start=T0, @@ -89,7 +91,7 @@ def test_v2_interval_round_trip() -> None: battery=BatteryBand(min_soc_pct=15, priority_until_soc_pct=60), expected=flows, prices=IntervalPrices(import_price=0.31, export_price=-0.01), - reasons=(Reason(key="planner.battery.charge_from_pv", args={"pv_p10_kwh": 0.2}),), + reasons=(Reason(key="planner.battery.charge_from_pv", args={"net_risk_kwh": -0.04}),), ) plan = Plan( created_at=T0, @@ -111,6 +113,10 @@ def test_v2_interval_round_trip() -> None: {"import_w": -1}, {"soc_pct": 101}, {"load_w": float("nan")}, + {"net_risk_w": -100}, # without its quantile + {"net_risk_quantile": 0.1}, # without its value + {"net_risk_quantile": 0, "net_risk_w": 0}, + {"net_risk_quantile": 1, "net_risk_w": 0}, ], ) def test_interval_flows_reject(bad: dict[str, float]) -> None: @@ -120,6 +126,8 @@ def test_interval_flows_reject(bad: dict[str, float]) -> None: def test_plan_source_stats_and_band_validation() -> None: + with pytest.raises(ValidationError): # bounded by the version this build knows + Plan.model_validate({"schema_version": PLAN_SCHEMA_VERSION + 1}) with pytest.raises(ValidationError): Plan.model_validate({"source": "magic"}) with pytest.raises(ValidationError): diff --git a/tests/core/test_series.py b/tests/core/test_series.py index f6bbba4..f7715cb 100644 --- a/tests/core/test_series.py +++ b/tests/core/test_series.py @@ -224,20 +224,92 @@ def test_export_prices_are_kept_when_every_segment_has_them() -> None: def test_segments_from_points_split_on_resolution_change() -> None: + # Nord Pool style: 15-minute prices today, hourly tomorrow (caller accepts 60 min). points = [(T0 + MIN15 * i, float(i)) for i in range(4)] + [ (T0 + HOUR, 10.0), (T0 + 2 * HOUR, 20.0), ] - segments = segments_from_points(points, SeriesUnit.PRICE) + segments = segments_from_points( + points, SeriesUnit.PRICE, expected_step=MIN15, other_steps={HOUR} + ) assert [(s.step, len(s)) for s in segments] == [(MIN15, 4), (HOUR, 2)] + assert segments[0].values == (0.0, 1.0, 2.0, 3.0) assert segments[-1].end == T0 + 3 * HOUR - single = segments_from_points([(T0, 1.0)], SeriesUnit.W, last_step=MIN15) + single = segments_from_points([(T0, 1.0)], SeriesUnit.W, expected_step=MIN15) assert single[0].end == T0 + MIN15 - assert segments_from_points([], SeriesUnit.W) == () - with pytest.raises(ValueError, match="last_step"): - segments_from_points([(T0, 1.0)], SeriesUnit.W) + assert segments_from_points([], SeriesUnit.W, expected_step=MIN15) == () with pytest.raises(ValueError, match="strictly increasing"): - segments_from_points([(T0, 1.0), (T0, 2.0)], SeriesUnit.W) + segments_from_points([(T0, 1.0), (T0, 2.0)], SeriesUnit.W, expected_step=MIN15) + + +def test_segments_from_points_gap_is_none_not_filled() -> None: + """Reviewer example 1: 15-minute prices 10:00, 10:15, 10:45 -> 10:30 is missing.""" + t10 = T0 + 10 * HOUR + points = [(t10, 0.30), (t10 + MIN15, 0.31), (t10 + 3 * MIN15, 0.33)] + (segment,) = segments_from_points(points, SeriesUnit.PRICE, expected_step=MIN15) + assert segment.step == MIN15 + assert segment.values == (0.30, 0.31, None, 0.33) + out = resample_prices( + PriceSeries(source_id="nordpool", price_includes="spot_only", import_price=segment), + Grid(start=t10, count=4), + ) + assert out.import_price.values == (0.30, 0.31, None, 0.33) + + +def test_segments_from_points_hourly_gap_is_not_a_resolution_change() -> None: + """Reviewer example 2: a 60-minute hole in 15-minute data stays a hole even when the + caller accepts hourly data, because the hourly rhythm does not continue; without + ``other_steps`` a real 15 -> 60 change is a gap pattern and the hourly part keeps its + 15-minute slots with ``None`` in between (nothing is invented either way).""" + points = [(T0, 1.0), (T0 + MIN15, 2.0), (T0 + MIN15 + HOUR, 3.0), (T0 + 2 * HOUR, 4.0)] + (segment,) = segments_from_points( + points, SeriesUnit.PRICE, expected_step=MIN15, other_steps={HOUR} + ) + assert segment.values == (1.0, 2.0, None, None, None, 3.0, None, None, 4.0) + hourly = [(T0 + HOUR * i, float(i)) for i in range(3)] + (strict,) = segments_from_points(hourly, SeriesUnit.PRICE, expected_step=MIN15) + assert strict.step == MIN15 + assert strict.values == (0.0, None, None, None, 1.0, None, None, None, 2.0) + (accepted,) = segments_from_points( + hourly, SeriesUnit.PRICE, expected_step=MIN15, other_steps={HOUR} + ) + assert (accepted.step, accepted.values) == (HOUR, (0.0, 1.0, 2.0)) + + +def test_segments_from_points_rejects_irregular_and_honours_last_step() -> None: + with pytest.raises(ValueError, match="neither an accepted step"): + segments_from_points( + [(T0, 1.0), (T0 + timedelta(minutes=20), 2.0)], + SeriesUnit.PRICE, + expected_step=MIN15, + ) + with pytest.raises(ValueError, match="positive"): + segments_from_points([(T0, 1.0)], SeriesUnit.W, expected_step=timedelta(0)) + out = segments_from_points( + [(T0, 1.0), (T0 + MIN15, 2.0)], SeriesUnit.PRICE, expected_step=MIN15, last_step=HOUR + ) + assert [(s.start, s.step, s.values) for s in out] == [ + (T0, MIN15, (1.0,)), + (T0 + MIN15, HOUR, (2.0,)), + ] + assert out[-1].end == T0 + MIN15 + HOUR + + +@settings(max_examples=100, deadline=None) +@given( + values=st.lists(st.floats(-0.5, 1.0, allow_nan=False), min_size=3, max_size=96), + data=st.data(), +) +def test_segments_from_points_gap_gives_none(values: list[float], data: st.DataObject) -> None: + """Dropping interior points of a regular series yields ``None`` exactly there.""" + n = len(values) + dropped = data.draw(st.sets(st.integers(1, n - 2), max_size=n - 2)) + points = [(T0 + MIN15 * i, v) for i, v in enumerate(values) if i not in dropped] + (segment,) = segments_from_points(points, SeriesUnit.PRICE, expected_step=MIN15) + assert segment.start == T0 + assert segment.step == MIN15 + expected = tuple(None if i in dropped else v for i, v in enumerate(values)) + assert segment.values == expected # --- properties -------------------------------------------------------------------------- diff --git a/tests/core/test_tariff.py b/tests/core/test_tariff.py index e5cd5bc..bbd6200 100644 --- a/tests/core/test_tariff.py +++ b/tests/core/test_tariff.py @@ -4,7 +4,7 @@ from __future__ import annotations from dataclasses import dataclass -from datetime import UTC, datetime, time, timedelta +from datetime import UTC, date, datetime, time, timedelta from zoneinfo import ZoneInfo import pytest @@ -43,33 +43,33 @@ def test_price_source_is_re_exported() -> None: def test_window_same_day() -> None: w = TouWindow(days=frozenset({4}), start="08:00", end="20:00") assert not w.crosses_midnight - assert w.contains(at(FRI, 8)) - assert w.contains(at(FRI, 19, 59)) - assert not w.contains(at(FRI, 20)) - assert not w.contains(at(FRI, 7, 59)) - assert not w.contains(at(FRI + timedelta(days=1), 12)) + assert w.contains(at(FRI, 8), BERLIN) + assert w.contains(at(FRI, 19, 59), BERLIN) + assert not w.contains(at(FRI, 20), BERLIN) + assert not w.contains(at(FRI, 7, 59), BERLIN) + assert not w.contains(at(FRI + timedelta(days=1), 12), BERLIN) def test_window_crossing_midnight_belongs_to_its_start_day() -> None: w = TouWindow(days=frozenset({4}), start="22:00", end="06:00") # Friday night assert w.crosses_midnight - assert w.contains(at(FRI, 22)) - assert w.contains(at(FRI + timedelta(days=1), 5, 59)) # Saturday morning - assert not w.contains(at(FRI, 5)) # Friday morning belongs to Thursday's window - assert not w.contains(at(FRI + timedelta(days=1), 22)) + assert w.contains(at(FRI, 22), BERLIN) + assert w.contains(at(FRI + timedelta(days=1), 5, 59), BERLIN) # Saturday morning + assert not w.contains(at(FRI, 5), BERLIN) # Friday morning belongs to Thursday's window + assert not w.contains(at(FRI + timedelta(days=1), 22), BERLIN) def test_whole_day_window_and_time_zone_conversion() -> None: w = TouWindow(days=frozenset({5, 6}), start="00:00", end="00:00") sat = FRI + timedelta(days=1) - assert w.contains(at(sat, 0)) - assert w.contains(at(sat, 23, 59)) - assert not w.contains(at(FRI, 23, 59)) + assert w.contains(at(sat, 0), BERLIN) + assert w.contains(at(sat, 23, 59), BERLIN) + assert not w.contains(at(FRI, 23, 59), BERLIN) # 21:30 UTC on Friday is 23:30 in Berlin (summer time): still Friday. assert not w.contains(datetime(2026, 10, 16, 21, 30, tzinfo=UTC), BERLIN) assert w.contains(datetime(2026, 10, 16, 22, 30, tzinfo=UTC), BERLIN) with pytest.raises(ValueError, match="aware"): - w.contains(datetime(2026, 1, 1)) # noqa: DTZ001 + w.contains(datetime(2026, 1, 1), BERLIN) # noqa: DTZ001 @pytest.mark.parametrize( @@ -87,9 +87,9 @@ def test_window_rejects(bad: dict[str, object]) -> None: def test_night_window_and_durations() -> None: - assert NIGHT_WINDOW.contains(at(FRI, 23)) - assert NIGHT_WINDOW.contains(at(FRI, 3)) - assert not NIGHT_WINDOW.contains(at(FRI, 12)) + assert NIGHT_WINDOW.contains(at(FRI, 23), BERLIN) + assert NIGHT_WINDOW.contains(at(FRI, 3), BERLIN) + assert not NIGHT_WINDOW.contains(at(FRI, 12), BERLIN) assert window_duration(NIGHT_WINDOW.windows[0]) == timedelta(hours=8) assert window_duration(TouWindow(start="05:00", end="05:00")) == timedelta(hours=24) assert hhmm(time(6, 5, 59)) == "06:05" @@ -105,20 +105,22 @@ def test_dst_night_is_judged_in_local_time() -> None: def test_price_formula() -> None: f = PriceFormula(grid_fee=0.08, levies=0.03, margin=0.02, vat_pct=19) - assert f.apply(0.10) == pytest.approx((0.10 + 0.13) * 1.19) - assert f.apply(-0.20) == pytest.approx((-0.20 + 0.13) * 1.19) # negative end price - assert PriceFormula().apply(0.1) == pytest.approx(0.1) - assert PriceFormula(spot_multiplier=1.1).apply(0.1) == pytest.approx(0.11) + assert f.apply(0.10, None, BERLIN) == pytest.approx((0.10 + 0.13) * 1.19) + assert f.apply(-0.20, None, BERLIN) == pytest.approx( + (-0.20 + 0.13) * 1.19 + ) # negative end price + assert PriceFormula().apply(0.1, None, BERLIN) == pytest.approx(0.1) + assert PriceFormula(spot_multiplier=1.1).apply(0.1, None, BERLIN) == pytest.approx(0.11) def test_price_formula_time_variable_grid_fee() -> None: low = GridFeeWindow(window=TouWindow(start="00:00", end="06:00"), fee=0.02) high = GridFeeWindow(window=TouWindow(start="17:00", end="20:00"), fee=0.15) f = PriceFormula(grid_fee=0.08, grid_fee_windows=(low, high)) - assert f.grid_fee_at(None) == 0.08 - assert f.apply(0.1, at(FRI, 3)) == pytest.approx(0.12) - assert f.apply(0.1, at(FRI, 18)) == pytest.approx(0.25) - assert f.apply(0.1, at(FRI, 12)) == pytest.approx(0.18) + assert f.grid_fee_at(None, BERLIN) == 0.08 + assert f.apply(0.1, at(FRI, 3), BERLIN) == pytest.approx(0.12) + assert f.apply(0.1, at(FRI, 18), BERLIN) == pytest.approx(0.25) + assert f.apply(0.1, at(FRI, 12), BERLIN) == pytest.approx(0.18) assert f.apply(0.1, datetime(2026, 10, 16, 1, tzinfo=UTC), BERLIN) == pytest.approx(0.12) with pytest.raises(ValidationError): PriceFormula(spot_multiplier=0) @@ -134,10 +136,10 @@ def test_tou_tariff_first_period_wins_and_seasons() -> None: months=frozenset({11, 12, 1, 2}), ) tariff = TouTariff(base_price=0.30, periods=(nt, winter_peak)) - assert tariff.price_at(at(FRI, 23)) == 0.22 - assert tariff.price_at(at(FRI, 18)) == 0.30 # October: not winter - assert tariff.price_at(datetime(2026, 12, 4, 18, tzinfo=BERLIN)) == 0.40 - assert tariff.price_at(datetime(2026, 12, 5, 18, tzinfo=BERLIN)) == 0.30 # Saturday + assert tariff.price_at(at(FRI, 23), BERLIN) == 0.22 + assert tariff.price_at(at(FRI, 18), BERLIN) == 0.30 # October: not winter + assert tariff.price_at(datetime(2026, 12, 4, 18, tzinfo=BERLIN), BERLIN) == 0.40 + assert tariff.price_at(datetime(2026, 12, 5, 18, tzinfo=BERLIN), BERLIN) == 0.30 # Saturday with pytest.raises(ValidationError): TouTariff(base_price=0.3, periods=()) with pytest.raises(ValidationError): @@ -184,3 +186,39 @@ def test_blended_rate_lies_between_the_tier_rates( used = [t.price for t in tiers] blended = blended_export_price(tiers, total) assert min(used) - 1e-12 <= blended <= max(used) + 1e-12 + + +def test_season_of_a_window_crossing_midnight_is_its_start_day() -> None: + """m4: 31 Oct 22:00 - 1 Nov 06:00 is an October night, also after midnight.""" + october_nt = TouPeriod( + window=TouWindow(start="22:00", end="06:00"), price=0.20, months=frozenset({10}) + ) + tariff = TouTariff(base_price=0.30, periods=(october_nt,)) + assert tariff.price_at(datetime(2026, 10, 31, 23, tzinfo=BERLIN), BERLIN) == 0.20 + assert tariff.price_at(datetime(2026, 11, 1, 3, tzinfo=BERLIN), BERLIN) == 0.20 + assert tariff.price_at(datetime(2026, 11, 1, 23, tzinfo=BERLIN), BERLIN) == 0.30 + assert tariff.price_at(datetime(2026, 10, 1, 3, tzinfo=BERLIN), BERLIN) == 0.30 # Sep night + w = october_nt.window + assert w.occurrence_date(datetime(2026, 11, 1, 3, tzinfo=BERLIN), BERLIN) == date(2026, 10, 31) + assert w.occurrence_date(datetime(2026, 11, 1, 12, tzinfo=BERLIN), BERLIN) is None + + +def test_grid_fee_window_months_for_para14a_module3_quarters() -> None: + """m3: a module 3 low fee only in Q1 and Q4; the month is the start day's.""" + q1_q4 = frozenset({1, 2, 3, 10, 11, 12}) + low = GridFeeWindow(window=TouWindow(start="22:00", end="06:00"), fee=0.02, months=q1_q4) + f = PriceFormula(grid_fee=0.08, grid_fee_windows=(low,)) + assert f.grid_fee_at(datetime(2026, 1, 10, 23, tzinfo=BERLIN), BERLIN) == 0.02 + assert f.grid_fee_at(datetime(2026, 7, 10, 23, tzinfo=BERLIN), BERLIN) == 0.08 + assert f.grid_fee_at(datetime(2026, 4, 1, 2, tzinfo=BERLIN), BERLIN) == 0.02 # 31 Mar night + assert f.grid_fee_at(datetime(2026, 10, 1, 2, tzinfo=BERLIN), BERLIN) == 0.08 # 30 Sep night + assert GridFeeWindow(window=low.window, fee=0.02).months is None # additive: all year + with pytest.raises(ValidationError): + GridFeeWindow(window=low.window, fee=0.02, months=frozenset({0})) + + +def test_utc_instant_is_judged_in_the_given_zone_not_its_own() -> None: + """m5: ``tz`` is required; a UTC timestamp is converted, never judged in UTC.""" + utc_23 = datetime(2026, 10, 16, 21, 30, tzinfo=UTC) # 23:30 in Berlin + assert NIGHT_WINDOW.contains(utc_23, BERLIN) + assert not NIGHT_WINDOW.contains(utc_23, UTC) diff --git a/tests/devices/drivers/test_battery_action.py b/tests/devices/drivers/test_battery_action.py new file mode 100644 index 0000000..c0311ee --- /dev/null +++ b/tests/devices/drivers/test_battery_action.py @@ -0,0 +1,57 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Battery actions reach no device driver (Phase 4 WP2, control-safety review of 4-A). + +The gate rejects every :class:`BatteryAction` before a driver is involved; this is the +second line of defence: each of the five Phase 3 drivers refuses such an intent and +treats a battery action as release baseline as "nothing to restore" - no service call. +""" + +from __future__ import annotations + +from collections.abc import Callable + +import pytest + +from suncore.core.decisions import BatteryAction +from suncore.core.devices import DeviceConfig +from suncore.devices.drivers import ActionNotSupportedError, create_driver +from suncore.devices.drivers.climate import ClimateDriver +from suncore.devices.drivers.mode import ModeDriver +from suncore.devices.drivers.number import NumberDriver +from suncore.devices.drivers.onoff import OnOffDriver +from suncore.devices.drivers.steps import StepsDriver +from tests.devices.drivers.fakes import ( + FakeHA, + climate_device, + intent, + mode_device, + number_device, + onoff_device, + select_steps_device, +) + +ACTION = BatteryAction(max_charge_w=3000, grid_charge_w=2000, min_soc_pct=20) + +DRIVERS: list[tuple[type[object], Callable[[], DeviceConfig], dict[str, str]]] = [ + (OnOffDriver, onoff_device, {"switch.pump": "on"}), + (ClimateDriver, climate_device, {"climate.ac": "cool"}), + (StepsDriver, select_steps_device, {"select.boiler_level": "low"}), + (NumberDriver, number_device, {"number.rod": "1000"}), + (ModeDriver, mode_device, {"select.mode": "eco"}), +] + + +@pytest.mark.parametrize(("kind", "make_device", "states"), DRIVERS) +async def test_driver_refuses_battery_action_and_release_writes_nothing( + kind: type[object], make_device: Callable[[], DeviceConfig], states: dict[str, str] +) -> None: + ha = FakeHA() + for entity_id, state in states.items(): + ha.set(entity_id, state, temperature=24, min=0, max=3000, options=["eco", "boost"]) + device = make_device() + driver = create_driver(device, write=ha.write, states=ha.lookup) + assert isinstance(driver, kind) + with pytest.raises(ActionNotSupportedError): + await driver.apply(intent(ACTION, device.id)) + await driver.release(ACTION) + assert ha.calls == [] diff --git a/tests/safety/test_gate.py b/tests/safety/test_gate.py index 3d0af22..e05ba04 100644 --- a/tests/safety/test_gate.py +++ b/tests/safety/test_gate.py @@ -28,6 +28,7 @@ OperatingMode, Origin, PowerSetpointAction, + Reason, ReleaseAction, SetpointAction, SwitchAction, @@ -1206,7 +1207,7 @@ def test_n1_a_deferred_release_is_not_re_deferred_on_every_tick() -> None: assert rt.release_pending -# --- battery actions (Phase 4 WP2): explicitly unsupported until WP9 --------------------- +# --- battery actions (Phase 4 WP2): explicitly unsupported until WP10 -------------------- @pytest.mark.parametrize("mode", [OperatingMode.SHADOW, OperatingMode.ASSISTED, AUTO]) @@ -1219,6 +1220,66 @@ def test_battery_action_is_rejected_and_never_written(mode: OperatingMode) -> No assert h.gate.proposals() == () +@st.composite +def battery_actions(draw: st.DrawFn) -> BatteryAction: + power = st.none() | st.floats(0, 20_000, allow_nan=False) + max_charge = draw(power) + grid = draw(power) + if grid is not None and max_charge is not None and grid > max_charge: + grid = max_charge + values = { + "max_charge_w": max_charge, + "max_discharge_w": draw(power), + "min_soc_pct": draw(st.none() | st.floats(0, 100, allow_nan=False)), + "grid_charge_w": grid, + } + if all(v is None for v in values.values()): + values["min_soc_pct"] = 20.0 + return BatteryAction.model_validate(values) + + +@settings(max_examples=150, deadline=None) +@given( + action=battery_actions(), + origin=st.sampled_from(Origin), + global_mode=st.sampled_from(OperatingMode), + device_mode=st.sampled_from(OperatingMode), + stopped=st.booleans(), + asset_id=st.sampled_from(("pool", "ac", "ghost")), +) +def test_battery_action_is_always_rejected_without_side_effects( # noqa: PLR0917 + action: BatteryAction, + origin: Origin, + global_mode: OperatingMode, + device_mode: OperatingMode, + stopped: bool, + asset_id: str, +) -> None: + """Every origin x mode x emergency stop: REJECTED, no write, no release, no state.""" + h = make_harness(global_mode=global_mode, device_mode=device_mode) + + async def scenario() -> tuple[Decision, int, tuple[DeviceRuntime, ...]]: + if stopped: + await h.gate.emergency_stop() # releases what it owns before the battery intent + releases = h.release_calls + before = tuple(await h.store.all_runtimes()) + decision = await h.gate.submit(make_intent(action, asset_id=asset_id, origin=origin)) + assert tuple(await h.store.all_runtimes()) == before # runtime store unchanged + return decision, releases, before + + decision, releases, _ = asyncio.run(scenario()) + assert decision.kind is DecisionKind.REJECTED + assert not decision.wrote_hardware + expected = REASON_EMERGENCY_STOP if stopped else Reason(key="safety.battery_not_supported") + assert decision.reasons == (expected,) + assert h.apply_calls == 0 + assert h.release_calls == releases # the battery intent released nothing + if not stopped: + assert releases == 0 + assert h.gate.proposals() == () + assert h.gate.emergency_stopped is stopped + + def test_invariant_helpers_treat_battery_actions_as_no_device_change() -> None: from suncore.safety.invariants import ( # noqa: PLC0415 action_matches_state, diff --git a/tests/storage/test_config_sections.py b/tests/storage/test_config_sections.py index 6563fa5..132c239 100644 --- a/tests/storage/test_config_sections.py +++ b/tests/storage/test_config_sections.py @@ -244,7 +244,6 @@ def _devices(draw: st.DrawFn) -> DevicesConfig: | st.builds(BatteryTarget, soc_pct=_pct, time=st.sampled_from(["06:30", "23:59", "00:00"])), climate_before_export=st.booleans(), grid_charge_schedule=st.sampled_from([None, NIGHT_WINDOW]), - conservative_planning=st.booleans(), ), "operation": st.builds( OperationConfig, diff --git a/tests/storage/test_phase4_migrations.py b/tests/storage/test_phase4_migrations.py index 32b45ea..f4ddbf5 100644 --- a/tests/storage/test_phase4_migrations.py +++ b/tests/storage/test_phase4_migrations.py @@ -8,12 +8,13 @@ from typing import Any import pytest +import structlog.testing from suncore.core.battery import BatterySettings, BatteryStrategy from suncore.core.config import ForecastConfig, GoalsConfig, TariffConfig from suncore.core.tariff import NIGHT_WINDOW from suncore.storage import ConfigStore, Database -from suncore.storage.config_store import DEFAULT_SECTIONS +from suncore.storage.config_store import DEFAULT_SECTIONS, migrate_forecast_v1_to_v2_logged EXPORT = json.loads((Path(__file__).parent / "fixtures" / "export_phase3.json").read_text("utf-8")) @@ -73,3 +74,43 @@ async def test_battery_section_round_trips(db: Database) -> None: await store.put("battery", settings) assert await store.get("battery") == settings assert not settings.grid_charge.enabled + + +@pytest.mark.parametrize("section", ["forecast", "goals"]) +async def test_restore_of_a_v1_history_entry_migrates(db: Database, section: str) -> None: + """A goals v1 / forecast v1 body in ``config_history`` restores as the current version.""" + doc = EXPORT["sections"][section] + assert doc["schema_version"] == 1 + await db.execute( + "INSERT INTO config_history (section, schema_version, body, replaced_at) " + "VALUES (?, 1, ?, 't')", + (section, json.dumps(doc["body"])), + ) + store = ConfigStore(db) + (entry,) = await store.history(section) + restored = await store.restore(section, entry.id) + assert restored.schema_version == 2 # type: ignore[attr-defined] + assert await store.get(section) == restored + if isinstance(restored, GoalsConfig): + assert restored.grid_charge_schedule == NIGHT_WINDOW + else: + assert isinstance(restored, ForecastConfig) + assert restored.template_ids == ["solcast.pv"] + + +async def test_forecast_v1_entities_without_template_are_logged(db: Database) -> None: + body = {"forecast_template_ids": [], "forecast_entities": {"today": {"entity_id": "sensor.t"}}} + await _store_raw(db, "forecast", 1, body) + with structlog.testing.capture_logs() as logs: + loaded = await ConfigStore(db).get("forecast") + assert isinstance(loaded, ForecastConfig) + assert loaded.sources == [] + warnings = [e for e in logs if e["event"] == "config.forecast_v1_entities_without_template"] + assert len(warnings) == 1 + assert warnings[0]["log_level"] == "warning" + assert warnings[0]["roles"] == ["today"] + with structlog.testing.capture_logs() as quiet: # with a template nothing is logged + migrate_forecast_v1_to_v2_logged( + {"forecast_template_ids": ["solcast.pv"], "forecast_entities": {"today": {}}} + ) + assert quiet == [] diff --git a/tests/templates/test_battery_template.py b/tests/templates/test_battery_template.py index 6dfba17..a5b891c 100644 --- a/tests/templates/test_battery_template.py +++ b/tests/templates/test_battery_template.py @@ -11,7 +11,12 @@ from pydantic import ValidationError from suncore.templates import BatteryTemplate, load_templates -from suncore.templates.models import TEMPLATE_ADAPTER, TEMPLATE_KINDS, ForecastTemplate +from suncore.templates.models import ( + TEMPLATE_ADAPTER, + TEMPLATE_KINDS, + ForecastTemplate, + ModeSelectControl, +) from tests.templates.conftest import _header, forecast_doc, write_template @@ -148,3 +153,14 @@ def test_forecast_series_list_path() -> None: doc["series"]["list_path"] = "a..b" with pytest.raises(ValidationError): TEMPLATE_ADAPTER.validate_python(doc) + + +def test_mode_option_value_is_kept_verbatim() -> None: + """The option is written to the select as listed; whitespace must not be stripped.""" + doc = battery_doc() + doc["controls"]["mode_select"]["options"][1]["value"] = " Force charge " + template = TEMPLATE_ADAPTER.validate_python(doc) + assert isinstance(template, BatteryTemplate) + control = template.controls["mode_select"] + assert isinstance(control, ModeSelectControl) + assert control.options[1].value == " Force charge " diff --git a/tests/templates/test_models.py b/tests/templates/test_models.py index 8ff87cc..a8fd44b 100644 --- a/tests/templates/test_models.py +++ b/tests/templates/test_models.py @@ -17,6 +17,7 @@ RegionTemplate, TariffTemplate, ) +from suncore.templates.models import price_unit_factor from tests.templates.conftest import DOCS, device_doc, forecast_doc, region_doc, tariff_doc EXPECTED = { @@ -246,3 +247,43 @@ def test_min_app_version_is_a_header_field() -> None: doc = device_doc() doc["min_app_version"] = "0.3.0" assert TEMPLATE_ADAPTER.validate_python(doc).min_app_version == "0.3.0" + + +def test_tariff_prices_per_mwh_are_accepted_and_converted() -> None: + """M4: ENTSO-E/EPEX style prices in /MWh; the adapter divides by 1000.""" + doc = tariff_doc() + doc["price_series"]["fields"]["value_unit"] = "{currency}/MWh" + assert TEMPLATE_ADAPTER.validate_python(doc).kind == "tariff" + doc["price_unit"] = "EUR/MWh" + doc["price_series"]["fields"]["value_unit"] = "EUR/MWh" + assert TEMPLATE_ADAPTER.validate_python(doc).kind == "tariff" + _bad(doc, lambda d: d["price_series"]["fields"].update(value_unit="USD/kWh")) + _bad(doc, lambda d: d["price_series"]["fields"].update(value_unit="EUR/Wh")) + _bad(doc, lambda d: d.update(price_unit="EUR/GWh")) + assert price_unit_factor("{currency}/kWh") == 1.0 + assert price_unit_factor("EUR/MWh") * 85.0 == pytest.approx(0.085) + with pytest.raises(ValueError, match="not a price unit"): + price_unit_factor("EUR") + + +def test_attribute_series_mapping_shape() -> None: + """M4: ``{iso_timestamp: value}`` attributes (Forecast.Solar ``wh_hours``).""" + doc = forecast_doc() + doc["series"].update(attribute="wh_hours", shape="mapping") + doc["series"]["fields"] = {"value_unit": "Wh"} + doc["resolution_min"] = 60 + template = TEMPLATE_ADAPTER.validate_python(doc) + assert isinstance(template, ForecastTemplate) + assert template.series.shape == "mapping" # type: ignore[union-attr] + _bad(doc, lambda d: d["series"]["fields"].update(timestamp_field="t")) + _bad(doc, lambda d: d["series"]["fields"].update(p10_field="p10")) + _bad(doc, lambda d: d["series"].update(shape="table")) + listed = forecast_doc() + assert TEMPLATE_ADAPTER.validate_python(listed).series.shape == "list" # type: ignore[union-attr] + _bad(listed, lambda d: d["series"]["fields"].pop("timestamp_field")) + action = tariff_doc() + _bad(action, lambda d: d["price_series"]["fields"].pop("timestamp_field")) + nested = forecast_doc() + nested["series"]["list_path"] = "result.watts" + assert TEMPLATE_ADAPTER.validate_python(nested).kind == "forecast" + _bad(nested, lambda d: d["series"].update(list_path="result..watts")) From 2c9a49ba7bd6c8b6fde5193218debe8a1a3a3d8a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 17 Oct 2026 02:10:12 +0000 Subject: [PATCH 08/12] docs: fix markdownlint findings in the Phase 4 research and status notes Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/STATUS.md | 4 ++-- docs/research/ha-websocket-api.md | 15 +++++++++++---- docs/research/integrations.md | 5 +++-- templates/README.md | 3 ++- 4 files changed, 18 insertions(+), 9 deletions(-) diff --git a/docs/STATUS.md b/docs/STATUS.md index 2352748..d3f3843 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -303,8 +303,8 @@ maintainer's shadow test · **Branch:** `claude/zealous-mayer-j5oda9` ## Open – Phase 4 - PR order (stacked, each on the previous, `phase-4/-`): 4-B storage/time series - (WP3) → 4-C tariffs (WP4) and 4-D PV forecast (WP5) → 4-E load + risk (WP6, WP7) → 4-F planner - + runtime (WP8, WP9) → 4-G battery control (WP10) → 4-H frontend + docs (WP11, WP13); 4-I ML load + (WP3) → 4-C tariffs (WP4) and 4-D PV forecast (WP5) → 4-E load + risk (WP6, WP7) → 4-F planner and + runtime (WP8, WP9) → 4-G battery control (WP10) → 4-H frontend + docs (WP11, WP13); 4-I ML load (WP12, optional, after 4-E). Merge only after the Phase 3 acceptance. - **ADR-0013 must be accepted before WP10** (4-G). Its contract changes (`BatteryAction` `None` = baseline, `BatteryBand.grid_charge_max_w`/`discharge_lock`, `effective_reserve_pct`, diff --git a/docs/research/ha-websocket-api.md b/docs/research/ha-websocket-api.md index c98d41c..72f6aed 100644 --- a/docs/research/ha-websocket-api.md +++ b/docs/research/ha-websocket-api.md @@ -357,7 +357,11 @@ Read from `home-assistant/core` `dev` `5cfe9aad7cb428c2c2ebb449719ae81976780de1` | `units` | object, optional | keys = unit class (e.g. `"power"`, `"energy"`, `"temperature"`), value = target unit from the class's `VALID_UNITS` (`PowerConverter`: `mW, W, kW, MW, GW, TW, BTU/h`; `EnergyConverter` includes `Wh, kWh, MWh`) | Response `result` (`_ws_get_statistics_during_period`): -`{ "": [ {"start": , "end": , "mean": f, "min": f, "max": f, "state": f, "sum": f, "change": f, "last_reset": |null}, ... ] }`. + +```text +{ "": [ {"start": , "end": , "mean": f, "min": f, "max": f, + "state": f, "sum": f, "change": f, "last_reset": |null}, ... ] } +``` - `start` and `end` are **integer milliseconds since the Unix epoch** (UTC) of the period (`end = start + 5 min / 1 h`; for day and longer the reduced @@ -383,14 +387,16 @@ Response `result` (`_ws_get_statistics_during_period`): - **Retention.** The `5minute` table is purged with the recorder's `purge_keep_days` (default **10**, `recorder/__init__.py:109`); the long-term (`hour` and coarser) rows are not touched by the purge code, which removes only - `statistics_runs` and short-term rows (`recorder/purge.py:93-110`). A 14-day history for the load profile must therefore use + `statistics_runs` and short-term rows (`recorder/purge.py:93-110`). A 14-day history for + the load profile must therefore use `period: "hour"`, or `5minute` only for the last 10 days. `hour` is the finest period beyond 10 days; sub-hourly profile slots (15 min) can come from `5minute` for the recent days only. **`recorder/list_statistic_ids`** (`websocket_api.py:344-353`): optional `statistic_type` ∈ `sum|mean`. Result: list of -`{"statistic_id", "display_unit_of_measurement", "has_mean", "mean_type", "has_sum", "name", "source", "statistics_unit_of_measurement", "unit_class"}` +`statistic_id`, `display_unit_of_measurement`, `has_mean`, `mean_type`, `has_sum`, `name`, +`source`, `statistics_unit_of_measurement`, `unit_class` (`statistics.py:1136-1154`; `has_mean` is marked for removal with 2026.4 in the source comment, use `mean_type`). Includes statistics that integrations will add in the next period. **`recorder/get_statistics_metadata`** (`:456`) with optional @@ -446,7 +452,8 @@ PyPI wheels `forecast-solar 5.0.1`, `open-meteo-solar-forecast 0.1.32`. Wh, ISO strings with the offset of the **API time zone** (the plant's location), one entry per hour in the free tier; the library sums values with `begin <= timestamp < end` for "current hour" - (`forecast-solar 5.0.1` wheel, `forecast_solar/models.py` `_interval_value_sum`), i.e. it treats the key as the *start* of + (`forecast-solar 5.0.1` wheel, `forecast_solar/models.py` `_interval_value_sum`), i.e. it + treats the key as the *start* of the period. Entries with value 0 are dropped except non-midnight ones (`val != 0 or time != 00:00:00`). `UNVERIFIED`: whether the vendor API itself labels `watt_hours_period` keys as period start or end (api.forecast.solar diff --git a/docs/research/integrations.md b/docs/research/integrations.md index fdfadd8..c03ad92 100644 --- a/docs/research/integrations.md +++ b/docs/research/integrations.md @@ -264,7 +264,8 @@ exists but is undocumented on the page. - **Unit**: **currency per MWh** – the integration's own sensors divide every such value by 1000 and label the result `"/kWh"` (`sensor.py:38,205-243,347,413`). The action returns the undivided MWh value; the adapter divides by 1000. - The values are the day-ahead market price per area; VAT, grid fees and levies are not part of them (no such field in the source). + The values are the day-ahead market price per area; VAT, grid fees and levies are not part + of them (no such field in the source). - **Resolution**: slot length = `end - start` per entry (the sensors compute `resolution = entry.end - entry.start`, `sensor.py:58`), 15 or 60 minutes; a coordinator tick every 15 minutes (`coordinator.py:57-73`). The adapter must not @@ -543,7 +544,7 @@ entity ids of all integrations (derived from code only). | `weather.get_forecasts` | yes | `core@5cfe9aad` `components/weather/{services.py,const.py,entity.py}` | `type` required, response `{entity: {forecast: [...]}}`, only `datetime` guaranteed, fall back hourly -> daily, check `supported_features` | | `workday` binary sensor + `workday.check_date` | yes | `core@5cfe9aad` `components/workday/{binary_sensor,entity,services}.py` | `on` = workday; `check_date` returns `{entity: {"workday": bool}}`; offset attribute must be 0 | | `number.set_value`, `select.select_option`, `switch.turn_on/off` | yes | `core@5cfe9aad` `components/{number,select,switch}/services.py` | Unchanged semantics (range check, option check); no step enforcement | -| `modbus.write_register` | yes | `core@5cfe9aad` `components/modbus/{services.py,services.yaml,modbus.py}` | Fields `hub`, `slave`|`unit` (default 1), `address`, `value` int or list of unsigned 16-bit words; no float32 (encode two words), no read action, **errors only logged -> read back every write** | +| `modbus.write_register` | yes | `core@5cfe9aad` `components/modbus/{services.py,services.yaml,modbus.py}` | Fields `hub`, `slave` or `unit` (default 1), `address`, `value` int or list of unsigned 16-bit words; no float32 (encode two words), no read action, **errors only logged -> read back every write** | | `nordpool.get_prices_for_date` | yes | `core@5cfe9aad` `components/nordpool/services.py`; `pynordpool 0.4.0` | Response `{AREA: [{start,end,price}]}`, UTC `+00:00`, price in currency/MWh (divide by 1000), excl. VAT/fees, 15 or 60 min per entry | | `tibber.get_prices` | yes (re-read) | `core@5cfe9aad` `components/tibber/services.py` | `{"prices": {home: [{start_time, price}]}}`, price incl. taxes, call with `return_response` | | `kostal_plenticore` numbers/select | yes | `core@5cfe9aad` `components/kostal_plenticore/{number,select,coordinator}.py` | Numbers disabled by default, no translation key, ids by `unique_id` suffix `devices:local_Battery:MinSoc`; limits from live `min`/`max`; failed REST write is silent; select read-back lags 30 s; no charge set-point | diff --git a/templates/README.md b/templates/README.md index dd1874c..b1cfc4a 100644 --- a/templates/README.md +++ b/templates/README.md @@ -3,7 +3,8 @@ Declarative device, tariff, forecast and region templates (ADR-0006, ADR-0012, schema v1). Layout: `templates//.yaml` with `kind` in `device`, `tariff`, `forecast`, `region`, `battery`; user templates in `/data/templates//` override built-in ones by -`id`. A `*.yaml` file in any other sub directory is reported as an error. The JSON Schemas in `schema/` are generated from the Pydantic +`id`. A `*.yaml` file in any other sub directory is reported as an error. +The JSON Schemas in `schema/` are generated from the Pydantic models in `suncore/templates/models.py` and `suncore/core/devices.py` – never edit them by hand (`uv run python scripts/gen_schemas.py`; CI fails on drift). From 97f21f421c4989dc006c178402c06768bb5f6956 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 16:34:49 +0000 Subject: [PATCH 09/12] docs: accept ADR-0013 and record the maintainer's default answers Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/STATUS.md | 2 +- docs/adr/0013-battery-as-controlled-asset.md | 2 +- docs/adr/README.md | 2 +- docs/plans/phase-4.md | 8 ++++++++ 4 files changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/STATUS.md b/docs/STATUS.md index d3f3843..a5daf39 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -306,7 +306,7 @@ maintainer's shadow test · **Branch:** `claude/zealous-mayer-j5oda9` (WP3) → 4-C tariffs (WP4) and 4-D PV forecast (WP5) → 4-E load + risk (WP6, WP7) → 4-F planner and runtime (WP8, WP9) → 4-G battery control (WP10) → 4-H frontend + docs (WP11, WP13); 4-I ML load (WP12, optional, after 4-E). Merge only after the Phase 3 acceptance. -- **ADR-0013 must be accepted before WP10** (4-G). Its contract changes (`BatteryAction` `None` = +- **ADR-0013 accepted (2026-10-19, defaults)**; prerequisite of WP10 (4-G) met. Its contract changes (`BatteryAction` `None` = baseline, `BatteryBand.grid_charge_max_w`/`discharge_lock`, `effective_reserve_pct`, `battery_identity`) land in a WP2 fix before 4-B. - Maintainer questions from 4-A: plan §6 "Offene Fragen aus PR 4-A" (grid-charge schedule diff --git a/docs/adr/0013-battery-as-controlled-asset.md b/docs/adr/0013-battery-as-controlled-asset.md index 04b47e4..e7c2182 100644 --- a/docs/adr/0013-battery-as-controlled-asset.md +++ b/docs/adr/0013-battery-as-controlled-asset.md @@ -1,6 +1,6 @@ # ADR-0013: Battery as a controlled asset -- **Status:** proposed +- **Status:** accepted (maintainer took the recommendation as default on 2026-10-19) - **Date:** 2026-10-16 - **Deciders:** maintainer, lead engineer (`architect`), `control-safety` - **Research:** [`integrations.md`](../research/integrations.md) (Kostal Plenticore, summary diff --git a/docs/adr/README.md b/docs/adr/README.md index 6e7cd90..3442f4c 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -18,4 +18,4 @@ the status line; a new ADR supersedes an old one. Template: | [0010](0010-target-platform.md) | Target platform | accepted | HA OS app on aarch64/amd64, HA connection abstracted | | [0011](0011-two-control-layers.md) | Two control layers and safety boundary | accepted (addendum 2026-10-16: new action kinds rejected until their invariants exist) | Planner → Controller → Safety (single writer) → Actuators | | [0012](0012-template-schema-v1.md) | Template schema v1 – review before freeze | accepted | Required changes (matcher regex/lists, same-device role, `ControlSpec`, series source, capability enum) before freeze; `battery` kind added later under the v1 compatibility policy | -| [0013](0013-battery-as-controlled-asset.md) | Battery as a controlled asset | proposed | `BatteryAction` as limits (not set-points); one reserve source of truth; planned grid charging as the only bounded exception to "no SunCore-caused import"; adapter heartbeat with hardware fallback; battery identity in the fingerprint; prerequisite of Phase 4 WP10 | +| [0013](0013-battery-as-controlled-asset.md) | Battery as a controlled asset | accepted | `BatteryAction` as limits (not set-points); one reserve source of truth; planned grid charging as the only bounded exception to "no SunCore-caused import"; adapter heartbeat with hardware fallback; battery identity in the fingerprint; prerequisite of Phase 4 WP10 | diff --git a/docs/plans/phase-4.md b/docs/plans/phase-4.md index 0fed4f4..2d8ba22 100644 --- a/docs/plans/phase-4.md +++ b/docs/plans/phase-4.md @@ -375,6 +375,14 @@ shadow and assisted run is the acceptance test. write at all (the inverter's own self-consumption logic does it). - Self-sufficiency weight: explained to the maintainer, default 0.01 currency/kWh kept. +### Answers (2026-10-19) + +- ADR-0013: the maintainer takes the recommendation as default → **accepted**; WP10 may start + once 4-F is done. +- All open questions of PRs 4-A, 4-B and 4-C: **defaults** as written in each PR. +- Phase 3 acceptance (shadow run, one device to automatic) is postponed because of the weather; + Phase 4 keeps being built, the PRs are merged only after that acceptance. + ### Offene Fragen aus PR 4-A (an den Maintainer) 1. **Doppelter Netzladeplan:** `goals.grid_charge_night_only` wird zu einem Zeitplan migriert, From aaf02101cba41f84055bc785765f37ed4903268e Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 16:37:57 +0000 Subject: [PATCH 10/12] docs: wrap the ADR-0013 STATUS line Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- docs/STATUS.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/STATUS.md b/docs/STATUS.md index a5daf39..53f6232 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -306,9 +306,9 @@ maintainer's shadow test · **Branch:** `claude/zealous-mayer-j5oda9` (WP3) → 4-C tariffs (WP4) and 4-D PV forecast (WP5) → 4-E load + risk (WP6, WP7) → 4-F planner and runtime (WP8, WP9) → 4-G battery control (WP10) → 4-H frontend + docs (WP11, WP13); 4-I ML load (WP12, optional, after 4-E). Merge only after the Phase 3 acceptance. -- **ADR-0013 accepted (2026-10-19, defaults)**; prerequisite of WP10 (4-G) met. Its contract changes (`BatteryAction` `None` = - baseline, `BatteryBand.grid_charge_max_w`/`discharge_lock`, `effective_reserve_pct`, - `battery_identity`) land in a WP2 fix before 4-B. +- **ADR-0013 accepted (2026-10-19, defaults)**; prerequisite of WP10 (4-G) met. Its contract + changes (`BatteryAction` `None` = baseline, `BatteryBand.grid_charge_max_w`/`discharge_lock`, + `effective_reserve_pct`, `battery_identity`) land at the start of 4-G. - Maintainer questions from 4-A: plan §6 "Offene Fragen aus PR 4-A" (grid-charge schedule precedence, VAT on negative spot prices, export tiers, grid charging on the Kostal). From 12a1ca80abc1a776e3b22e670585c7e99c3b63cc Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 30 Oct 2026 00:23:39 +0000 Subject: [PATCH 11/12] ci: pin the frontend job to ubuntu-24.04 (Playwright 1.56.1 has no Chromium for ubuntu 26.04) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e (cherry picked from commit d9f2dc641aaad83083db8c87902aa98951592cb7) --- .github/workflows/ci.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3f36ba6..7de7733 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -72,7 +72,9 @@ jobs: frontend: name: Frontend - runs-on: ubuntu-latest + # Pinned: Playwright 1.56.1 cannot install Chromium on the ubuntu-latest image + # (ubuntu 26.04, "Playwright does not support chromium on ubuntu26.04-x64"). + runs-on: ubuntu-24.04 steps: - uses: actions/checkout@v5 with: From 9d2acd6202fc543eb3948ad4179b923595356353 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 30 Oct 2026 16:46:21 +0000 Subject: [PATCH 12/12] test(safety): accept the DST gap in the next-day resume property Hypothesis found 2026-03-28 00:00 UTC with resume time 02:00: that wall time does not exist in Berlin on the spring-forward day and correctly resolves to 03:00. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VHXsfH6PkT62o9idByAd6e --- tests/safety/test_overrides.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/safety/test_overrides.py b/tests/safety/test_overrides.py index 74e73aa..fd4adb5 100644 --- a/tests/safety/test_overrides.py +++ b/tests/safety/test_overrides.py @@ -128,7 +128,12 @@ def test_next_day_resume_is_strictly_later_and_within_48_hours(now: datetime, ho policy = OverridePolicy(resume_time=datetime(2000, 1, 1, hour).time()) # noqa: DTZ001 resume = overrides.resume_at(policy, now, BERLIN) assert now < resume <= now + timedelta(hours=48) - assert resume.astimezone(BERLIN).hour == hour + local = resume.astimezone(BERLIN) + # On the spring-forward day the wall time in the DST gap does not exist (02:xx in + # Berlin); it resolves to the same instant one hour later on the clock. + wall = datetime(local.year, local.month, local.day, hour, tzinfo=BERLIN) + in_gap = wall.astimezone(UTC).astimezone(BERLIN).hour != hour + assert local.hour == (hour + 1 if in_gap else hour) def test_hold_resume_expire() -> None: