Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 58 additions & 4 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
Read this first in every session, then `CLAUDE.md` (once it exists) and the
ADRs relevant to your task. Keep it short; history lives in git.

**Last update:** 2026-10-19 · **Phase:** 4 in progress – PR 4-A contracts, PR 4-B storage, PR 4-D PV
forecast; Phase 3 (3-A `phase-3/a-safety`, 3-B `phase-3/b-controller`, 3-C `phase-3/c-ui`)
implemented, waiting for the maintainer's shadow test · **Branch:** `phase-4/d-pv-forecast`
(stacked on `phase-4/b-storage`)
**Last update:** 2026-10-22 · **Phase:** 4 in progress – PR 4-A contracts, PR 4-B storage, PR 4-D PV
forecast, PR 4-E load forecast and risk model; Phase 3 (3-A `phase-3/a-safety`, 3-B
`phase-3/b-controller`, 3-C `phase-3/c-ui`) implemented, waiting for the maintainer's shadow test ·
**Branch:** `phase-4/e-load-risk` (stacked on `phase-4/d-pv-forecast`)

## Done

Expand Down Expand Up @@ -97,6 +97,56 @@ implemented, waiting for the maintainer's shadow test · **Branch:** `phase-4/d-
- Follow-ups: the weights come from the *uncalibrated* sources' MAE (a calibrated source is
under-weighted until a calibrated accuracy is scored); no UI yet (4-H).

- **Phase 4 – PR 4-E (WP6 load forecast, WP7 risk model; branch `phase-4/e-load-risk`, stacked on
4-D)**:
- WP6 `suncore/forecast/load.py` + `load_service.py`: base load = `site.house_w` minus every
configured device's `device.<id>.power_w` (the history rebuild also fetches the devices' power
sensors; unmeasured quarter hours count 0 W, are reported per device and are left out of a day
type's profile once it has 7 fully measured days, else kept and flagged; negative rests
clipped); profiles per day type (workday/weekend/holiday) × 96 wall-clock quarter hours,
exponential smoothing (half-life 21 days, 56 days look-back), clipped z-score (3 × max(MAD
scale, ½ median)), shrinkage of thin day types (holiday → weekend → all non-holiday days),
kernel smoothing; optional temperature term (heating/cooling degrees 15/22 °C, bounded 0–300
W/K, active after 14 days with a recorded temperature, offset ≤ 50 % of the day's mean, only for
forecast days with ≥ 20 h of temperatures); residual P10/P90 per time-of-day block; cold start
(< 3 days: flat at the 7-day mean, P10 0.5×, P90 2×), `no_history` without data. Day types: the
`workday` sensor's state for today + `days_offset`, `workday.check_date(d − days_offset)` for
other dates (Home Assistant adds the offset; ≤ 12 calls per refresh, answers cached in
`learned_param` `load_profile/day_types` with a fingerprint of entity, `workdays`, `excludes`
and offset, dropped when it changes), else the weekday rule. Refresh every 15 min, refit
hourly/after the history rebuild/nightly, stored hourly as kind `load` source `load_profile`,
accuracy (all quarter hours) in the nightly job; `GET /api/v1/forecast/load?hours=48`; the
controlled devices' histories are kept for 4-F. Benchmark (n5): a refit (read 56 days of house,
3 devices and temperature from SQLite + fit) takes 118–163 ms on the session's amd64 container
(4 vCPU), the forecast itself about 1 ms; aarch64 not measured yet.
- Config: `forecast.load.workday` / `forecast.load.weather` (additive in `forecast` v2, domain
checked; the wizard form carries them through, UI in 4-H). `workday.check_date` joined
`READ_ONLY_ACTIONS`. The weather entity's `temperature` is recorded as
`weather.temperature_c` (HA keeps no statistics of weather entities).
- WP7 `suncore/forecast/risk.py` + `risk_learning.py`: windows `rest_of_night` (before
sunrise), `rest_of_today`, `night`, `next_day`, `bridge` (night + next day, one quantile) and
`day_and_night` (coming daytime + the night after it) anchored on today's sunset from the PV
forecast; weather class from the PV clear-sky index; net energy quantile `learned` (relative
errors per window × class, pooled per window when thin) from 30 days with samples, else
`source_quantiles` (two-piece normal, PV and load independent, slot quantiles summed) or
`default_band` (PV forecast without a band: default relative band per weather class);
`evening_target_pct`, `morning_buffer_pct` (one quantile over `day_and_night` against the
median night need), floor `core.battery.effective_reserve_pct` (ADR-0013), `prefer_full` =
max SoC, manual limits and max SoC always win; `goals.battery_target` is a floor of the
evening target, its time is ignored there (timed constraint in 4-F; maintainer 2026-10-22).
Reasons `planner.battery.morning_buffer` / `.evening_target` (en/de, naming the risk level).
Nightly samples from the issues at sunrise + 1 h and sunset − 1 h against measured PV and
base load, only for windows both forecasts cover, without clipped daytime quarter hours,
kept 365 days (`learned_param` `net_error`). `GET /api/v1/forecast/risk`. Hypothesis tests
(monotone in risk level, in PV and load uncertainty and on the learned path, limits incl.
the site min SoC) and the four scenarios.
- Review fixes (M1–M5, m2–m8, n1–n6) on 2026-10-22/23, see the plan's deviations.
- Demo: deterministic workday sensor and weather entity (`binary_sensor.sim_workday_sensor`,
`weather.sim_home`, both actions answered), battery model 10 kWh / 5 kW (still observe only).
- Decisions and deviations: plan §"Deviations during implementation" (PR 4-E).
- Follow-ups: the planner (4-F) consumes `morning_buffer_pct`/`evening_target_pct` and the
device histories; no UI yet (4-H); ML load model in shadow (4-I).

- **Fix after the first live test (2026-10-16, branch `fix/device-freshness`)**: on the maintainer's
HA the SunSpec battery SoC (84 % for 25 min) and the `smartthinq_sensors` power sensors (5 W for a
day) never advance `last_reported` for an unchanged value, so SunCore marked them "stale" and the
Expand Down Expand Up @@ -593,6 +643,10 @@ implemented, waiting for the maintainer's shadow test · **Branch:** `phase-4/d-
| Supervisor proxy forwards `energy/solar_forecast` | `suncore/ha/` (`fetch_solar_forecast`) | first run on HA OS |
| PV calibration constants (classes kt 0.3/0.65, shrink k = 16, decay 0.98, factor 0.5–1.5) and the clipping thresholds (plateau ±0.3 % for ≥ 1 h on ≥ 2 days; export within max(3 %, 50 W)) are reasoned, not fitted to real data | `suncore/forecast/pv/` | 30+ days of the maintainer's data |
| Solve times on Raspberry Pi 4 | ADR-0003 | Phase 4 benchmark |
| `weather.get_forecasts` `hourly` support of the maintainer's weather integration (the `temperature_unit` attribute and the conversion of the forecast to it are verified from source, `research/ha-websocket-api.md`) | `suncore/app/adapters.py` (`HaLoadReader`) | maintainer's HA |
| `workday.check_date` over the Supervisor proxy and with the maintainer's Workday configuration (the `days_offset` handling is verified from source; a changed country/holiday list is not visible in the attributes, so the day-type cache only resets on a changed entity, `workdays`, `excludes` or offset) | `suncore/app/adapters.py`, `suncore/forecast/load_service.py` | maintainer maps a Workday sensor |
| Load and risk constants (half-life 21 days, outlier z 3, shrink 3 days, 7 clean days, degree-day bases 15/22 °C, coverage 75 %/90 %, 20 h temperature days, anchors sunrise + 1 h / sunset − 1 h, sunrise threshold 2 %/20 W) and the default PV bands per weather class (clear 0.85/1.10, mixed 0.55/1.35, overcast 0.35/1.70) are reasoned, not fitted | `suncore/forecast/load.py`, `suncore/forecast/risk.py`, `risk_learning.py` | 30+ days of the maintainer's data |
| Known limitation (n6): the load history and the risk samples are not re-derived when the device list changes - a device added later is only subtracted from the base load from the day its power is recorded (or rebuilt), and older risk samples keep the old base load | `suncore/forecast/load_service.py`, `risk_learning.py` | accepted for Phase 4; revisit with 4-I |

## Environment notes

Expand Down
41 changes: 41 additions & 0 deletions docs/plans/phase-4.md
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,47 @@ shadow and assisted run is the acceptance test.
Open-Meteo Solar Forecast, generic template).
- `BatteryAction` is part of the 4-A contracts but rejected by the gate until WP10
(ADR-0011 addendum); its semantics are proposed in ADR-0013.
- PR 4-E (WP6, WP7):
- The optional `workday` binary sensor and `weather` entity of the load forecast live in
`forecast.load` (additive in `forecast` v2, no migration; the wizard carries them through,
the UI is 4-H). `workday.check_date` joined the read-only action allowlist (verified in
`research/ha-websocket-api.md` "Workday").
- Home Assistant keeps no long-term statistics of a weather entity, so the temperature term
learns from SunCore's own recording of the entity's `temperature` attribute (key
`weather.temperature_c`) and becomes active after 14 days with a temperature.
- The risk model has a fourth window `bridge` (night + next day, sunset to sunset): the
evening target takes **one** quantile of it instead of adding the night's and the next
day's quantiles (which would count caution twice, §0).
- Learnt errors are relative to the window's forecast energy (PV P50 + load P50), so a short
`rest_of_today` and a long one, a summer and a winter day share one distribution. Samples
are taken from the forecasts as they stood at sunrise + 1 h (`rest_of_today`,
`day_and_night`) and sunset − 1 h (`night`, `next_day`, `bridge`) and kept 365 days (review
m3: fixed 06:00/18:00 anchors meant different situations in winter and summer). A window is
learnt only if the PV and load forecasts cover it (≥ 90 %); without a PV forecast only the
night; daytime windows with a clipped quarter hour (the 4-D clipping flags) are dropped.
- Before sunrise the rest of the night is its own window `rest_of_night` (review m2).
- Morning buffer (review m7, replaces the first rule with a comfort factor 2): **one** quantile
over `day_and_night` (the coming daytime and the night after it) gives the margin left after
both are covered; `share = clamp(1 − margin / median night need, 0, 1)` and
`X = floor + share × (evening target − floor)`. The evening target is only the upper bound
of X, so no caution is counted twice.
- Floor of buffer and target: `core.battery.effective_reserve_pct` (ADR-0013 decision 3;
review M4), never below the site battery's minimum SoC.
- A PV forecast without a band (P10 = P50) gets a default relative band per weather class
(`default_band`, `UNVERIFIED` constants) until the learnt distribution takes over (review
M5).
- `goals.battery_target` is a floor of the evening target; its **time** is ignored there, the
timed constraint follows in the planner (4-F) – confirmed by the maintainer on 2026-10-22.
A Monday–Friday holiday counts as a weekend day while few holidays were seen (shrinkage
towards the weekend profile) – also confirmed.
- Workday offset (review M1): Home Assistant evaluates the sensor and `workday.check_date`
for the date plus `days_offset` (`workday/entity.py`); SunCore places the state on
today + offset and asks `check_date(d − offset)`. Cached answers carry a fingerprint
(entity, `workdays`, `excludes`, `days_offset`) and are dropped when it changes (m6).
- Devices in the base load (review M2): the history rebuild also fetches the devices' power
sensors (`mean`, W); quarter hours with an unmeasured device are left out of a day type's
profile once it has 7 fully measured days, before that they are kept and flagged.
- The temperature term shifts a forecast day only with ≥ 20 hours of temperatures (m5).

## 7. Out of scope for Phase 4

Expand Down
19 changes: 14 additions & 5 deletions docs/research/ha-websocket-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,9 +299,18 @@ files `homeassistant/components/weather/{services.py,services.yaml,const.py,enti
string in practice, `ATTR_FORECAST_TIME = "datetime"`). `temperature` is set
to `null` when the entity delivered none. Keys the integration does not
provide are simply absent: the adapter must treat every key but `datetime` as
optional. The temperature unit is **not** in the response; read
`unit_of_measurement`-like info from the entity's `temperature_unit` state
attribute (not read in this pass, `UNVERIFIED`).
optional. The temperature unit is **not** in the response; it is the
entity's `temperature_unit` state attribute. Verified 2026-10-23 at the same
commit: `weather/entity.py` `state_attributes` always sets
`temperature_unit = self._temperature_unit` (line 525, key `temperature_unit`
from `weather/const.py:86/104`) and `_convert_forecast` converts every
forecast temperature to the same `_temperature_unit` (lines 608–634); values
are `UnitOfTemperature` (`°C`, `°F`, `K`, `homeassistant/const.py:587`).
- **Workday offset** (`workday/entity.py:103–115`, same commit): `check_date`
returns `date_is_workday(check_date)`, which evaluates
`adjusted_date = check_date + timedelta(days=days_offset)`; the sensor state
is the same function for today. SunCore asks `check_date(d - days_offset)`
for date `d` and places the state on `today + days_offset`.
- Whether `hourly` is offered depends on the integration (`supported_features`
bit `FORECAST_HOURLY = 2`); not every weather entity has it. Fall back to
`daily` (a coarser temperature profile) or no temperature term.
Expand Down Expand Up @@ -921,8 +930,8 @@ native range, `select` still validates against `options`); `switch.turn_on/off/t
take no fields.

Still `UNVERIFIED`: the Supervisor proxy forwarding of `recorder/*` and
`energy/*` commands; the temperature unit of a `weather.get_forecasts` response
(taken from the entity's attributes, not read); the vendor meaning of the
`energy/*` commands (the temperature unit of a `weather.get_forecasts` response
was verified on 2026-10-23, see "Weather forecasts"); the vendor meaning of the
Forecast.Solar `watt_hours_period` keys (start vs end of the period, API
unreachable); the HA error for an unknown Modbus hub name; whether a plain Modbus
hub can be added in the UI.
Loading
Loading