Feat/add mpc algo

.github/agents/mpc-specialist.agent.md

@@ -0,0 +1,35 @@
+---
+name: "MPC Specialist"
+description: "Use when tuning or debugging the Smart Fan Controller MPC, thermal learning, hysteresis, dead time, disturbance bias, step-down guards, fan-mode profiles, or for French requests like agent MPC, analyse MPC, optimiser le MPC."
+tools: [read, search, edit, execute, todo]
+argument-hint: "Describe the MPC behavior, fan decision, thermal-learning issue, or change to implement."
+user-invocable: true
+---
+You are the Smart Fan Controller MPC specialist. Your job is to analyze, tune, and validate the predictive fan controller for this repository.
+
+## Scope
+- Work first in `custom_components/smart_fan_controller/mpc_controller.py`.
+- Then inspect `custom_components/smart_fan_controller/thermal_learning.py`.
+- Use `custom_components/smart_fan_controller/__init__.py` for control-loop and disturbance gating.
+- Update `tests/test_mpc_controller.py`, `tests/test_learning.py`, and nearby focused tests when behavior changes.
+
+## Constraints
+- Preserve the project vocabulary: error is positive when the system needs more heating or cooling.
+- Do not add second-order slope or parabolic prediction terms.
+- Keep learning-data integrity guards for window-open, defrost, HVAC idle, setpoint-drop cooldown, and insufficiently established periods.
+- Keep per-mode effective slope based on the median of accepted samples.
+- Apply monotone slope enforcement only when all fan-mode profiles are learned.
+- Prefer small, behavior-scoped edits with targeted pytest validation.
+
+## Approach
+1. Start from the deciding code path or failing test nearest to the MPC behavior.
+2. Form one falsifiable local hypothesis before editing.
+3. Make the smallest change that tests the hypothesis.
+4. Run the narrowest relevant pytest file or test.
+5. Report the behavioral impact, validation, and residual risks.
+
+## Output Format
+- Findings or change summary
+- Files inspected or changed
+- Validation run
+- Remaining risks or missing data

.github/copilot-instructions.md

@@ -6,49 +6,59 @@ The integration lives under `custom_components/smart_fan_controller/`. Key modul
 
 | File | Role |
 |------|------|
-| `controller.py` | `SmartFanController` — rule-based decision engine (zones A–F) |
-| `mpc_shadow.py` | `MPCShadowController` — observation-only MPC-lite, never writes fan commands |
+| `mpc_controller.py` | `MPCController` — MPC-lite: learned thermal model, cost-based fan selection, hysteresis guards |
 | `thermal_learning.py` | `ThermalLearning` — slope samples, response-time events, profile calibration |
-| `__init__.py` | HA integration entry point: config, control loop, services |
+| `__init__.py` | HA integration entry point: control loop, learning collection, services |
 | `config_flow.py` | Config and options UI flows |
-| `sensor.py` / `switch.py` | HA entity platforms |
+| `sensor.py` | HA sensor entity platform |
 | `data_collection.py` | CSV logger for offline analysis |
 
 ## Domain Vocabulary
 
 - **error**: always positive when the system needs more heating/cooling (`target - current` in heat, `current - target` in cool)
-- **effective_slope**: `vtherm_slope` (°C/h, EMA-smoothed by Versatile Thermostat)
-- **dead_time**: thermal lag between a fan change and first observable slope response
-- **defrost_active**: heat-pump defrost cycle (auto-detected or via external entity); blocks step-down decisions and learning samples
-- **hvac_idle**: heat-pump compressor is off (detected via operating entity or power consumption); blocks step-up decisions (zones C, D) and learning samples — no cooldown unlike defrost
-- **Zones A–F**: priority-ordered decision rules; zone A = emergency/setpoint-drop, F = comfort hold
-
-## Decision Engine Zones (in priority order)
-
-| Zone   | Condition                                           | Action needed to modify       |
-|--------|-----------------------------------------------------|-------------------------------|
-| A      | `error ≥ hard_error` OR `error < TARGET_DROP (-1°C)` | `controller.py:_decide_zone_a` |
-| B      | Projected overshoot > deadband AND slope changed    | `controller.py:_decide_zone_b` |
-| C      | `error > soft_error`                                | `controller.py:_decide_zone_c` |
-| D      | `0 < error < soft_error`                            | `controller.py:_decide_zone_d` |
-| E      | `error < -deadband`                                 | `controller.py:_decide_zone_e` |
-| F      | `-deadband ≤ error ≤ 0`                             | `controller.py:_decide_zone_f` |
+- **effective_slope**: learned slope per fan mode (°C/h, from `ThermalLearning`); raw slope comes from VTherm EMA
+- **dead_time**: thermal lag between a fan change and first observable slope response (learned via response events)
+- **defrost_active**: heat-pump defrost cycle (via external entity); pauses MPC decisions and learning
+- **hvac_idle**: heat-pump compressor is off (via operating entity); pauses MPC decisions and learning
+- **ctrl_state**: closure dict in `__init__.py` tracking `last_change_time`, `previous_slope`, `defrost` state, `last_setpoint_drop_time`
+
+## Control Flow (MPC-only)
+
+Each cycle (`__init__.py → run_control_loop`):
+
+1. Read climate entity state (temp, target, slope, fan, window)
+2. Detect disturbances (defrost, HVAC idle)
+3. Call `mpc_controller.evaluate()` → returns `mpc_decision` dict
+4. If MPC status is actionable → apply MPC fan choice; otherwise → hold current fan
+5. Collect learning data (slope samples, response events) with gating for phase, defrost, idle, window, setpoint-drop cooldown
+6. Update sensors via `sensor.update_from_mpc(mpc_decision)`
+7. Log to CSV if data collection enabled
+
+## MPC Decision Engine
+
+The MPC controller (`mpc_controller.py`) evaluates all candidate fan modes over a 30-min horizon:
+
+- **Cost function**: comfort error + overshoot penalty + floor violation + mode-change cost + min-interval penalty
+- **Hysteresis**: requires minimum cost improvement before switching (margin scales with proximity to target)
+- **Step-down guards**: blocks downward moves when under target and not established or predicted shortfall
+- **Disturbance bias**: EMA tracker for unmodeled effects (solar, occupancy); decays during paused periods
+- **Monotone constraint**: when all profiles learned, enforces slope(mode_i) ≤ slope(mode_i+1)
+- **Pause conditions**: window open, defrost, HVAC idle → returns "Disturbed" status
 
 ## Test Conventions
 
 - Framework: **pytest** in `tests/`; run with `python -m pytest tests/ -q`
 - Never use `time.sleep`; mock `time.time` via `unittest.mock.patch`
-- Protected-member access (`controller._state`) is normal in tests — add `# pylint: disable=protected-access` at module top
+- Protected-member access (`mpc._state`) is normal in tests — add `# pylint: disable=protected-access` at module top
 - Fixtures that shadow outer names need `# pylint: disable=redefined-outer-name`
-- Every test function and helper needs a docstring
-- HA switch classes only implement `async_turn_on`/`async_turn_off` — add `# pylint: disable=abstract-method` on the class
+- Every test function and helper needs a docstring\n- Test helpers: `_build_learning()` returns `ThermalLearning()`, `_build_mpc(learning)` returns configured `MPCController`
 
 ## Home Assistant Patterns
 
-- Switches inherit `SwitchEntity` and only override async variants; sync `turn_on`/`turn_off` are intentionally omitted
-- `async_write_ha_state()` is called after any state mutation in switch/sensor entities
+- `async_write_ha_state()` is called after any state mutation in sensor entities
 - Config flow uses `vol.Schema` with selectors from `homeassistant.helpers.selector`
 - Entity IDs are built with `build_entity_id()` / `build_unique_id()` from `const.py`
+- Data dict: `hass.data[DOMAIN][entry_id]` contains `"learning"`, `"mpc_controller"`, `"climate_entity"`, `"sensors"`, `"store"`
 
 ## Build & Test
 
@@ -59,16 +69,24 @@ python -m pytest tests/test_X.py -q # run one file
 ./container hassfest                 # validate manifest / translations
 ```
 
+## Dev Container Workflow
+
+- When working inside this repository's provided devcontainer / Docker environment, do **not** spend time creating or configuring a separate Python environment by default.
+- Treat the container runtime and the already available project tooling as the authoritative execution environment.
+- Prefer the existing shell environment and direct test commands such as `pytest tests/ -q` unless the user explicitly asks for Python environment debugging.
+
 ## Key Constants (const.py)
 
 - `THRESHOLD_TARGET_DROP = -1.0` — setpoint-drop trigger (°C)
-- `DEFAULT_DEADBAND`, `DEFAULT_SOFT_ERROR`, `DEFAULT_HARD_ERROR` — all tunable via options flow
+- `DEFAULT_DEADBAND` — tunable via options flow
 - `CONF_DEFROST_ENTITY` — optional entity for external defrost signal
 - `CONF_OPERATING_ENTITY` — optional entity for heat-pump compressor running state
+- `SETPOINT_DROP_LEARNING_COOLDOWN = 30.0` — minutes to suppress learning after a large setpoint drop
+- `MIN_ESTABLISHED_RATIO = 2.0` — multiplier on dead_time; fan mode must be active this long before learning
 
 ## Important Constraints
 
 - **Avoid over-engineering**: only add code that directly addresses the requirement
 - **No second-order slope terms**: VTherm slope is already EMA-smoothed; parabolic projection amplifies noise
-- **Step-limited fan changes**: braking is limited to -1 step; recovery is usually +1 step, but Zone C may step up by +2 when error exceeds `0.75*hard_error`
-- **Learning data integrity**: exclude window-open, defrost, and HVAC idle periods from slope samples and response-time events
+- **Learning data integrity**: exclude window-open, defrost, HVAC idle, setpoint-drop cooldown (30 min), and insufficiently-stable periods (< 2× dead_time) from slope samples; effective slope uses **median** (not mean) for outlier robustness
+- **Monotone constraint**: when all fan-mode profiles are learned, MPC enforces slope(mode_i) ≤ slope(mode_i+1) via isotonic forward pass; partial profiles skip the constraint

.github/skills/analyze-control-data/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: analyze-control-data
-description: "Analyze Smart Fan Controller CSV data files to diagnose algorithm behavior, identify missed MPC opportunities, detect defrost artifacts, and propose improvements. Use when: the user provides a CSV data file; asking to analyze overnight or morning behavior; asking why the fan did or did not change; evaluating MPC shadow accuracy; diagnosing defrost or setpoint-drop events."
+description: "Analyze Smart Fan Controller CSV data files to diagnose algorithm behavior, identify missed MPC opportunities, detect defrost artifacts, and propose improvements. Use when: the user provides a CSV data file; asking to analyze overnight or morning behavior; asking why the fan did or did not change; evaluating MPC decision quality; diagnosing defrost or setpoint-drop events."
 argument-hint: "CSV file path or description of the behavior to diagnose"
 ---
 
@@ -10,7 +10,7 @@ argument-hint: "CSV file path or description of the behavior to diagnose"
 
 - User provides a `smart_fan_controller_data_*.csv` file (from `data/`)
 - User describes unexpected behavior ("fan stayed on med all night", "slow morning rise")
-- User wants to compare MPC shadow decisions vs live decisions
+- User wants to evaluate MPC decision quality
 - Diagnosing defrost artifacts, setpoint drops, or window-open disturbances
 
 ## Column Reference
@@ -25,18 +25,18 @@ argument-hint: "CSV file path or description of the behavior to diagnose"
 | `error` | Signed error (positive = needs action) |
 | `current_fan` | Fan mode at start of cycle |
 | `fan_mode` | Fan mode selected by controller |
-| `reason` | Zone decision reason string |
+| `reason` | MPC decision reason string |
 | `phase` | `DEAD_TIME` / `TRANSIENT` / `ESTABLISHED` |
 | `effective_slope` | Slope actually used for decisions |
 | `is_window_open` | `True`/`False` |
 | `force` | `True` if timer was bypassed |
-| `mpc_shadow_fan_mode` | Fan mode the shadow would choose |
-| `mpc_shadow_status` | `Ready` / `Disabled` / `Setpoint drop` / `Disturbed` |
-| `mpc_shadow_matches_live` | `yes` / `no` / `disabled` |
-| `mpc_shadow_would_change` | `yes` / `no` |
-| `mpc_shadow_cost` | MPC optimization cost |
-| `mpc_shadow_confidence` | Profile coverage % |
-| `mpc_shadow_reason` | Shadow decision explanation |
+| `mpc_status` | `Ready` / `Not ready` / `Setpoint drop` / `Disturbed` / `Idle` / `Low confidence` |
+| `mpc_fan` | Fan mode chosen by MPC |
+| `mpc_would_change` | `yes` / `no` |
+| `mpc_cost` | MPC optimization cost |
+| `mpc_confidence` | Profile coverage % |
+| `mpc_temp_10m` | MPC 10-minute temperature prediction |
+| `mpc_temp_30m` | MPC 30-minute temperature prediction |
 | `defrost_active` | `True` when defrost protection is active |
 | `learning_ready` | `True` when ≥10 samples per profile |
 
@@ -52,41 +52,41 @@ argument-hint: "CSV file path or description of the behavior to diagnose"
 **Setpoint drop events (`reason` contains "Setpoint drop")**
 - Check `target_temp` drop magnitude
 - Verify `fan_mode` went to lowest mode
-- Check MPC shadow also reported "Setpoint drop"
+- Check MPC also reported "Setpoint drop"
 
 **Defrost events (`defrost_active == True`)**
 - Find the slope drop that triggered detection: look for `vtherm_slope` crossing sharply negative
-- Verify zones B/D were blocked (reason should contain "Defrost hold")
-- Check duration: 20-min cooldown from detection to next step-down
+- Verify MPC paused (status should be "Disturbed")
+- Check duration: 20-min cooldown from detection
 
-**MPC divergence periods (`mpc_shadow_matches_live == "no"`)**
-- Identify what shadow chose vs live
-- Check `mpc_shadow_confidence` — low confidence = not enough learning data
-- Check `mpc_shadow_known_profiles` — shadow needs ≥10 samples per mode
+**MPC disturbed periods (`mpc_status == "Disturbed"`)**
+- Identify what triggered the disturbance: defrost, window open, HVAC idle
+- Check `mpc_confidence` — low confidence = not enough learning data
+- Check `mpc_known_profiles` — MPC needs ≥10 samples per mode
 
 **Slow temperature recovery**
 - Plot `current_temp` vs `target_temp` over time
-- Look for unnecessary step-downs (reason "Maintenance: favorable slope" or "Braking")
+- Look for unnecessary step-downs (reason contains "Hysteresis" or step-down hold)
 - Cross-check with `defrost_active` — was defrost correctly detected?
 
 **Night setpoint drop mismatch (MPC on "med" instead of "silent")**
 - Filter to period where `target_temp` dropped significantly
-- Check `mpc_shadow_status` — should be "Setpoint drop" with lowest fan
-- If not, the snapshot was taken before the setpoint-drop fix was deployed
+- Check `mpc_status` — should be "Setpoint drop" with lowest fan
+- If not, check whether defrost or idle paused the MPC
 
 ### 3. Report Format
 
 Provide a narrative structured as:
 1. **Period summary**: time range, hvac mode, temperature trajectory
 2. **Key events**: defrost cycles, setpoint drops, fan changes — with timestamps
-3. **Algorithm assessment**: zones triggered correctly? any missed steps?
-4. **MPC shadow comparison**: agreement rate, key divergences and their cause
+3. **Algorithm assessment**: MPC decisions triggered correctly? any missed steps?
+4. **MPC quality**: confidence levels, key disturbance periods and their cause
 5. **Recommendations**: concrete parameter changes or code fixes if warranted
 
 ## Code Hints
 
 - Load CSV: `pd.read_csv(path, parse_dates=["timestamp"])`
 - Filter defrost: `df[df["defrost_active"] == True]`
 - Find mode changes: `df[df["current_fan"] != df["fan_mode"]]`
-- MPC divergence: `df[df["mpc_shadow_matches_live"] == "no"]`
-- Zone summary: `df["reason"].value_counts()`
+- MPC disturbance: `df[df["mpc_status"] == "Disturbed"]`
+- Reason summary: `df["reason"].value_counts()`