Per-endpoint polling intervals to reduce API token usage

.planning/codebase/INTEGRATIONS.md

@@ -1,206 +1,130 @@
-# External Integrations
-
-**Analysis Date:** 2026-02-01
-
-## APIs & External Services
-
-**Netro Smart Irrigation API:**
-- Service: Netro Public API (NPA) v1
-- What it's used for: Complete control and monitoring of Netro sprinkler controllers and Whisperer sensors
-  - SDK/Client: `requests` library (2.32.5)
-  - Auth: Device serial number as URL parameter (`key={serial}`)
-  - Base URL: `http://api.netrohome.com/npa/v1/`
-
-**Supported Devices:**
-- Netro Sprite
-- Netro Pixie
-- Netro Spark
-- Whisperer soil moisture sensors
-
-## API Endpoints
-
-Location: `Netro Sprinklers.indigoPlugin/Contents/Server Plugin/plugin.py:60-74`
-
-| Endpoint | Method | Purpose |
-|----------|--------|---------|
-| `info.json` | GET | Get device status, zones, model, serial number |
-| `schedules.json` | GET | Get current and upcoming watering schedules |
-| `moistures.json` | GET | Get soil moisture levels per zone |
-| `sensor_data.json` | GET | Get Whisperer sensor readings (moisture, temp, sunlight) |
-| `water.json` | POST | Start watering zone(s) with optional delay/schedule |
-| `stop_water.json` | POST | Stop all watering |
-| `set_status.json` | POST | Set standby mode on/off |
-| `no_water.json` | POST | Set rain delay (N days no watering) |
-| `report_weather.json` | POST | Report local weather for AI scheduling |
-| `zone/start` | PUT | Start specific zone (alternative endpoint) |
-
-## Rate Limiting
-
-**API Quota:**
-- Daily limit: 2,000 API calls per day
-- Reset time: Midnight UTC (stored in `meta.token_reset`)
-- Remaining quota: Tracked in `meta.token_remaining`
-
-**Plugin Rate Limit Management:**
-- Location: `plugin.py:91-98, 195-335`
-- Throttle enforcement: 61-minute backoff on HTTP 429 or API error code 3
-- Throttle state: `self.throttle_next_call` datetime
-- Automatic reset: Clears when expiry time passes
-- Default polling (3 min): ~480 calls/day (safe)
-- Aggressive polling (1 min): ~1440 calls/day (risky)
-
-**Error Handling:**
-- HTTP 429 responses trigger `ThrottleDelayError` exception
-- Netro API error code 3 (rate limit) parsed from JSON response
-- Fallback to 61-minute delay if reset time cannot be parsed
-- Logs warning and fires "rateLimitExceeded" trigger
-
-## Data Storage
-
-**In-Memory Caching:**
-- `self.person` - Cached device data from last API call
-  - Structure: `{"id": serial, "devices": [...]}`
-  - Updated every poll cycle in `_update_from_netro()`
-- `self.netro_devices` - Flattened list of cached devices
-- `self.key_val_list` - Cached Whisperer sensor readings
-
-**Indigo Device State Storage:**
-- Device states for sprinkler controllers:
-  - `id` - Controller serial number
-  - `status` - ONLINE/OFFLINE
-  - `activeZone` - Currently watering zone (1-16)
-  - `activeSchedule` - Current schedule type (Smart, Manual, etc.)
-  - `nextScheduleTime` - Next scheduled watering timestamp
-  - `nextScheduleZone` - Next zone name
-  - `token_remaining` - API calls left today
-  - `zone_1_moisture` through `zone_16_moisture` - Per-zone moisture %
-  - `paused`, `scheduleModeType`, `model`, `api_version` - Metadata
-
-- Device states for Whisperer sensors:
-  - `sensorValue` - Current moisture percentage
-  - `humidity` - Soil moisture
-  - `temperature` - Celsius reading
-  - `soilTemperature` - Same as temperature
-  - `sunlight` - Lux reading
-  - `batteryLevel` - Battery percentage
-  - `readingTime`, `readingLocalDate`, `readingLocalTime` - Timestamps
-  - `token_remaining`, `token_reset` - API quota info
-
-**Databases:**
-- None used - pure API integration with in-memory caching
-
-**File Storage:**
-- None - all configuration in Indigo database
-
-**Caching:**
-- In-memory only, no persistence across plugin restarts
-- Cache refreshed every polling interval (default 3 minutes)
-
-## Authentication & Identity
-
-**Auth Provider:**
-- Custom: Serial number-based authentication
-- Implementation: Device serial number passed as `key` parameter in all API calls
-  - Query parameter format: `?key={serial}` for GET requests
-  - JSON body format: `{"key": "{serial}", ...}` for POST/PUT requests
-- Serial number location: Device configuration (set by user during setup)
-- Serial number source: Found on physical device or Netro mobile app Settings
-
-**Security Considerations:**
-- Serial numbers are not secret - Netro treats them as public
-- Serial number grants full API access to controller
-- No bearer tokens, API keys, or OAuth used
-- Plugin requires active internet connection to api.netrohome.com
-
-## Monitoring & Observability
-
-**Error Tracking:**
-- None (no external error tracking service)
-- All errors logged to Indigo Event Log via `self.logger`
-
-**Logging:**
-- Framework: Indigo's built-in logging (`indigo.PluginBase.logger`)
-- Levels used:
-  - `logger.debug()` - Verbose output, API call details
-  - `logger.info()` - Normal operation, state changes
-  - `logger.warning()` - Rate limit warnings, token low
-  - `logger.error()` - API errors, validation failures
-  - `logger.exception()` - Full stack traces (via traceback format)
-- Destination: Indigo Event Log (viewable in Indigo UI)
-
-**Monitoring Points:**
-- API token counts logged at polling (tokens <50: error, <200: warning, <500: info)
-- Connection errors logged once, then silent retries
-- Throttle state displayed in error messages with retry time
-- Device online/offline status updated every poll
-
-## Webhooks & Callbacks
-
-**Incoming:**
-- None - plugin uses pull model (polling) not push (webhooks)
-
-**Outgoing:**
-- None - Netro API does not provide webhook support
-- Plugin can send data via `report_weather.json` endpoint but does not receive callbacks
-
-**Internal Callbacks (Indigo Framework):**
-- Location: `plugin.py:1162-1233`
-- `deviceStartComm()` - Triggers initial API update when device enabled
-- `deviceStopComm()` - Called when device disabled
-- `triggerStartProcessing()` - Called when Indigo trigger enabled
-- `triggerStopProcessing()` - Called when Indigo trigger disabled
-- `closedPrefsConfigUi()` - Applies config changes without plugin restart
-
-## Events & Triggers
-
-**Plugin-defined Triggers** (defined in `Events.xml`):
-- `sprinklerError` - Zone start, stop, or standby mode failures
-- `commError` - API communication failures
-- `rateLimitExceeded` - API rate limit hit
-- `setNoWater` - Rain delay action failed
-- `setStandbyFailed` - Standby mode action failed
-- `startZoneFailed` - Zone start action failed
-- `stopFailed` - Stop all zones action failed
-- `getScheduleCall` - Schedule fetch failed
-- `personInfoCall` - Device info fetch failed
-- `forecastCall` - Forecast/weather fetch failed
-
-**Trigger Firing:**
-- Location: `plugin.py:1134-1174`
-- Method: `_fireTrigger(event, dev_id=None)`
-- Fired during API errors to enable user automation
-- Example: Alert user when rate limit exceeded
-
-**Standard Indigo Triggers Used:**
-- `RequestStatus` action - Forces immediate API update
-- Sprinkler Zone On/Off actions - Standard Indigo sprinkler control
-
-## Plugin Configuration Flow
-
-1. **User installs plugin** → Indigo loads `Info.plist`
-2. **User configures plugin** → Sets polling interval, timeout, max zone runtime in `PluginConfig.xml` UI
-3. **User creates device** → Specifies Netro controller serial number
-4. **Plugin validates config** → `validateDeviceConfigUi()` and `validatePrefsConfigUi()`
-5. **Device enabled** → `deviceStartComm()` triggers immediate `_update_from_netro()` call
-6. **Concurrent thread starts** → Polls API every N minutes (minimum 3)
-7. **API responses parsed** → Device states updated in Indigo
-8. **User creates actions** → Custom actions for rain delay, weather reporting, zone delay
-9. **User creates triggers** → Event triggers fire on errors
-
-## Environment Configuration
-
-**Required env vars:**
-- None - all configuration via Indigo UI
-
-**Secrets location:**
-- Device serial numbers stored in Indigo device configuration
-- Not in environment variables or config files
-- Stored encrypted by Indigo database
-
-**Connection Testing:**
-- Plugin includes standalone test utility: `docs/test_local_api.py`
-- Useful for debugging API connectivity before plugin integration
+# INTEGRATIONS.md — External API Integrations
+
+## 1. Netro Public API (NPA)
+
+### Versions
+
+Two API versions are supported simultaneously. Per-device version is
+auto-detected from device config: if an `apiKey` prop is set, v2 is used;
+otherwise v1 serial-number auth is used. Detection happens in
+`Plugin._get_device_auth(dev)` in
+`Netro Sprinklers.indigoPlugin/Contents/Server Plugin/plugin.py`.
+
+### API v1
+
+- **Base URL**: `https://api.netrohome.com/npa/v1/`
+- **Auth**: Serial number passed as `?key={serial}` (GET) or `{"key": serial}` (POST body)
+- **Official docs**: https://www.netrohome.com/en/shop/articles/10
+- **Timestamp format**: Millisecond Unix epoch — **but often returned as strings**
+  (see `docs/API_NOTES.md` quirk #1)
+
+### API v2
+
+- **Base URL**: `https://api.netrohome.com/npa/v2/`
+- **Auth**: Per-device 32-char API key, same parameter name `key`
+- **Obtain**: netrohome.com → Account → API Key → Generate (per device)
+- **Official docs**: https://netrohome.com/en/shop/user_guides/7
+- **Timestamp format**: ISO 8601 strings
+- **New in v2**: expanded device statuses, `events.json` endpoint, metric units
+  for weather, `token_limit` field in meta
+
+### Endpoints
+
+All defined as `Final[str]` constants in
+`Netro Sprinklers.indigoPlugin/Contents/Server Plugin/constants.py`:
+
+| Constant | v1 URL | v2 URL | Method |
+|----------|--------|--------|--------|
+| `DEVICE_INFO_ENDPOINT` / `_V2_` | `info.json` | `info.json` | GET |
+| `DEVICE_SCHEDULES_ENDPOINT` / `_V2_` | `schedules.json` | `schedules.json` | GET |
+| `DEVICE_MOISTURES_ENDPOINT` / `_V2_` | `moistures.json` | `moistures.json` | GET |
+| `DEVICE_SENSOR_DATA_ENDPOINT` / `_V2_` | `sensor_data.json` | `sensor_data.json` | GET |
+| `DEVICE_WATER_ENDPOINT` / `_V2_` | `water.json` | `water.json` | POST |
+| `DEVICE_STOP_WATER_ENDPOINT` / `_V2_` | `stop_water.json` | `stop_water.json` | POST |
+| `DEVICE_SET_STATUS_ENDPOINT` / `_V2_` | `set_status.json` | `set_status.json` | POST |
+| `DEVICE_NO_WATER_ENDPOINT` / `_V2_` | `no_water.json` | `no_water.json` | POST |
+| `DEVICE_REPORT_WEATHER_ENDPOINT` / `_V2_` | `report_weather.json` | `report_weather.json` | POST |
+| `DEVICE_SET_MOISTURE_ENDPOINT` / `_V2_` | `set_moisture.json` | `set_moisture.json` | POST |
+| `DEVICE_EVENTS_V2_ENDPOINT` | — | `events.json` | GET (v2 only) |
+
+### Rate Limiting
+
+- **Daily quota**: 2,000 calls/day per device (shared between v1 and v2 keys for the same device)
+- **Reset**: Midnight UTC
+- **HTTP 429**: Rate limit exceeded — plugin enforces a 61-minute lockout
+  (`THROTTLE_LIMIT_MINUTES = 61` in `constants.py`)
+- **Proactive pause**: When `token_remaining < TOKEN_PAUSE_THRESHOLD` (100),
+  polling is suspended before hitting the limit
+- **Warning threshold**: `TOKEN_WARNING_THRESHOLD = 200` — logs a warning
+
+### Polling budgets at default intervals
+
+| Interval | Calls/day | Safety |
+|----------|-----------|--------|
+| 3 min (minimum) | ~480 | Safe |
+| 5 min (events default) | ~288 | Very safe |
+| 10 min (device info/moisture default) | ~144 | Comfortable |
+| 30 min (schedules/sensor default) | ~48 | Very conservative |
+
+### Response envelope
+
+```json
+{
+  "status": "OK",
+  "data": { ... },
+  "meta": {
+    "token_remaining": 1850,
+    "token_reset": "2026-04-08T00:00:00"   // v2 ISO; v1 is Unix timestamp
+  }
+}
+```
+
+Error codes: `1`=invalid key, `3`=rate limit, `4`=invalid device,
+`5`=server error, `6`=parameter error.
+
+### Known API Quirks (see `docs/API_NOTES.md` for full details)
+
+1. Timestamps sometimes returned as strings, not numbers — plugin normalises with
+   `float(raw) if isinstance(raw, str) else raw`
+2. Device info response uses singular `device` key, not `devices` array
+3. `STANDBY` status means offline OR user-set standby (ambiguous)
+4. `zones[].smart` can be boolean (v1) or string enum `SMART`/`ASSISTANT`/`TIMER` (v2)
+5. Moisture data updates once per day maximum
+6. Whisperer sensor reports every 4-6 hours (no force-refresh API)
+7. Weather units differ: v1 uses US units (°F, inches, mph, inHg); v2 uses metric
 
 ---
 
-*Integration audit: 2026-02-01*
+## 2. Tomorrow.io Weather API (optional)
+
+- **Purpose**: Fetch real-time and forecast weather to report to Netro for
+  smarter scheduling decisions
+- **Client**: `TomorrowClient` in
+  `Netro Sprinklers.indigoPlugin/Contents/Server Plugin/tomorrow_client.py`
+- **Auth**: API key configured in plugin prefs (`tomorrowApiKey`)
+- **Location**: Lat/lon string configured in plugin prefs (`tomorrowLocation`)
+- **Endpoints used**:
+  - Realtime weather (current conditions)
+  - Daily forecast (multi-day ahead)
+- **Units**: Tomorrow.io returns metric; client passes metric through. `plugin.py`
+  converts to US units when forwarding to Netro v1 via
+  `convert_weather_metric_to_us()` in `utils.py`
+- **Condition mapping**: Tomorrow.io weather codes (1000–8000) mapped to Netro
+  condition codes (0=Clear, 1=Cloudy, 2=Rain, 3=Snow, 4=Wind) via
+  `_TOMORROW_TO_NETRO_CONDITION` dict in `tomorrow_client.py`
+- **Optional**: Feature disabled if `tomorrowEnabled` pref is False or API key /
+  location not set; `_tomorrow_client` will be `None`
+- **Polling intervals**: Realtime every 30 min (`DEFAULT_WEATHER_UPDATE_INTERVAL_MINUTES`),
+  forecast every 4 hours (`DEFAULT_FORECAST_INTERVAL_MINUTES`)
+
+### v1 vs v2 unit handling for weather
+
+`utils.py` provides bidirectional converters:
+
+| Function | Direction |
+|----------|-----------|
+| `convert_weather_metric_to_us()` | °C→°F, mm→in, m/s→mph, hPa→inHg |
+| `convert_weather_us_to_metric()` | reverse |
+
+Note: v1 Netro API does not accept `t_dew` field — plugin strips it before
+sending to v1 devices.