Configurable state precision and last_update attribute mode to reduce recorder/exporter write volume

[seaosair]

PR: State precision (rounding) + configurable last_update attribute mode, to cut recorder/exporter write volume

Summary

Purpose: reduce recorder/exporter write volume by reducing noise in the data (rounding)
and configuring the frequency of emitting the last update attribute.

This allows for a significant reduction in data volume logged to InfluxDB -
without sacrificing data quality and details.

Two complementary, independently-configurable options-flow features that together
address a large, measured source of redundant recorder/InfluxDB writes on
multi-module installations:

• State precision — optional per-physical-quantity rounding of float sensor
  states (voltage, current, power, temperature, percent, energy), applied at
  the entity boundary. Default off per group = bit-identical old behaviour.
• Last Update Attribute — a configurable mode (raw / quantized / off)
  for the last_update timestamp attribute PyTap exposes on every module
  entity, which today is set on every coordinator update and therefore
  forces a write on every poll even when the measured value hasn't changed.
  quantized writes it once per raster period (sparse write-once) instead of
  on every reading; off removes it entirely. Default raw = current
  behaviour, unchanged.

Why these two are in one PR: they were designed, deployed and measured
together on a 23-module production installation, and the measurements below
show they are not really independent — rounding alone has negligible effect,
and only delivers its benefit once the attribute fix is also active (see
"Why bundling isn't arbitrary" below).

Root cause

Reconstructed from the running source:

• coordinator.py: "last_update": now.isoformat() — a microsecond ISO
  timestamp, re-set on every processed reading and attached to every module
  entity through extra_state_attributes.
• Home Assistant emits state_changed — and the recorder writes — when
  anything on the entity changes, attributes included. The ever-changing
  timestamp guarantees a write on every coordinator update, even when the
  (rounded) measured value is identical, and each such write carries the
  two timestamp fields (last_update + last_update_str) in addition to the
  value field.
• Two dead ends were ruled out: there is no force_update in the code, and
  the InfluxDB integration's ignore_attributes does not change whether a
  write happens — it only filters what gets serialised afterwards. HA's
  _unrecorded_attributes behaves the same way.

Float sensor states also carry raw sub-decimal noise (e.g. 34.4999961853)
inherited from the underlying protocol math, which by itself changes on
almost every reading regardless of any real change in the panel.

Feature 1: State precision

• New options-flow step "Change state precision", between Reporting and
  Add module: six dropdowns by physical quantity (Voltage, Current, Power,
  Temperature, Percent, Energy — voltage_in/voltage_out share one
  setting), each with identical levels off · 10 · 1 · 0.1 · 0.01 · 0.001
  (round() semantics, decimals -1..3), with a worked example in the label
  (e.g. "0.01 → 48.77 V"). Default off for every group — key absent from
  entry.data → non-breaking, no config migration.
• Rounding happens only at the entity/state boundary
  (_apply_state_precision in sensor.py's _handle_coordinator_update
  paths for both per-module and aggregate entities): aggregates sum the
  raw coordinator data, so rounding a module's displayed value never
  propagates a rounding error into string/installation totals; one setting
  per physical quantity applies to both the module entity and its
  aggregate(s).
• Integer sensors (rssi, readings_today) are deliberately excluded from
  the key→group map and are never rounded. dc_dc_duty_cycle keeps its
  existing ×100 conversion, then rounds. Applies in both reporting modes
  (live and interval/averaged).
• Robustness: the stored {group: decimals} dict is validated on load —
  unknown groups or non-numeric values are dropped with a warning (not a
  crash), decimals are clamped to [-1, 3], and a non-dict value is treated
  as off.
• Dialog help text explains the effect on write volume and recommends
  starting points (voltage 0.1, current 0.01, power 1, temperature 0.1,
  percent 0.1, energy 0.1) — constant relative resolution in the ~0.1–0.5 %
  range, targeting the sub-percent current/voltage jitter that otherwise
  changes on nearly every poll.
• Precision and attribute-mode changes both take effect immediately via
  add_update_listener → async_reload — no HA restart needed.

Feature 2: Last Update Attribute mode

See the design notes, implementation details (including the push-boundary
subtlety for sparse quantized mode), and full measurements already written
up in pytap-pr-text-v1_1.md — unchanged, reproduced in full below for a
single self-contained PR body.

• raw (default) — current behaviour, byte-for-byte unchanged (non-breaking)
• quantized — the timestamp is floored to an n-second grid and written
  once per grid period per module (sparse write-once); the attribute is
  omitted in between
• off — attribute not exposed at all

The write-trigger logic is factored into a small Home-Assistant-free module,
last_update.py, unit-testable in isolation (compute_last_update,
normalize_quantization). The sparse decision is made at the actual push
boundary (per barcode updated since the last live push, or per module in an
averaged-mode flush) rather than per reading — the naive per-reading approach
silently drops the emission in both live mode (chunked socket reads collapse
multiple readings of a module to the last one before pushing) and
averaged/interval mode (the flush inherits last_update from the last
buffered reading). This was found live, not by inspection — see
pytap-pr-text-v1_1.md for the full bug writeup and the regression test
(PushBoundaryCadence) it produced.

Why bundling isn't arbitrary — the measurements show they interact

All measured on the same 23-module production installation, from raw
InfluxDB field counts over matched windows (the write rate here is
configuration-driven — polling ceiling × sensor count — not weather- or
time-of-day-dependent; a linear weather correction used in an early pass was
dropped once the hourly profile showed the rate barely moves between ~4.3 kW
and ~0.5 kW of panel output).

1. Rounding alone does essentially nothing. With the attribute still
raw (unchanged), a matched-window comparison of rounding off vs. on showed
a change within measurement noise (≈ −2 %). The write trigger is the
ever-changing timestamp attribute, not the value — so stabilising the value
alone doesn't stop the write.

2. The attribute fix alone, net effect (raw → quantized/60 s,
rounding held constant, matched 08–10 UTC windows):

Window	value points	Avg. plant power
A: raw, rounding on	179,240	4,348 W
B: quantized/60 s, rounding on	125,625	4,467 W

−29.9 % raw at +2.8 % power → ≈ −32 % power-normalised.

3. off is the single largest lever on its own (attribute removed
entirely), matched 20-minute windows, counting all fields per write
(value + last_update + last_update_str, verified 1:1:1 per entity — a
value-only count understates storage by that factor):

State	Fields	/ min
live + attribute raw	152,787	7,639
live + attribute off	61,833	3,092

−59.5 % fields (2.47×) — bigger than trigger-quantisation and value
rounding combined.

4. Rounding, with the attribute fix active (the actual interaction):
with the fix active, off (no rounding) produces more writes than any
rounded configuration — i.e. rounding only has an effect once the
attribute fix is active, confirming #1. A blunt 0-decimal rounding reached
up to −54 % (on the value field) but asymmetrically — current x.xx→x costs
~6 % relative resolution, power barely any — so the recommended
constant-relative-resolution precision above was chosen instead, keeping
most of the reduction while preserving the current/voltage precision that
actually matters.

5. Sparse quantized (v2.3.2 refinement — write-once, not dense).
Over a 5-minute window at 60 s grid with ~10 s polling:

Field	Writes / 5 min
value	9,788
last_update	1,584
last_update_str	1,584

value is unchanged (every reading is still a poll); the timestamp fields
drop to 9,788 / 1,584 ≈ 6.18 — once per grid period per module. Against a
dense baseline (timestamp on every write, same window): timestamp-field
writes −83.8 % (3,168 vs 19,576), total field volume for these entities
−55.9 % (12,956 vs 29,364). off alone would be −66.7 %; sparse therefore
captures ~84 % of the off saving while retaining a once-per-period
timestamp for outage detection.

Note on a remaining, related overhead (not addressed by this PR)

After last_update is disabled/quantized, a residual string overhead
remains, traced to last_reset_str on the total-type sensors (daily_energy,
readings_today at module/string/installation level). This only changes
once per day (at the counter reset), so it does not trigger writes — it is
dead weight carried on every write of those entities. It is inherent to HA's
total state_class semantics (the Energy Dashboard/statistics engine uses
last_reset to detect resets), so it is best handled at the exporter level
(recorder / InfluxDB ignore_attributes) rather than in integration code, to
avoid breaking the Energy Dashboard. Mentioned here only for completeness.

Compatibility & tests

• Both features default to their pre-existing behaviour bit-for-bit
  (state_precision absent → off per group; last_update_mode absent →
  raw); no config migration for either.
• State precision applies in both reporting modes; the last_update sparse
  logic accounts for both live (per-chunk push) and averaged
  (per-flush) reporting.
• Stored config validated for both: unknown precision groups / non-numeric
  values dropped with a warning (decimals clamped to [-1, 3]); unknown
  last_update_mode values rejected, quantization seconds clamped to
  [5, 300] and interval-normalised (≥ 2× in averaged mode).
• 38 unit tests total, each isolated to pure logic as much as the
  underlying code allows: last_update.py is fully Home-Assistant-free, so
  its tests run standalone with plain unittest. sensor.py's precision
  helpers still require Home Assistant to be importable (they're not
  HA-free), so those tests run under the project's standard
  pytest-homeassistant-custom-component setup instead:
  • tests/test_last_update.py — 15 tests against last_update.py's pure
    helpers: raw reproduces the original isoformat() (regression); off
    always None; grid flooring matches the previous dense formula; the
    sparse write-once cadence (emit / suppress / next cell); exactly one
    emission per cell over a reading stream; per-module independence; the
    push-boundary cadence regression; and quantization normalisation
    (live clamp, averaged multiples, the ≥2× floor, max-cap precedence).
  • tests/test_state_precision.py — 23 tests against sensor.py's
    precision helpers: pass-through behaviour for off/non-float values;
    correct rounding at every supported decimal level including the -1
    (tens) case; every _PRECISION_GROUP_BY_KEY mapping (including
    deliberately-unmapped integer sensors always returning None);
    config validation (missing key, non-dict, unknown group dropped,
    non-coercible value dropped, numeric-string/int/float coercion, clamping
    above/below range); and an explicit regression that an empty/missing
    config is bit-identical to pre-feature behaviour.

Notes

Changes were developed with AI assistance; The design decisions, code review, testing and
live validation were performed by me on a 23-module production installation,
including the matched-window measurements above and confirming both the
rounding/attribute interaction and the sparse write-once cadence live in
InfluxDB.

[Copilot]

🟡 Changes recommended

There are correctness/robustness issues in precision rounding behavior and config validation/error-handling that should be fixed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces two independently configurable mechanisms to reduce Home Assistant recorder/exporter write volume for PyTap entities: (1) optional per-physical-quantity rounding of float sensor states at the entity boundary, and (2) a configurable last_update attribute mode (raw / quantized / off) to avoid attribute-driven writes on every coordinator update.

Changes:

• Add state-precision (rounding) configuration via options flow and apply rounding at sensor update time.
• Add a Home-Assistant-free last_update helper module and integrate sparse/quantized last_update emission into the coordinator.
• Add unit tests covering both rounding helpers and sparse last_update cadence/normalization.

File summaries

File	Description
custom_components/pytap/config_flow.py	Adds options-flow steps for state precision and last_update mode/quantization.
custom_components/pytap/const.py	Introduces constants for state precision groups and last_update configuration.
custom_components/pytap/coordinator.py	Implements quantized/off last_update behavior and sparse emission at push boundaries.
custom_components/pytap/last_update.py	Adds pure helper functions for computing/normalizing last_update behavior.
custom_components/pytap/sensor.py	Applies configured rounding to entity native values at the state boundary.
custom_components/pytap/manifest.json	Bumps integration version.
custom_components/pytap/strings.json	Adds UI strings for new options flow steps.
custom_components/pytap/translations/en.json	Adds English translations for the new options flow steps and descriptions.
tests/test_last_update.py	Adds unit tests for last_update helper logic and sparse cadence.
tests/test_state_precision.py	Adds unit tests for state precision mapping/validation/rounding helpers.

Review details

Suppressed comments (3)

custom_components/pytap/sensor.py:703

• Aggregate performance is pre-rounded to 2 decimals before applying configurable state precision, which prevents users from selecting 3-decimal precision for the percent group and makes precision settings inconsistent across sensors.

            if total_peak_power > 0:
                self._attr_native_value = _apply_state_precision(
                    round((total_power / total_peak_power) * 100.0, 2),
                    self._state_precision,
                )

custom_components/pytap/config_flow.py:579

• current_quant = int(...) can raise on invalid stored data, which would prevent the options form from rendering. It’s safer to catch conversion errors and fall back to the default (and optionally normalize for the current reporting mode).

        current_mode = data.get(CONF_LAST_UPDATE_MODE, DEFAULT_LAST_UPDATE_MODE)
        current_quant = int(
            data.get(
                CONF_LAST_UPDATE_QUANTIZATION, DEFAULT_LAST_UPDATE_QUANTIZATION
            )
        )

custom_components/pytap/coordinator.py:150

• last_update_quantization is only parsed as a positive int, but it is not clamped/normalized (min/max bounds, and the averaged-mode multiple-of-write-interval rule). This can make coordinator behavior diverge from what the options flow enforces and can break assumptions in sparse quantized mode.

        try:
            self._last_update_quantization: int = int(
                entry.data.get(
                    CONF_LAST_UPDATE_QUANTIZATION,
                    DEFAULT_LAST_UPDATE_QUANTIZATION,
                )
            )
        except (TypeError, ValueError):
            self._last_update_quantization = DEFAULT_LAST_UPDATE_QUANTIZATION
        if self._last_update_quantization <= 0:
            self._last_update_quantization = DEFAULT_LAST_UPDATE_QUANTIZATION

• Files reviewed: 10/10 changed files
• Comments generated: 7
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.