Add Matrix Beacon controls and SendSpin visualizer support

README.md

@@ -87,6 +87,9 @@ The client only needs a browser and a network connection to the CoreView server.
 Matrix Beacons are a second, purpose-built target type. They receive a compact
 MQTT scene description and render it locally, so they do not need a browser,
 the Home Assistant native API, or a direct connection to CoreView.
+The reference HUB75 S3 profile can also render a Matrix-only Audio Reactive
+scene from its onboard microphones; this stays on the device and never carries
+raw room audio through CoreView.
 
 ---
 

docs/matrix-mqtt-protocol.md

@@ -14,6 +14,7 @@ With `MQTT_TOPIC_ROOT=home/signage` and a target ID of `kitchen-matrix`:
 | CoreView → target | `home/signage/matrix/kitchen-matrix/event` | no |
 | target → CoreView | `home/signage/matrix/kitchen-matrix/status` | yes |
 | target → CoreView | `home/signage/matrix/kitchen-matrix/ack` | no |
+| target → CoreView | `home/signage/matrix/kitchen-matrix/control` | no |
 
 The target must only subscribe to its own `state` and `event` topics. Broker
 ACLs should likewise limit its publish rights to its own `status` and `ack`
@@ -67,17 +68,51 @@ Notification scenes may also include an optional `icon` (`info`, `warning`,
 These are presentation hints: clients that do not support them can render the
 same title, detail, and severity without loss of meaning.
 
-Music scenes carry a `mode` (`spectrum`, `meter`, or `pulse`) and a
+Music scenes carry a `mode` (`spectrum`, `meter`, `pulse`, or `local`) and a
 `visualizer` object containing normalized 0–255 `bands`, `level`, `peak`, and
 an `active` flag. They may include now-playing `title` and `detail`. This is
 live display data only; CoreView never sends or relays audio.
 
+`local` is a Matrix-only capability. A target that advertises `localAudio`
+uses its onboard microphone/audio hardware to generate bands locally and
+ignores the remote `visualizer` values. The reference HUB75 S3 profile retains
+no samples and publishes no raw audio; it is intentionally a room-audio effect,
+not a network audio source.
+
 `theme` is the effective CoreView Theme after normal assignment, schedules, and
 manual overrides are resolved. All colors are six-digit CSS hex values and
 `brightness` is an integer from 1 through 255. Clients that do not implement
 theme support may ignore this object. Effects use `theme.effectPalette`, so a
 theme change restyles both normal scenes and local animations consistently.
 
+## Home Assistant / ESPHome control surface
+
+The reference firmware exposes a small Home Assistant control card through
+ESPHome: Matrix mode, effect, palette, brightness, speed, intensity, device
+restart, and health sensors. Those entities do **not** mutate the renderer
+directly. A user change publishes this non-retained request:
+
+```json
+{
+  "schema": "coreview.matrix.control.v1",
+  "control": "effect",
+  "value": "Digital Rain"
+}
+```
+
+CoreView validates the request, persists the target's Matrix control state,
+and republishes a new retained `state` revision. The target reflects that state
+back into its ESPHome entities without emitting another request. This keeps
+CoreView, Themes, alerts, and Home Assistant from competing for device state.
+
+Supported controls are `mode` (`Display`, `Lights`, `Music`), `effect`,
+`palette`, `brightness`, `speed`, `intensity`, `scanner_text`, and
+`bouncing_text`. The two text values are single words limited to 16 characters
+and are used only by their matching effect. `Lights` selects one of the
+portable firmware effects; `Music` enables the Matrix's local microphone
+visualizer. CoreView notifications continue to take priority over this manual
+control surface.
+
 Transient events use `coreview.matrix.event.v1`, carry a UUID `eventId`, a
 `kind` of `notification` or `clear`, an optional RFC 3339 `expiresAt`, and a
 severity (`info`, `success`, `warning`, `critical`). Events do not replace the

esphome/README.md

@@ -60,8 +60,18 @@ Create a profile for other boards, pinouts, panel chains, or geometries.
   Matrix Effect widget.
 - Notification/alert presentation with optional standard icons and flashing
   border, plus transient ticker support.
-- SendSpin Music Visualizer scenes: local spectrum bars, level meter, and
-  pulse modes driven by normalized live data from CoreView (never audio).
+- SendSpin Music Visualizer scenes: spectrum bars, level meter, and pulse
+  modes driven by normalized live data from CoreView (never audio).
+- A Home Assistant-native control card exposed by ESPHome: Matrix mode,
+  lighting effect, palette, brightness, speed, intensity, Scanner/Bouncing
+  Text words, restart, Wi-Fi, uptime, scene revision, and online status.
+  Controls request changes from
+  CoreView over MQTT; they do not bypass Themes or retained state.
+- The bundled RGB Matrix HUB75 S3 hardware profile also supports the board's
+  ES7210-connected microphones for the Matrix-only **Audio Reactive** widget.
+  It analyzes room sound locally into eight display bands and immediately
+  discards the samples—no microphone audio is published, stored, or sent to
+  CoreView. Use automation to select that profile when music is playing.
 
 CoreView sends scenes and effect parameters, not a pixel stream. That keeps
 the MQTT traffic small and lets the firmware remain responsive on different

esphome/components/coreview_audio_reactive.h

@@ -0,0 +1,97 @@
+#pragma once
+
+#include "esphome.h"
+
+#include <array>
+#include <cmath>
+#include <cstdint>
+
+// A deliberately small local audio analyzer for the Matrix Beacon. It takes
+// 16-bit microphone samples, derives sixteen visual bands, and discards the
+// samples immediately. This is not a recorder or an audio transport.
+class CoreViewAudioReactive {
+ public:
+  explicit CoreViewAudioReactive(esphome::microphone::Microphone *microphone) : microphone_(microphone) {}
+
+  void setup() {
+    this->microphone_->add_data_callback([this](const std::vector<uint8_t> &data) { this->consume_(data); });
+    this->microphone_->start();
+  }
+
+  bool is_active() const { return this->has_samples_; }
+  uint8_t level() const { return this->level_; }
+  uint8_t band(uint8_t index) const { return this->bands_[std::min<uint8_t>(index, this->bands_.size() - 1)]; }
+
+ protected:
+  static constexpr size_t WINDOW_SIZE = 256;
+  static constexpr float SAMPLE_RATE = 16000.0f;
+  static constexpr float PI = 3.14159265358979323846f;
+  static constexpr std::array<float, 16> CENTERS = {
+      70.0f, 100.0f, 150.0f, 220.0f, 320.0f, 450.0f, 640.0f, 900.0f,
+      1250.0f, 1750.0f, 2400.0f, 3200.0f, 4200.0f, 5200.0f, 6200.0f, 7000.0f};
+
+  void consume_(const std::vector<uint8_t> &data) {
+    for (size_t offset = 0; offset + 1 < data.size(); offset += 2) {
+      const int16_t sample = static_cast<int16_t>(static_cast<uint16_t>(data[offset]) |
+                                                   (static_cast<uint16_t>(data[offset + 1]) << 8));
+      this->samples_[this->sample_count_++] = sample;
+      if (this->sample_count_ == WINDOW_SIZE) {
+        this->analyze_();
+        this->sample_count_ = 0;
+      }
+    }
+  }
+
+  void analyze_() {
+    float rms_sum = 0.0f;
+    for (int16_t sample : this->samples_) {
+      const float normalized = static_cast<float>(sample) / 32768.0f;
+      rms_sum += normalized * normalized;
+    }
+    const float rms = std::sqrt(rms_sum / WINDOW_SIZE);
+    this->peak_ = std::max(rms, this->peak_ * 0.985f);
+    const float divisor = std::max(0.015f, this->peak_);
+    const float normalized_level = std::min(1.0f, std::max(0.0f, (rms - 0.008f) / divisor));
+    const uint8_t next_level = static_cast<uint8_t>(normalized_level * 255.0f);
+    this->level_ = static_cast<uint8_t>((this->level_ * 3U + next_level) / 4U);
+
+    for (size_t band = 0; band < CENTERS.size(); band++) {
+      const float coefficient = 2.0f * std::cos(2.0f * PI * CENTERS[band] / SAMPLE_RATE);
+      float q0 = 0.0f;
+      float q1 = 0.0f;
+      float q2 = 0.0f;
+      for (int16_t sample : this->samples_) {
+        q0 = static_cast<float>(sample) + coefficient * q1 - q2;
+        q2 = q1;
+        q1 = q0;
+      }
+      const float magnitude = std::sqrt(std::max(0.0f, q1 * q1 + q2 * q2 - coefficient * q1 * q2)) / WINDOW_SIZE;
+      // Room microphones and the Matrix enclosure naturally roll off higher
+      // frequencies. Do not hard-gate individual bands: quiet treble should
+      // remain visible while music is playing, rather than vanishing outright.
+      const float compensated = magnitude * (1.0f + static_cast<float>(band) * 0.85f);
+      const float band_value = std::min(1.0f, compensated / 1450.0f);
+      const uint8_t next_band = static_cast<uint8_t>(band_value * 255.0f);
+      this->bands_[band] = static_cast<uint8_t>((this->bands_[band] * 2U + next_band) / 3U);
+    }
+    this->has_samples_ = true;
+    const uint32_t now = millis();
+    if (now - this->last_log_ms_ >= 1000U) {
+      this->last_log_ms_ = now;
+      ESP_LOGD("coreview.audio", "rms=%.4f level=%u bands=%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u", rms, this->level_,
+               this->bands_[0], this->bands_[1], this->bands_[2], this->bands_[3], this->bands_[4],
+               this->bands_[5], this->bands_[6], this->bands_[7], this->bands_[8], this->bands_[9],
+               this->bands_[10], this->bands_[11], this->bands_[12], this->bands_[13], this->bands_[14],
+               this->bands_[15]);
+    }
+  }
+
+  esphome::microphone::Microphone *microphone_;
+  std::array<int16_t, WINDOW_SIZE> samples_{};
+  std::array<uint8_t, 16> bands_{};
+  size_t sample_count_{0};
+  float peak_{0.015f};
+  uint8_t level_{0};
+  bool has_samples_{false};
+  uint32_t last_log_ms_{0};
+};

esphome/hardware/esp32s3-hub75-3x2-32x16.yaml

@@ -13,6 +13,48 @@ psram:
   mode: octal
   speed: 80MHZ
 
+# The RGB Matrix HUB75 S3 board provides dual microphones through an ES7210
+# audio ADC. This is intentionally local-only: the analyzer turns room sound
+# into compact display bands and never publishes or records raw audio.
+i2c:
+  sda: GPIO1
+  scl: GPIO2
+  scan: false
+
+audio_adc:
+  - platform: es7210
+    id: coreview_es7210
+    sample_rate: 16000
+    bits_per_sample: 16bit
+    mic_gain: 30db
+
+i2s_audio:
+  - id: coreview_audio_bus
+    i2s_mclk_pin: GPIO38
+    i2s_bclk_pin: GPIO48
+    i2s_lrclk_pin: GPIO21
+
+microphone:
+  - platform: i2s_audio
+    id: coreview_microphone
+    i2s_audio_id: coreview_audio_bus
+    adc_type: external
+    # The ES7210's SDOUT is connected to GPIO47. GPIO14 is the reverse
+    # direction for the board's output codec.
+    i2s_din_pin: GPIO47
+    channel: right
+    sample_rate: 16000
+    bits_per_sample: 16bit
+    correct_dc_offset: true
+
+esphome:
+  on_boot:
+    priority: -90
+    then:
+      - lambda: |-
+          id(coreview_audio_reactive) = new CoreViewAudioReactive(id(coreview_microphone));
+          static_cast<CoreViewAudioReactive *>(id(coreview_audio_reactive))->setup();
+
 display:
   - platform: hub75
     id: coreview_matrix
@@ -288,7 +330,12 @@ display:
         // reserve the bottom lane for it and move the metadata line above.
         const int text_y = id(coreview_ticker).empty() ? height - 11 : height - 20;
         const int chart_height = std::max(5, text_y - 2);
-        if (!id(coreview_music_active)) {
+        const bool local_audio = id(coreview_music_mode) == "local";
+        const auto *audio_reactive = static_cast<CoreViewAudioReactive *>(id(coreview_audio_reactive));
+        const bool audio_active = local_audio && audio_reactive != nullptr
+          ? audio_reactive->is_active()
+          : id(coreview_music_active);
+        if (!audio_active) {
           print_line("WAITING FOR AUDIO", chart_height / 2 - 4, secondary);
         } else if (id(coreview_music_mode) == "pulse") {
           const int radius = std::max(2, (std::min(width, chart_height) * (70 + id(coreview_music_level) * 30 / 255)) / 200);
@@ -307,7 +354,16 @@ display:
           const int chart_width = bar_count * bar_width + (bar_count - 1) * gap;
           const int offset_x = (width - chart_width) / 2;
           for (int bar = 0; bar < bar_count; bar++) {
-            const int bar_height = std::max(1, chart_height * music_band(bar) / 255);
+            const int raw_band_level = local_audio && audio_reactive != nullptr
+              ? audio_reactive->band(bar * 16 / bar_count)
+              : music_band(bar);
+            // Room microphones have substantially less energy than a direct
+            // digital visualizer feed. Expand local bands for this 32-pixel
+            // Matrix without changing the SendSpin scale.
+            const int band_level = local_audio
+              ? std::min(255, raw_band_level * 5)
+              : raw_band_level;
+            const int bar_height = std::max(1, chart_height * band_level / 255);
             const int x = offset_x + bar * (bar_width + gap);
             it.filled_rectangle(x, chart_height - bar_height, bar_width, bar_height, accent);
           }