feat(#14): mic record → speaker playback (M5StickS3 hardware bring-up) by junnncct1106 · Pull Request #1 · junnncct1106/zephyr-m5stack-sticks3

junnncct1106 · 2026-06-10T14:12:53Z

What

Implements issue thc1006#14 (microphone record → speaker playback) on the M5StickS3, plus fixes three hardware-level blockers found during serial bring-up on real hardware (ES8311 codec + AW8737 amp + esp32-i2s).

Root causes fixed

ES8311 deferred-init — the codec shares the L3B rail (lcd_power, regulator-boot-on via the M5PM1 PMIC). With no supply/reset link on the node and CONFIG_DEVICE_DEPS off, the driver probed the chip-id over I2C at its POST_KERNEL priority before L3B was powered → -EFAULT "Failed to read chip id" → audio init failed. The node is now zephyr,deferred-init and device_init()'d from audio_init() (main context, L3B up and settled).
I2S TX mem_slab leak — the esp32 i2s driver does not return TX blocks to the slab on DROP from the ERROR state an underrun leaves, so tx_slab leaks empty and the next session's prequeue fails with -EAGAIN (-11) (eventually wedging record too). Recovered with I2S_TRIGGER_PREPARE (the documented ERROR→READY recovery that frees the queues) in loop_tx() and at do_record() / do_play() start; TX writes bounded via loop_tx() (a wedge times out instead of K_FOREVER-hanging the UI); playback is TX-only (full-duplex playback under-ran because the RX read stalled TX); BLOCK_COUNT 4→8 for underrun headroom.
K2 mid-playback guard — rec_page_k2() only blocked page-exit while RECORDING; leaving mid-PLAYING repainted another page's first frame while the audio thread still owned I2S/codec (HW-016e). Now inert in both RECORDING and PLAYING per the Audio demo: record a short mic clip and play it back thc1006/zephyr-m5stack-sticks3#14 button map.

Verified on hardware

Codec init, recording, UI navigation, and I2S slab recovery (no more -EAGAIN / wedge across repeated record→play sessions).

Pending

Final by-ear loudness tuning on the 2011 micro-speaker — gain/level only; the playback path itself is up.

Note for maintainers

The esp32 i2s driver not returning mem_slab blocks on DROP-from-ERROR looks like an upstream Zephyr driver issue and may be worth a separate report.

Test

west build -p always -b m5stack_sticks3/esp32s3/procpu app -- -DEXTRA_CONF_FILE=overlay-audio.conf
west flash

On the REC page: KEY1 to record, KEY1 to play back; repeat to confirm no -EAGAIN and clean page navigation.

… button-mode design Define the mic-record → speaker-playback feature from the requirements side before any code: - Purpose: first end-to-end proof that real captured audio data survives the full round trip (mic→ADC→I2S→RAM→I2S→DAC→amp→speaker), beyond thc1006#6 (meter) and thc1006#3 (synthesised tone). - Functional/quality requirements + platform constraints. - Quantified baseline: default vs audio-build memory; dram0 has ~314 KB free, so a 5 s mono clip (156 KB) fits with margin. Includes verified RMS refs. - Proposed modal REC-page button design (KEY1=action/long=re-record, KEY2=exit) with on-screen labels; short/long-press detection. Agreed: A=>=short phrase (5 s), C=intelligible (digital gain), D=baseline first. B (button/mode) proposed, under review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…elpers Pure-logic primitives for the record->playback path (issue thc1006#14), the inverse pair / companion to the existing capture DSP: - audio_interleave_mono(): expand a recorded mono clip to interleaved stereo for the ES8311/I2S TX path (inverse of audio_deinterleave()). - audio_gain_clip_i16(): Q8 fixed-point gain with int16 saturation, to make a quiet mic recording audible on playback without wrapping (req QR-1). native_sim ztest extended (now 8/8): identity/2x/1.5x gain, +/- saturation, INT16_MIN, gain=0, in-place, mono->stereo round-trip vs deinterleave, plus NULL/0 no-op guards. No hardware; existing signatures unchanged. Part 1/5 of the feature/14-record-playback plan (docs/14_RECORD_PLAYBACK_DESIGN.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add the core mic-record -> speaker-playback engine (issue thc1006#14), all on the existing audio thread so the UI thread never touches I2S (HW-016e): - do_record(): full-duplex capture (amp OFF) of CONFIG_APP_AUDIO_REC_SECONDS into rec_buf, tracking peak RMS; bounded I/O, can't hang on a wedged clock. - do_play(): replays the held clip with the audio_beep() anti-pop amp sequence, applying CONFIG_APP_AUDIO_REC_GAIN_Q8 via audio_gain_clip_i16() and audio_interleave_mono(); short final block is zero-padded. - Thread refactored into a dispatcher: AUDIO-page meter (extracted to meter_session()) takes priority, else service one record/play request. Only one of {meter, record, play} runs at a time, sharing rx_buf/codec safely. - Public API for the UI: audio_record_request/audio_play_request/ audio_rec_get_state/audio_rec_peak/audio_rec_len_ms. UI only reads state. - Kconfig: APP_AUDIO_REC_SECONDS (1..8, default 5) and APP_AUDIO_REC_GAIN_Q8 (256..8192, default 1024 = 4.0x). Builds for m5stack_sticks3/esp32s3/procpu with overlay-audio.conf: dram0 60.1% (+160 KB clip buffer), 155 KB free. Not yet flashed; UI/buttons are Part 3. Part 2/5 of feature/14-record-playback (docs/14_RECORD_PLAYBACK_DESIGN.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…achine Wire the record->playback engine to a new modal PAGE_AUDIO_REC (issue thc1006#14): - pages.h: add PAGE_AUDIO_REC after PAGE_AUDIO (gated by CONFIG_APP_AUDIO). - main.c: on the REC page the two buttons drive a state machine instead of page nav (other pages unchanged): K1 short = record (IDLE) / play (REVIEW) / stop (RECORDING), K1 long-press (>=1 s, decided on release) in REVIEW = re-record, K2 = exit to the next page. A k1_armed guard stops the release of the press that navigated INTO the page from firing a spurious action. REC page also ticks fast (250 ms) for the live countdown/state. - ui.c: render_audio_rec_body() draws READY / RECORDING (SPEAK NOW + countdown) / REVIEW (length, peak, K1 play / hold re-rec) / PLAYING; the page clears on a mode change so screens start clean. UI only READS state, never touches I2S. - audio.c/.h: audio_record_stop_request() + rec_abort for the early-stop. ASCII font => English labels. Builds green both with overlay-audio.conf (dram0 60.1%) and the default no-audio config (unchanged). Hardware test is Part 4. Part 3/5 of feature/14-record-playback (docs/14_RECORD_PLAYBACK_DESIGN.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…hiccup The record loop stopped at the first esp32-i2s RX/TX hiccup, so recordings were ~256 ms of silence (peak_rms=0). The live meter never showed this because the thread re-enters its session each time, masking the same break. do_record now captures across session restarts (drop + restart, keep accumulating into rec_buf) until full or stopped, with a no-progress guard so a dead clock can't spin, and logs the first hiccup's errno + a restarts count. HW-verified on m5stack_sticks3 (MAC 70:04:1d:db:ab:b8): two recordings reach the full 5000 ms with real audio (peak_rms=684/717) and restarts=1 (a one-time startup hiccup, then stable); playback runs the full clip. Evidence: evidence/20260609-hw014-record-playback.log. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…page User feedback: the standalone AUDIO meter page is unnecessary (its only value is confirming the mic hears sound), and the REC screen looked frozen with no ready state. So: - Remove PAGE_AUDIO; fold the live '[####]' mic meter into the REC page's READY and REVIEW screens (draw_mic_meter()), so the device's hearing is always visible and the screen is never static. Pages: HOME/IMU/POWER/REC/DIAG. - Label every state 'state: ready|rec|review|play' (READY shows the live meter). - Run the meter on the REC page only in IDLE/REVIEW; the thread services a record/play request AHEAD of the meter and meter_session() yields on a pending rec_cmd, so pressing K1 on the meter screen records/plays without an I2S tug-of-war (no capture_on toggle race). Builds green: overlay-audio.conf (dram0 60.1%) and the default no-audio config. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…on hints Addresses hardware-test feedback: - No audible playback: the clip had real audio (peak_rms=1555) but was too quiet. Raise codec volume -20 -> -6 dB and the default digital gain 4x -> 8x (CONFIG_APP_AUDIO_REC_GAIN_Q8 1024 -> 2048); both stack for an audible replay. - No on-screen guidance on the PLAYING screen, and to reach the next page: every REC state now shows button hints (K2 labelled 'next page'); PLAYING shows '(auto-ends) / K2 next page'. - Cramped text: LINE_GAP 4 -> 8 px and all REC states lay out on consecutive rows (no skipped lines), so spacing is even. Builds green (overlay-audio.conf + default). Volume pending HW confirm. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Three hardware-level issues blocked record→playback on real M5StickS3 hardware (ES8311 + AW8737 + esp32-i2s), all found via serial bring-up: - ES8311 deferred-init: the codec shares the L3B rail (lcd_power, regulator-boot-on via the M5PM1 PMIC). With no supply/reset link on the node and CONFIG_DEVICE_DEPS off, the driver probed (chip-id read over I2C) at its POST_KERNEL priority BEFORE L3B was powered -> -EFAULT "Failed to read chip id" -> audio init failed. Mark the node zephyr,deferred-init and device_init() it from audio_init() (main context), where L3B is up and settled. - I2S TX mem_slab leak: the esp32 i2s driver does not return TX blocks to the slab on DROP from the ERROR state an underrun leaves, so tx_slab leaks empty and the next session's prequeue fails with -EAGAIN (-11), eventually wedging record too. Recover with I2S_TRIGGER_PREPARE (the documented ERROR->READY recovery that frees the queues) in loop_tx() and at do_record()/do_play() start. Bound do_play()'s TX writes via loop_tx() so a wedge times out instead of K_FOREVER-hanging the UI; play TX-only (full-duplex playback under-ran as the RX read stalled TX); BLOCK_COUNT 4->8 for underrun headroom. - The ES8311 mono ADC duplicates onto both I2S slots (slot index is a no-op); comment corrected. Codec DAC volume to 0 dB; playback make-up gain 4x.

rec_page_k2() only blocked page-exit while RECORDING; leaving mid-PLAYING let the main loop repaint another page's first frame while the audio thread still owned I2S/codec, wedging I2S (HW-016e). Per the thc1006#14 button map K2 is inert in both RECORDING and PLAYING -- guard both.

Copilot

Pull request overview

Implements issue thc1006#14’s end-to-end mic record → RAM → speaker playback bring-up for the M5StickS3, integrating it into the app’s UI as a modal “REC” page and addressing hardware sequencing/reliability issues encountered on real devices (codec power/probe ordering and ESP32 I2S TX queue recovery).

Changes:

Adds a record/playback engine on the audio thread (fixed-length mono buffer, gain+clip on playback, bounded TX writes, ERROR→READY recovery via I2S PREPARE).
Reworks the UI to a dedicated PAGE_AUDIO_REC modal recorder with short/long press behavior and safe navigation guards.
Adds new pure DSP helpers (mono↔stereo interleave + gain/clip) with ztests, plus design/evidence artifacts and new Kconfig knobs.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`tests/drivers/audio_dsp/src/main.c`	Adds ztests for new DSP helpers (interleave + gain/clip).
`evidence/20260609-hw014-record-playback.log`	Adds a short serial log demonstrating repeatable record→play on hardware.
`docs/14_RECORD_PLAYBACK_DESIGN.md`	Adds a requirements/baseline/design write-up for thc1006#14.
`boards/m5stack/m5stack_sticks3/m5stack_sticks3_procpu.dts`	Marks ES8311 as `zephyr,deferred-init` to avoid probing before its rail is up.
`app/src/ui.c`	Replaces the old AUDIO page with a modal REC page UI (state-driven rendering + live meter).
`app/src/pages.h`	Renames the audio page enum to `PAGE_AUDIO_REC`.
`app/src/main.c`	Implements modal button handling for REC page + capture gating while rendering.
`app/src/audio.h`	Adds the public record/playback state machine API for the UI.
`app/src/audio.c`	Implements deferred codec init, slab recovery, record/playback engine, and updated capture behavior.
`app/src/audio_dsp.h`	Declares new DSP helpers used by playback (interleave + gain/clip).
`app/src/audio_dsp.c`	Implements `audio_interleave_mono()` and `audio_gain_clip_i16()`.
`app/overlay-audio.conf`	Sets a default record-playback make-up gain for the audio overlay.
`app/Kconfig`	Adds Kconfig options for clip length and playback gain.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+		cmd = rec_cmd;
+		rec_cmd = REC_CMD_NONE;
+		if (cmd == REC_CMD_RECORD) {
+			do_record();


+ * Mic capture (issue #6). The ES8311 mono ADC duplicates its data onto both I2S
+ * slots, so either slot index works; AUDIO_MIC_SLOT picks one. AUDIO_MIC_FULL is
+ * the empirical full-scale RMS for the PAGE_AUDIO live bar: quiet ~70-100, a
+ * normal voice a few hundred to a few thousand, a loud clap >= 12000, so 2000
+ * gives visible bars for speech without pinning the bar to 4 on every breath.


+		} else if (evt->code == INPUT_KEY_1 && evt->value != 0) {
+			rec_page_k2();
+		}


+Status: **requirements agreed; button/mode design proposed (under review)**; not yet
+implemented. Tracks GitHub issue #14 ("Audio demo: record a short mic clip and
+play it back"). Feature branch: `feature/14-record-playback`.


+- Mic is mono on I2S slot 0; codec fixed at 16 kHz / 16-bit; playback volume
+  −20 dB; the on-board speaker is small/quiet.


Three correctness/robustness fixes from the PR thc1006#15 / #1 Copilot review: - rec_cmd is now an atomic_t consumed with a single atomic_set() exchange (returns the previous value), so a record/play command set by the UI thread can no longer be lost in the gap between the audio thread's separate volatile read and clear. Requests/meter-loop read it atomically. - fill_play_block() snapshots rec_samples and returns 0 when pos >= total, so the unsigned `rec_samples - pos` can never underflow into a huge n / out-of-bounds rec_buf read even if a future caller drops the pos<rec_samples invariant. - main.c disarms K1 long-press tracking when K2 exits the REC page, so a held-K1 release that lands off-page no longer leaves k1_armed set and fires a spurious REC action on the next visit. Also corrects the mic-slot comment in audio.c: the ES8311 mono ADC drives slot 0 with slot 1 silent (HW-016d) — it does not duplicate onto both slots, which contradicted the deinterleave test and could send someone to the silent slot. Builds clean for m5stack_sticks3/esp32s3/procpu with overlay-audio.conf. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Per the Copilot review, the design doc was stale against the merged implementation: - Status header said "not yet implemented"; it is implemented and HW-verified (2026-06-09, dell-0830) — point at the evidence log. - Constraints/baseline hard-coded playback volume at -20 dB; the thc1006#14 path runs the codec at 0 dB and makes up level digitally via CONFIG_APP_AUDIO_REC_GAIN_Q8. Noted the -20 dB figure as the legacy thc1006#3 tone baseline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ate in meter gate Follow-up polish from a deeper self-review (no firmware behaviour change): - audio_dsp gain test now pins the exact contract a plausible rewrite could break: truncation toward zero (-3 * 1.5 -> -4, not floored -5), the exact saturation boundary (v==32767/-32768 not clamped; 32768/-32769 clamped), and INT16_MIN at the max uint16 gain (product stays inside int32, clamps instead of wrapping). 8/8 cases pass on native_sim. - main loop reads audio_rec_get_state() once into a local for the meter gate instead of calling it twice, so the audio thread transitioning state between the two reads can't produce a torn meter decision. Builds clean for m5stack_sticks3/esp32s3/procpu; twister audio_dsp green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Per maintainer review: do_play() ended with i2s_trigger(TX, DRAIN), which has no timeout and was also reached on the TX-timeout break path. If loop_tx() timed out because the TX clock was wedged, the break still hit DRAIN against the stuck clock, hanging the audio thread with the amp still on and rec_state stuck at PLAYING (K2 inert -> the REC page could not be left): a stuck speaker. Remove the DRAIN. The existing fixed k_msleep already waits a bounded time for the pre-queued DMA tail (<= BLOCK_COUNT blocks) to play out, then the stop: path does a wedge-safe I2S_TRIGGER_DROP — the same DROP-not-DRAIN teardown do_record() and meter_session() already use. Normal and error/break paths now both converge on bounded-wait -> DROP, so nothing can hang here. Builds clean for m5stack_sticks3/esp32s3/procpu with overlay-audio.conf. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Maintainer asked for a short phone clip of the playback to lock down QR-1 (intelligible playback), the one claim the serial log can't carry. Adds a 33s on-hardware record->play clip and cross-links it from the hw014 log. evidence/20260611-hw014-record-playback-demo.mp4 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

junnncct1106 and others added 9 commits June 9, 2026 04:35

Copilot AI review requested due to automatic review settings June 10, 2026 14:12

Copilot started reviewing on behalf of junnncct1106 June 10, 2026 14:13 View session

Copilot AI reviewed Jun 10, 2026

View reviewed changes

junnncct1106 and others added 5 commits June 11, 2026 00:44

junnncct1106 force-pushed the feature/14-record-playback-clean branch from a757e7b to 63de615 Compare June 10, 2026 20:27

Navigation Menu

feat(#14): mic record → speaker playback (M5StickS3 hardware bring-up) - #1

feat(#14): mic record → speaker playback (M5StickS3 hardware bring-up)#1
junnncct1106 wants to merge 14 commits into
mainfrom
feature/14-record-playback-clean

junnncct1106 commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		- Mic is mono on I2S slot 0; codec fixed at 16 kHz / 16-bit; playback volume
		−20 dB; the on-board speaker is small/quiet.

Conversation

junnncct1106 commented Jun 10, 2026

What

Root causes fixed

Verified on hardware

Pending

Note for maintainers

Test

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants