Real-Time Sensor Data Acquisition System

C++17 · ESP-IDF / FreeRTOS · ESP32-S3

An accelerometer logger that samples at exactly 1000 Hz, buffers through a lock-free ring, writes every sample to SD, and streams the same data over Wi-Fi — with the timing characterised rather than assumed.

Built to answer a specific question: what does an active Wi-Fi stack actually do to hard real-time acquisition, and what does it cost to fix?

---

Hardware setup

Breadboard (top) used purely as a common ground bus — the Freenove header exposes only one GND pin and two modules need one. microSD module over SPI (left), MPU-6050 accelerometer over I2C (right), ESP32-S3 (bottom). Everything else is direct female-to-female jumpers; GPIO14 is left free as the logic analyser probe point.

---

Results

All figures measured on real hardware, ~1–2 million samples per run. Every capture records the firmware configuration that produced it in its CSV header, so no two runs can be confused.

Timing

Configuration	p99 jitter	max jitter	dropped
Wi-Fi off	2.0 µs	28 µs	0
Wi-Fi on, mitigated	35.0 µs	136 µs	0
Wi-Fi on, unmitigated	144.0 µs	741 µs	0

p99 jitter = the 99th-percentile deviation of the actual sample interval from the ideal 1000 µs.

Two mitigations account for the 4.2× difference between the last two rows:

1. Acquisition pinned to core 1, away from the Wi-Fi/LWIP stack, which ESP-IDF places on core 0 by default.
2. WIFI_PS_NONE — modem sleep disabled, so the radio never parks between beacons and never stalls other work waking back up.

The mitigated result reproduced independently: 34.0 µs and 35.0 µs on separate days with different AP associations, across 3 million samples total.

Reliability

Metric	Result
Longest clean run	2 012 835 consecutive samples, zero gaps
Sample rate held	1000.00 Hz
SD card stalls absorbed	14 (worst 750 ms)
I2C errors	0
Watchdog resets	0

---

The interesting bug

Early runs lost several thousand samples per hour, in bursts.

The first hypothesis — that calling fsync() on every write (50×/second) was provoking the card — was wrong. Instrumenting fwrite and fsync separately showed the truth: the SD card's own firmware blocks for up to 750 ms, and it does so regardless of which call triggers the flush. Wear-levelling, invisible from the host side, and invisible to normal error handling because every write returned success.

The fix was not in the write path at all. It was sizing the ring buffer from the measured worst-case stall: 4096 samples ≈ 4 s of slack against a 750 ms worst case. Zero drops since.

Two things this taught, both the expensive way:

• A ring buffer that overflows cannot measure the stall that overflowed it. The observed gap is bounded below by (true stall − buffer depth), so any figure derived from it understates the problem.
• A capture with no provenance is not evidence. Two runs were once compared without confirming which firmware wrote each. Every CSV now stamps its own configuration and build time in the header.

---

Architecture

 gptimer (1 kHz, hardware)
     │
     ├─ ISR (IRAM): toggle GPIO14 probe, notify task, return.
     │              No I2C here -- ESP-IDF I2C is not ISR-safe.
     ▼
 Acquisition task ── core 1, priority 23
     │  reads MPU-6050 over I2C @ 400 kHz, timestamps, sequence-numbers
     ▼
 SpscRingBuffer<Sample, 4096>   lock-free, acquire/release atomics
     │
     ▼
 Drain task ── core 0, priority 5
     ├─→ SD card  (CSV, batched 4 KB, fsync every 2 s)
     └─→ Wi-Fi    (UDP, ~48 packets/s)

The split exists because I2C transactions are not ISR-safe on ESP-IDF. The interrupt does nothing but wake a task; the task does the work. The ring buffer decouples a hard-real-time producer from a consumer that can block for the best part of a second.

---

Hardware

Part	Interface	Pins
Freenove ESP32-S3 DevKit (16 MB flash, 8 MB octal PSRAM)	—	—
SHILLEHTEK MPU-6050 (GY-521) accelerometer	I2C @ 400 kHz, addr 0x68	SDA GPIO8, SCL GPIO9, VCC 3V3
Hutomwua microSD module	SPI @ 4 MHz	MOSI GPIO11, MISO GPIO13, SCK GPIO12, CS GPIO10, VCC 5V
Logic analyser probe point	—	GPIO14 (ISR toggle)

Three details that are not obvious and cost real time to discover:

• The SD module runs from 5V, not 3V3. Its AMS1117 regulator needs ~4.5 V to hold 3.3 V; on 3V3 the card browns out and mounts intermittently. Its onboard level shifter keeps 5 V off the ESP32's GPIOs.
• SD in SPI mode needs pull-ups on MISO/MOSI/CS. Flying jumper leads supply none, and without them the mount fails with ESP_ERR_TIMEOUT despite perfectly correct wiring.
• The MPU-6050 boots asleep. Write 0x00 to PWR_MGMT_1 (0x6B) or every accelerometer register reads back frozen — indistinguishable from a wiring fault.

The MPU-6050 is I2C-only. Worst-case measured acquisition time is 752 µs of the 1000 µs budget, which puts the practical ceiling near 1.3 kHz. An SPI sensor would remove that limit.

---

Build and flash

Requires PlatformIO. ESP-IDF 6.1, C++17.

cp include/wifi_credentials.hpp.example include/wifi_credentials.hpp
# edit it: 2.4 GHz SSID only, the ESP32-S3 has no 5 GHz radio

pio run -t upload && pio device monitor

Watch for the configuration line, which is also written into the CSV header:

config: ring=4096 sync_ms=2000 spi_khz=4000 rate_hz=1000 accel=8g wifi=on ps_none=1 acq_core=1

Analysing a capture

python3 tools/analyze_log.py /path/to/daq_000.csv

Reports dropped samples (gaps in the monotonic sequence number) and p50/p99/max period jitter.

Receiving telemetry

python tools/udp_sink.py        # run on the host, not inside WSL

WSL2 is NAT'd onto its own subnet, so a listener inside it never sees LAN traffic.

Reproducing the ablation

Edit include/daq_config.hpp, rebuild, reflash:

#define DAQ_WIFI_PS_NONE 1   // 0 = leave modem sleep enabled
#define DAQ_ACQ_CORE     1   // 0 = acquisition on Wi-Fi's core

Both values are stamped into every CSV header. (These are plain #defines rather than platformio.ini build flags because framework = espidf silently drops build_flags before the component compile — all four build environments otherwise produce identical firmware.)

---

Layout

src/          main.cpp (timer, tasks), mpu6050, sd_logger, telemetry
include/      ring_buffer.hpp (lock-free SPSC), sample, daq_config
test/host/    ring buffer unit tests -- run on the host, no hardware needed
tools/        analyze_log.py, udp_sink.py

Run the host-side ring buffer tests without any hardware:

g++ -std=c++17 -O2 -Wall -Wextra -pthread -Iinclude \
    test/host/test_ring_buffer.cpp -o /tmp/test_rb && /tmp/test_rb