Skip to content

Raw sensor collection: avoid String heap churn by writing directly into SD buffer #27

Description

@awickert

Background

As part of ongoing standardization of the Northern Widget sensor API, a generic raw-reading collection function is planned for the Margay library (using C++ templates so it works with any NW sensor library without a runtime dependency):

template<typename SensorType>
void collectRawReadings(SensorType& sensor, uint16_t nReadings,
                        uint8_t component, File& sdFile);

Each sensor library will implement a standard takeRawReading() method that returns one reading's worth of data.

Issue

If takeRawReading() returns an Arduino String, the collection loop will make N heap allocations in rapid succession — one per reading. On the ATMega1284p (16 KB SRAM), this risks heap fragmentation over a long-running deployment, which can cause silent failures.

Proposed enhancement

Rather than returning a String, takeRawReading() should write directly into a caller-provided char buffer at a given offset:

uint16_t takeRawReading(char* buf, uint16_t offset);
// writes CSV data into buf starting at offset, returns new offset

The Margay collection function maintains a 512-byte char buffer (matching the SD card sector size) and flushes to SD at sector boundaries. This:

  • Eliminates all heap allocation during the collection loop
  • Batches SD writes at the natural sector size (minimizing write operations and power consumption)
  • Keeps memory usage fixed and predictable

Context

  • SD card sector size = 512 bytes → natural buffer size
  • ATMega1284p: 16 KB SRAM, currently ~1940 bytes used by globals in a typical Margay sketch
  • This design is part of the broader NW sensor API standardization effort across Apis, Haar, MaxBotix, T9602, and Walrus libraries

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions