Implement a memory-bounded lock-free ring buffer for streaming real-time token transfer events

[Damola-Sodiq]

Description

Replace unbounded arrays with a lock-free ring buffer utilizing SharedArrayBuffer to ingest high-frequency Web3 token transfer events without Garbage Collection (GC) pauses.

Architecture & Context

During heavy on-chain liquidations, the backend is flooded with RPC events. Standard JavaScript arrays require continuous reallocation and aggressive garbage collection, leading to event loop blocking. A pre-allocated ring buffer provides O(1) ingestion.

Technical Requirements

• Utilize SharedArrayBuffer and Atomics to manage concurrent read/write pointers across worker threads.
• Implement backpressure signals if the write head approaches the read tail.
• Serialize raw event data into fixed-size binary structs before insertion.

Acceptance Criteria

• The backend sustains 20,000+ events per second without dropping payloads.
• V8 heap size remains completely flat during ingestion bursts.
• Worker threads read and process events without mutex locking.