NotificationConsumer: add pluggable Queue strategy#1198
Conversation
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Introduce a Queue interface backing NotificationConsumer with two implementations: StandardQueue (default FIFO) and DedupQueue (collapses duplicate messages and re-orders them to the back). The queue is injected via the constructor, defaulting to StandardQueue so existing callers are unaffected. Signed-off-by: manish1 <manish1@arista.com>
manish1-arista
force-pushed
the
add-notification-consumer-queue-strategy
branch
from
5fc3832 to
e9bcd5c
Compare
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
@manish1-arista I presented a HLD for an opt-in DeDup queue policy last week in the SONiC Foundation meeting that can be seen here: LruDedup Queue Policy. The HLD has links to the code PRs that implements the design. It appears there is some duplication of work in the set of PRs you have raised. Can we have a quick meeting to discuss how to move this forward? Thanks. |
|
Our FDB Notification consumer queue Dedup approach targeted fdborch specifically, reducing redundant notifications at a single consumer. The Other PRs (#1191, sonic-net/sonic-swss#4586 and sonic-net/sonic-sairedis#1899) fixes this memory growth for other orch consumer queue also by addressing the problem at the infrastructure layer rather than a single orch, by adding allowlist for each orchagent and Dedup queues for multiple orchangents. Closing this PR as other PRs had merged |
3 participants
What I did
Refactored NotificationConsumer to delegate its internal storage to a pluggable Queue strategy injected via the constructor. Added two implementations:
StandardQueue — plain FIFO, used as the default so existing callers compile and behave unchanged.
DedupQueue — backed by a
std::list + std::unordered_map, collapses duplicate messages and re-orders the surviving entry to the back of the queue.DedupQueue also tracks total_push and duplicates_skipped counters.
Why I did it
Some notification channels (e.g. FDB) can be flooded with duplicate events under load (MAC move storms), causing the consumer to backlog and process stale state. Until now NotificationConsumer always used a plain std::queue, with no way to apply dedup or any other policy on a per-channel basis. Making the queue strategy injectable lets each caller pick the right policy for its channel without touching NotificationConsumer or affecting other consumers and leaves the door open for additional strategies in the future.