Skip to content

netlink: add parseMessagesIter to reduce memory usage#266

Merged
nickgarlis merged 1 commit intomdlayher:mainfrom
nickgarlis:parse-messages-iter
Mar 12, 2026
Merged

netlink: add parseMessagesIter to reduce memory usage#266
nickgarlis merged 1 commit intomdlayher:mainfrom
nickgarlis:parse-messages-iter

Conversation

@nickgarlis
Copy link
Collaborator

Introduce a parseMessages iterator to improve parsing performance and serve as the foundation for a future receive iterator in the public API.

The functionality in syscall.ParseNetlinkMessage is very similar to to Message.UnmarshalFromBinary; the only missing part was walking the buffer, which has now been implemented in this iterator.

Synthetic benchmarks show a performance improvement when parsing and collecting messages using this approach.

In more detail, the benchmark compared the old approach (syscall.ParseNetlinkMessage + conversion) with the new
iterator approach (parseMessagesIter + collection into slice).

goos: linux
goarch: amd64
pkg: github.com/mdlayher/netlink
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
BenchmarkParseMessages/small/1msg/old-20         	 9447193	       117.5 ns/op	 136.17 MB/s	      96 B/op	       2 allocs/op
BenchmarkParseMessages/small/1msg/new-20         	15606260	        70.79 ns/op	 226.02 MB/s	      48 B/op	       1 allocs/op
BenchmarkParseMessages/small/10msgs/old-20       	  951582	      1174 ns/op	 136.23 MB/s	    1728 B/op	       6 allocs/op
BenchmarkParseMessages/small/10msgs/new-20       	 1230187	       915.4 ns/op	 174.79 MB/s	    1312 B/op	       5 allocs/op
BenchmarkParseMessages/medium/50msgs/old-20      	  221370	      5210 ns/op	 307.13 MB/s	    7840 B/op	       8 allocs/op
BenchmarkParseMessages/medium/50msgs/new-20      	  477787	      3678 ns/op	 434.96 MB/s	    5792 B/op	       7 allocs/op
BenchmarkParseMessages/large/100msgs/old-20      	  113692	      9720 ns/op	 823.07 MB/s	   16032 B/op	       9 allocs/op
BenchmarkParseMessages/large/100msgs/new-20      	  206064	      7015 ns/op	1140.45 MB/s	   11936 B/op	       8 allocs/op

// Each iteration yields a Message and any error encountered during parsing.
// The iterator stops on the first error or when all messages are parsed.
//
// If b is less that NLMSG_HDRLEN bytes, no error or messages will be returned.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to keep the behavior similar to syscall.ParseNetlinkMessage

@nickgarlis
Copy link
Collaborator Author

cc @aojea in case you're interested

@nickgarlis nickgarlis force-pushed the parse-messages-iter branch from 6b8fc8c to b365d7a Compare March 9, 2026 20:54
Introduce a parseMessages iterator to improve parsing performance and
serve as the foundation for a future receive iterator in the public API.

The functionality in syscall.ParseNetlinkMessage is very similar to
to Message.UnmarshalFromBinary; the only missing part was walking the
buffer, which has now been implemented in this iterator.

Synthetic benchmarks show a performance improvement when parsing and
collecting messages using this approach.
@nickgarlis nickgarlis force-pushed the parse-messages-iter branch from b365d7a to 3dad657 Compare March 11, 2026 19:03
@nickgarlis nickgarlis changed the title netlink: add parseMessagesIter to speed up parsing netlink: add parseMessagesIter to reduce memory usage Mar 11, 2026
@nickgarlis
Copy link
Collaborator Author

BenchmarkNftablesDump shows about a 1.8% reduction in memory and a small drop in allocations. Nothing huge, but the larger improvements will come in the next change.

@nickgarlis nickgarlis merged commit 2a932c0 into mdlayher:main Mar 12, 2026
8 checks passed
@aojea
Copy link

aojea commented Mar 17, 2026

cc @aojea in case you're interested

thanks for doing this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants