perf: preallocate slices on swarm and protocol hot paths#3524
Open
Sahil-4555 wants to merge 2 commits into
Open
perf: preallocate slices on swarm and protocol hot paths#3524Sahil-4555 wants to merge 2 commits into
Sahil-4555 wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In reviewing some performance paths I observed that there are quite a few spots in our hot paths (dialling connections, filtering addresses, doing protocol negotiation, pushing identify messages, etc) where we declare slices as
nilor with zero capacity and then continue to append to them in a loop.Since we know the exact size or the max length of the collection up-front, Go is doing extra work under the hood. It allocates a backing array, and when the slice grows, it allocates another bigger one and copies all the data over. Then it leaves the old ones for GC to clean up.
So in this PR I've updated these places to preallocate the slices with the right capacity upfront.
What’s the benefit?
ConvertFromStringsincore/protocol/id.go, we now assign directly via index (res[i] = ...) rather than usingappend(), avoiding slice capacity checks.What has changed ?
todial/tojoinindial_worker.gowithlen(addrRanking)and did similar updates for resolution and filtering slices inswarm_dial.goandswarm.go.Dial Ranker: We saved the original address count prior to filtering so we can size the
resdelay slice correctly from the start.diffslices.Address book & Autorelay/Autonat: Preallocated slices in basic host’s
diffAddrs, autonat’sskipPeerand autorelay’scleanupAddressSetCheck
I have run all the test cases for the modified packages and all is fine. No behavioural change, just clean memory optimisation.