memmove speed optimized#13
Conversation
| aligned_src = (uintxlen_t*)src; | ||
|
|
||
| /* Copy 9X SZREG bytes at a time if possible. */ | ||
| while (length >= (SZREG*9)) { |
There was a problem hiding this comment.
when i see such relatively long code segments that i have seen before, I will usually ask you to make an inline function.
Makes the code more readable and maintainable.
As a rule of thumb, if your function does not fit on one screen, you should probably structure it better
There was a problem hiding this comment.
I see that this is forward vs. backwards, but this could be an option to the compiler (similar to how the assembly version does it). I would strongly assume that the compiler then inlines two different version of the function.
There was a problem hiding this comment.
i'm not sure i understand what you mean by "option to the compiler" – do you mean to have a variable that contains either -1 or 1 depending on the kind of overlap, so then the same code can be used for both scenarios?
There was a problem hiding this comment.
i'm not sure i understand what you mean by "option to the compiler" – do you mean to have a variable that contains either -1 or 1 depending on the kind of overlap, so then the same code can be used for both scenarios?
I formulated this weirdly but you got the point.
There was a problem hiding this comment.
just pushed a new suggestion for this
Copy the common implementation of memmove() to the RISC-V port. Rename memmove.S to memmove-asm.S to keep naming of files consistent between functions. Update Makefile.inc with the changed filenames.
…types Remove macros or replace them with static inline functions or RISC-V specific macros where applicable. Change data types to fixed-width and/or RISC-V specific types.
Add loop-unrolling for the case where both source and destination address are aligned in the case of a destructive overlap, and increase the unroll factor from four to nine for the word-by-word copy loop in the non-destructive case. This matches the loop-unrolling done in memcpy() and increases performance for lenghts >= SZREG*9 while not degrading performance for shorter lengths.
m-fally
force-pushed
the
memmove_speed_opt
branch
from
e775909 to
31ed784
Compare
|
@christian-herber-nxp I've pushed an updated version of the implementation with the changes you suggested and tried to organize the data better/made some plots to visualize the differences between implementations: For I have also tested a version based on the All tests were run on Graphical plots for all scenarios tested can be found here: https://cloud.servus.at/s/a7Gqnoigxak7y4q -march=rv32ic -mabi=ilp32 -mtune=thead-c906 -mstrict-align -O3
Total number of tests: 8174 -march=rv32ic -mabi=ilp32 -mtune=thead-c906 -mno-strict-align -O3
Total number of tests: 8174 -march=rv64ic -mabi=lp64 -mtune=thead-c906 -mstrict-align -O3
Total number of tests: 8166 -march=rv64ic -mabi=lp64 -mtune=thead-c906 -mno-strict-align -O3
Total number of tests: 8166 |
|
Since there are less cases where the "check for misalignment"-version is slower than the original |
|
Forgot to add the information about code sizes, here it is: Original Implementation
New Implementation (Check for misalignment)
New Implementation (Check for matching alignment)
|
If misaligned accesses are slow or prohibited, either source or destination address are unaligned and the number of bytes to be copied is > SZREG*2, align the source address to xlen. This speeds up the function in the case where at least one address is unaligned, since now one word (or doubleword for rv64) is loaded at a time, therefore reducing the amount of memory accesses necessary. We still need to store back individual bytes since the destination address might (still) be unaligned after aligning the source. The threshold of SZREG*2 was chosen because the additional overhead caused by aligning the source would decrease performance for shorter lengths.
Regenerate the configuration files since a stub-file was replaced with a new implementation in the RISC-V port (memmove.c), and one other file has been renamed (memmove.S => memmove-asm.S).
m-fally
force-pushed
the
memmove_speed_opt
branch
from
31ed784 to
dcc2e1d
Compare
2 participants
Here is the current state of my optimization-suggestion for memmove().
I have used the common implementation of the function as a base, which first checks if at least one of the addresses is unaligned, and only uses an optimized algorithm if both addresses are aligned. Otherwhise it copies byte by byte.
I have added an unrolled loop for the destructive overlap case as well as increased the unroll-factor of the loop in the non-destructive overlap case. I chose an unroll factor of 9*SZREG since that is what is also used in memcpy(), and according to my tests it seems like a reasonable choice.
The other bigger change is aligning the source address when one of the addresses is unaligned. In that case, one word (or doubleword for rv64) is loaded at a time, and individual bytes are stored to the destination. For the vast majority of cases I get better or equally as good results when aligning the source than when aligning the destination.
This change also affects the case when source and destination are aligned, there is a destructive overlap and length is not a multiple of SZREG, because in this scenario the length is added to both source and destination, which causes them to become unaligned.
I have also tried always aligning the source and then checking whether source and destination have the same alignment. This performs better than checking for misalignment when at least one address is unaligned (especially when the addresses already have the same alignment). However, when the addresses are both aligned (or when -mno-strict-align is used) in a lot of cases this approach performs worse than just checking for misalignment of one of the addresses (in the case of -mno-strict-align performance is even slightly worse than in the original implementation).
Maybe it would be an idea to use the check for matching alignment only in the case of a destructive overlap, since having two aligned addresses and an "unaligned" length would produce the scenario where that approach performs well, and in the case of two aligned addresses and an aligned length it does not do too much worse than the other approach. But that would favour the case where length is not a multiple of SZREG, so I am not sure if it is good to do that.
The following tables compare the data for both approaches I have tried (checking for misalignment of at least one address and only aligning if one is misaligned vs. always aligning the source and then checking whether the alignments of source and destination match) and the original algorithm for rv32 and rv64:
memmove_speed_opt_comparison_rv32.ods
memmove_speed_opt_comparison_rv64.ods
With regards to incorporating zilsd I haven't had any success yet, since there are additional checks and alignment operations needed to be sure the addresses are aligned by 8 bytes.
I think I will try to also unroll the misaligned-copy-loop and see if in that case the extra overhead is worth it.