Skip to content

riscv: optimize memcpy/memmove/memset

Jisheng Zhang linux-riscv lkml llvm oe-kbuild-all 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest3 patches 15 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3riscv: optimized memcpyarch/riscv/include/asm/string.h arch/riscv/kernel/riscv_ksyms.c arch/riscv/lib/Makefile +3 more+136 -1202 replies
2/3riscv: optimized memmovearch/riscv/include/asm/string.h arch/riscv/kernel/riscv_ksyms.c arch/riscv/lib/Makefile +2 more+27 -3228 replies
3/3riscv: optimized memsetarch/riscv/include/asm/string.h arch/riscv/kernel/riscv_ksyms.c arch/riscv/lib/Makefile +3 more+44 -1223 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/3] riscv: optimize memcpy/memmove/memset

This series is to renew Matteo's "riscv: optimized mem* functions" sereies.

Compared with Matteo's original series, Jisheng made below changes:
1. adopt Emil's change to fix boot failure when build with clang
2. add corresponding changes to purgatory
3. always build optimized string.c rather than only build when optimize for performance
4. implement unroll support when src & dst are both aligned to keep the same performance as assembly version. After disassembling, I found that the unroll version looks something like below, so it acchieves the "unroll" effect as asm version but in C programming language:

	ld	t2,0(a5)
	ld	t0,8(a5)
	ld	t6,16(a5)
	ld	t5,24(a5)
	ld	t4,32(a5)
	ld	t3,40(a5)
	ld	t1,48(a5)
	ld	a1,56(a5)
	sd	t2,0(a6)
	sd	t0,8(a6)
	sd	t6,16(a6)
	sd	t5,24(a6)
	sd	t4,32(a6)
	sd	t3,40(a6)
	sd	t1,48(a6)
	sd	a1,56(a6)

And per my testing, unrolling more doesn't help performance, so the "c" version only unrolls by using 8 GP regs rather than 16 ones as asm version.
5. Add proper __pi_memcpy and __pi___memcpy alias
6. more performance numbers.

Per my benchmark with [1] on TH1520, CV1800B and JH7110 platforms, the unaligned medium memcpy performance is running about 3.5x ~ 8.6x speed of the unpatched versions's! Check patch1 for more details and performance numbers.

Here is the original cover letter msg from Matteo:
Replace the assembly mem{cpy,move,set} with C equivalent.

Try to access RAM with the largest bit width possible, but without doing unaligned accesses.

A further improvement could be to use multiple read and writes as the assembly version was trying to do.

Tested on a BeagleV Starlight with a SiFive U74 core, where the improvement is noticeable.

Matteo Croce (3):
riscv: optimized memcpy
riscv: optimized memmove
riscv: optimized memset

 arch/riscv/include/asm/string.h |  14 +-
 arch/riscv/kernel/riscv_ksyms.c |   6 -
 arch/riscv/lib/Makefile         |   9 +-
 arch/riscv/lib/memcpy.S         | 110 -----------
 arch/riscv/lib/memmove.S        | 317 --------------------------------
 arch/riscv/lib/memset.S         | 113 ------------
 arch/riscv/lib/string.c         | 187 +++++++++++++++++++
 arch/riscv/purgatory/Makefile   |  13 +-
 8 files changed, 206 insertions(+), 563 deletions(-)
 delete mode 100644 arch/riscv/lib/memcpy.S
 delete mode 100644 arch/riscv/lib/memmove.S
 delete mode 100644 arch/riscv/lib/memset.S
 create mode 100644 arch/riscv/lib/string.c
Signature · 1 line
2.43.0