riscv: lib: add optimized memcmp() and extend KUnit tests
Subsystems (2)
v6 cover riscv: lib: add optimized memcmp() and extend KUnit tests
This v6 series provides an optimized RISC-V memcmp() implementation along with extended KUnit tests for functional verification and benchmarking
The memcmp() implementation now uses word-aligned loads for the generic path and leverages Zbb for fast sign resolution on supported hardware. A robust fallback to byte-by-byte comparison is implemented for non-Zbb systems to ensure correctness and stability across different toolchains and hardware revisions.
---
v6 changes:
- Fixed build errors kernel test robot noticed.
- Link to v5: https://lore.kernel.org/all/[email protected]/
v5 changes:
- Removed redundant warm-up from test case
- Optimized generic memcmp path with word-aligned loads.
- Implemented non-ZBB fallback to byte comparison per review.
- Link to v4: https://lore.kernel.org/all/[email protected]/
v4 changes:
- Fixed build error: call to undeclared function 'STRING_BENCH'
- Link to v3: https://lore.kernel.org/all/[email protected]/
v3 changes:
- Split memcmp benchmark into wrapper (string_bench_memcmp) and worker function (do_string_bench_memcmp).
- Removed all C99 mixed declarations; moved all variable declarations
to the top of each function.
- Converted len, iterations and loop counters in the benchmark to u64
to avoid implicit casts.
- Cleaned up spacing, indentation and minor style issues.
- Added #if defined(CONFIG_RISCV_ISA_ZBB)... in memcmp.S
- Link to v2: https://lore.kernel.org/all/[email protected]/
v2 changes:
- Added alignment checks for buffers to avoid expensive misaligned loads.
- Optimized the loop using end-pointers to reduce per-iteration overhead.
- Implemented word-aligned tail handling using ZBB shifts.
- Removed redundant pointer equality (a0 == a1) check.
- Retained BE support via #ifndef; ZBB rev8 is used for the LE fast-path.
- Fixed KUnit build failures for Clang and non-benchmark configs.
- Link to v1: https://lore.kernel.org/all/[email protected]/
Milan Tripkovic (2):
riscv: lib: add memcmp() implementation lib/string_kunit: extend benchmarks and unit test to memcmp()
arch/riscv/include/asm/string.h | 2 + arch/riscv/lib/Makefile | 1 + arch/riscv/lib/memcmp.S | 107 ++++++++++++++++++++++++++++++++ arch/riscv/purgatory/Makefile | 5 +- lib/tests/string_kunit.c | 106 +++++++++++++++++++++++++++++++ 5 files changed, 220 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/lib/memcmp.S
Signature · 1 line
2.43.0
PATCH v6 1/2 riscv: lib: add memcmp() implementation
From: Milan Tripkovic <[email protected]>
Add an assembly implementation of memcmp() for RISC-V. The implementation uses word-aligned loads for the generic path and the ZBB extension for optimized sign resolution, with an assembly fallback for non-ZBB systems.
Benchmark results (QEMU TCG, rv64):
Len | Default |NoALIGN | ALIGN | %NoALIGN | %ALIGN ------|---------|--------|--------|----------|-------- 1 B | 20.3 | 21.5 | 20.9 | +5.9% | +3.0% 7 B | 88.9 | 96.9 | 155.7 | +9.0% | +75.1% 8 B | 89.6 | 110.5 | 176.2 | +23.3% | +96.7% 16 B | 134.4 | 172.8 | 334.8 | +28.6% | +149.1% 31 B | 163.5 | 211.9 | 606.2 | +29.6% | +270.8% 64 B | 203.8 | 240.3 | 968.6 | +17.9% | +375.3% 127 B | 224.6 | 269.0 | 1362.8 | +19.8% | +506.8% 512 B | 235.7 | 269.9 | 1913.7 | +14.5% | +711.9% 1024 B| 256.8 | 283.5 | 2123.6 | +10.4% | +726.9% 4096 B| 263.8 | 299.7 | 2290.4 | +13.6% | +768.2%
---
v6 changes:
- Fixed build errors kernel test robot noticed.
- Link to v5: https://lore.kernel.org/all/[email protected]/
v5 changes:
- Optimized generic memcmp path with word-aligned loads.
- Implemented non-ZBB fallback to byte comparison per review.
- Link to v4: https://lore.kernel.org/all/[email protected]/
v2 changes:
- Added alignment checks for buffers to avoid expensive misaligned loads.
- Optimized the loop using end-pointers to reduce per-iteration overhead.
- Implemented word-aligned tail handling using ZBB shifts.
- Removed redundant pointer equality (a0 == a1) check.
- Retained BE support via #ifndef; ZBB rev8 is used for the LE fast-path.
- Link to v1: https://lore.kernel.org/all/[email protected]/
arch/riscv/include/asm/string.h | 2 + arch/riscv/lib/Makefile | 1 + arch/riscv/lib/memcmp.S | 107 ++++++++++++++++++++++++++++++++ arch/riscv/purgatory/Makefile | 5 +- 4 files changed, 114 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/lib/memcmp.S
arch/riscv/include/asm/string.h +2 -0
@@ -18,6 +18,8 @@ extern asmlinkage void *__memcpy(void *, const void *, size_t);#define __HAVE_ARCH_MEMMOVE extern asmlinkage void *memmove(void *, const void *, size_t); extern asmlinkage void *__memmove(void *, const void *, size_t); +#define __HAVE_ARCH_MEMCMP +extern asmlinkage int memcmp(const void *, const void *, size_t);#if !(defined(CONFIG_KASAN_GENERIC ) || defined(CONFIG_KASAN_SW_TAGS ))#define __HAVE_ARCH_STRCMP
arch/riscv/lib/Makefile +1 -0
@@ -3,6 +3,7 @@ lib-y += delay.o lib-y += memcpy.o lib-y += memset.o lib-y += memmove.o +lib-y += memcmp.o ifeq ($(CONFIG_KASAN_GENERIC)$(CONFIG_KASAN_SW_TAGS),) lib-y += strcmp.o lib-y += strlen.o
arch/riscv/lib/memcmp.S new file +107 -0
107 lines not on this page · raw message
arch/riscv/purgatory/Makefile +4 -1
@@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -purgatory-y := purgatory.o sha256.o entry.o string.o ctype.o memcpy.o memset.o +purgatory-y := purgatory.o sha256.o entry.o string.o ctype.o memcpy.o memset.o memcmp.o ifeq ($(CONFIG_KASAN_GENERIC)$(CONFIG_KASAN_SW_TAGS),) purgatory-y += strcmp.o strlen.o strncmp.o strnlen.o strchr.o strrchr.o endif
@@ -41,6 +41,9 @@ $(obj)/strchr.o: $(srctree)/arch/riscv/lib/strchr.S FORCE $(obj)/strrchr.o: $(srctree)/arch/riscv/lib/strrchr.S FORCE $(call if_changed_rule,as_o_S) +$(obj)/memcmp.o: $(srctree)/arch/riscv/lib/memcmp.S FORCE + $(call if_changed_rule,as_o_S) + CFLAGS_sha256.o := -D__DISABLE_EXPORTS -D__NO_FORTIFY CFLAGS_string.o := -D__DISABLE_EXPORTS CFLAGS_ctype.o := -D__DISABLE_EXPORTS
Signature · 1 line
2.43.0
PATCH v6 2/2 lib/string_kunit: extend benchmarks and unit test to memcmp()
In reply to Milan TripkovicFrom: Milan Tripkovic <[email protected]>
Extend the string benchmarking suite to include memcmp(). Extend the string unit test to include memcmp().
---
v6 changes:
- Fixed build errors kernel test robot noticed.
- Link to v5: https://lore.kernel.org/all/[email protected]/
v5 changes:
- Removed redundant warm-up from test case
- Link to v4: https://lore.kernel.org/all/[email protected]/
v4 changes:
- Fixed build error: call to undeclared function 'STRING_BENCH'
- Link to v3: https://lore.kernel.org/all/[email protected]/
v3 changes:
- Split memcmp benchmark into wrapper (string_bench_memcmp) and worker function (do_string_bench_memcmp).
- Removed all C99 mixed declarations; moved all variable declarations
to the top of each function.
- Converted len, iterations and loop counters in the benchmark to u64
to avoid implicit casts.
- Cleaned up spacing, indentation and minor style issues.
- Added #if defined(CONFIG_RISCV_ISA_ZBB)... in memcmp.S
- Link to v2: https://lore.kernel.org/all/[email protected]/
v2 changes:
- Fixed KUnit build failures for Clang and non-benchmark configs.
- Link to v1: https://lore.kernel.org/all/[email protected]/
lib/tests/string_kunit.c | 106 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+)
lib/tests/string_kunit.c +106 -0
@@ -881,6 +881,110 @@ static void string_bench_strrchr(struct kunit *test)STRING_BENCH_BUF (test, buf, len, strrchr, buf,'\0' ); } +static void string_test_memcmp (struct kunit *test) +{ +const unsigned int max_offset =16 ; +const unsigned int max_len =32 ; +const unsigned int buf_size = max_offset + max_len +32 ; +u8 *buf1, *buf2; +unsigned int i, j, len, k; +int res; + + buf1 =kunit_kzalloc (test, buf_size,GFP_KERNEL ); + buf2 =kunit_kzalloc (test, buf_size,GFP_KERNEL ); +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, buf1); +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, buf2); + +for (i =0 ; i < max_offset; i++) { +for (j =0 ; j < max_offset; j++) { +for (len =0 ; len <= max_len; len++) { +memset (buf1,'A' , buf_size); +memset (buf2,'A' , buf_size); +KUNIT_EXPECT_EQ_MSG (test,memcmp (buf1 + i, buf2 + j, len),0 , +"Should be equal: i:%u j:%u len:%u" , i, j, len); +for (k =0 ; k < len; k++) { +memset (buf1,'A' , buf_size); +memset (buf2,'A' , buf_size); + buf2[j + k] ='B' ; + res =memcmp (buf1 + i, buf2 + j, len); +KUNIT_EXPECT_NE_MSG (test, res,0 , +"Should detect difference at k:%u (i:%u j:%u len:%u)" , + k, i, j, len); +if (buf1[i + k] < buf2[j + k]) +KUNIT_EXPECT_LT (test, res,0 ); +else +KUNIT_EXPECT_GT (test, res,0 ); + } + } + } + } +} + +#ifndef STRING_BENCH +#define STRING_BENCH (...) 0 +#endif + +static void do_string_bench_memcmp (struct kunit *test) +{ +char *buf1 =NULL ; +char *buf2 =NULL ; +const u64 lengths[] = {1 ,7 ,8 ,16 ,32 ,64 ,128 ,512 ,1024 ,4096 }; +const int offsets[] = {0 ,1 ,3 ,7 }; +const u64 max_len =4096 +64 ; +unsigned int o, i; +unsigned int off; +u64 len; +u64 iterations; +u64 elapsed; +u64 ns_per_call; +u64 mbps; + + buf1 =vmalloc (max_len); + buf2 =vmalloc (max_len); + +if (!buf1 || !buf2) { +vfree (buf1); +vfree (buf2); +kunit_err (test,"vmalloc failed\n" ); +return ; + } + +memset (buf1,'A' , max_len); +memset (buf2,'A' , max_len); + +for (o =0 ; o <ARRAY_SIZE (offsets); o++) { + off = offsets[o]; + +for (i =0 ; i <ARRAY_SIZE (lengths); i++) { + len = lengths[i]; + iterations = (len <512 ) ?100000ULL :10000ULL ; + elapsed =STRING_BENCH (iterations, memcmp, buf1, buf2 + off, len); + ns_per_call =div_u64 (elapsed, iterations); + mbps = len ?div_u64 (iterations * len * (NSEC_PER_SEC /MEGA ), elapsed) :0 ; + +if (off ==0 ) { +kunit_info (test,"bench_memcmp_aligned: len=%-4llu: %llu MB/s (%llu ns/call)\n" , + len, mbps, ns_per_call); + }else { +kunit_info (test,"bench_memcmp_unaligned(off=%u): len=%-4llu: %llu MB/s (%llu ns/call)\n" , + off, len, mbps, ns_per_call); + } + } + } + +vfree (buf1); +vfree (buf2); +} + +static void string_bench_memcmp (struct kunit *test) +{ +if (!IS_ENABLED (CONFIG_STRING_KUNIT_BENCH )) { +kunit_skip (test,"CONFIG_STRING_KUNIT_BENCH not enabled" ); +return ; + } +do_string_bench_memcmp (test); +} +static struct kunit_case string_test_cases[] = {KUNIT_CASE (string_test_memset16),KUNIT_CASE (string_test_memset32),
@@ -910,6 +1014,8 @@ static struct kunit_case string_test_cases[] = {KUNIT_CASE (string_bench_strnlen),KUNIT_CASE (string_bench_strchr),KUNIT_CASE (string_bench_strrchr), +KUNIT_CASE (string_test_memcmp), + KUNIT_CASE_SLOW(string_bench_memcmp), {} };
Signature · 1 line
2.43.0
Paul Walmsley
maintainerRe: [PATCH v6 2/2] lib/string_kunit: extend benchmarks and unit test to memcmp()
Quoted 6 lines · Milan Tripkovic
From: Milan Tripkovic <[email protected]>
Extend the string benchmarking suite to include
memcmp(). Extend the string unit test to includememcmp().Signed-off-by Milan Tripkovic <[email protected]>
Same comments on this patch as were posted here:
https://lore.kernel.org/linux-riscv/[email protected]/T/#u
- Paul