lib: memcmp optimization
Subsystems (2)
PATCH 1/1 lib: memcmp optimization
From: Florian-Ewald Mueller <[email protected]>
During testing, I have configured 128 md/raid1's and, while under heavy IO, I started a check on each of them
(echo check > /sys/block/mdx/md/sync_action).
The CPU utilization went through the ceiling and when looking for the cause (with 'perf top'). I've discovered that ~50% of the time was spend in memcmp() called from process_checks().
With this patch applied, it drops to 4% - 10%.
--- lib/string.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
lib/string.c +15 -1
@@ -852,7 +852,7 @@ EXPORT_SYMBOL(memmove);* @count: The size of the area. */ #undef memcmp -__visible int memcmp(const void *cs, const void *ct, size_t count) +static inline int __memcmp (const void *cs,const void *ct,size_t count) {const unsigned char *su1, *su2;int res =0 ;
@@ -862,6 +862,20 @@ __visible int memcmp(const void *cs, const void *ct, size_t count)break ;return res; } +__visible int memcmp(const void *cs, const void *ct, size_t count) +{ +const uint64_t *l1p = cs; +const uint64_t *l2p = ct; + +while (count >=sizeof (*l1p)) { +if (*l1p != *l2p) +return __memcmp (l1p, l2p,sizeof (*l1p)); + count -=sizeof (*l1p); + ++l1p; + ++l2p; + } +return __memcmp (l1p, l2p, count); +}EXPORT_SYMBOL (memcmp);#endif
Signature · 1 line
2.7.4
Andrew Morton
maintainerQuoted 11 lines · On Tue, 9 Oct 2018 16:28:11 +0200 Jack Wang <[email protected]> wrote:
From: Florian-Ewald Mueller <[email protected]>
During testing, I have configured 128 md/raid1's and, while under heavy IO, I started a check on each of them
(echo check >/sys/block/mdx/md/sync_action).The CPU utilization went through the ceiling and when looking for the cause (with 'perf top'). I've discovered that ~50% of the time was spend in
memcmp()called fromprocess_checks().With this patch applied, it drops to 4% - 10%.
Which CPU architecture? Most important architectures appear to define __HAVE_ARCH_MEMCMP.
lib/string.c EXPORT_SYMBOL(memmove); · line 855
Quoted 8 lines
lib/string.c +1
-1@@ -852,7 +852,7 @@ EXPORT_SYMBOL(memmove);* @count: The size of the area. */ #undef memcmp -__visible int memcmp(const void *cs, const void *ct, size_t count) +static inline int __memcmp (const void *cs,const void *ct,size_t count)
What the heck does __visible do?
Quoted 11 lines
lib/string.c +4
-0{const unsigned char *su1, *su2;int res =0 ;@@ -862,6 +862,20 @@ __visible int memcmp(const void *cs, const void *ct, size_t count)break ;return res; } +__visible int memcmp(const void *cs, const void *ct, size_t count) +{ +const uint64_t *l1p = cs; +const uint64_t *l2p = ct;
lib/string.c __visible int memcmp(const void *cs, const void *ct, size_t count) · line 878
Quoted 10 lines
lib/string.c +10
-0+ +while (count >=sizeof (*l1p)) { +if (*l1p != *l2p) +return __memcmp (l1p, l2p,sizeof (*l1p)); + count -=sizeof (*l1p); + ++l1p; + ++l2p; + } +return __memcmp (l1p, l2p, count); +}
This is going to do bad things if the incoming addresses aren't suitably aligned.
Certainly, byte-at-a-time is a pretty lame implementation when the addresses are suitably aligned. A fallback to the lame version when there is misalignment will be simple to do. And presumably there will be decent benefits to whoever is actually using this code. But I'm wondering who is actually using this code!
Quoted 9 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Oct 2018 16:28:11 +0200 Jack Wang <[email protected]> wrote:
From: Florian-Ewald Mueller <[email protected]>
During testing, I have configured 128 md/raid1's and, while under heavy IO, I started a check on each of them
(echo check >/sys/block/mdx/md/sync_action).
Quoted 8 lines
Quoted 5 lines
The CPU utilization went through the ceiling and when looking for the cause (with 'perf top'). I've discovered that ~50% of the time was spend in
memcmp()called fromprocess_checks().With this patch applied, it drops to 4% - 10%.
Which CPU architecture? Most important architectures appear to define
__HAVE_ARCH_MEMCMP.
x86_64 is our case, I should have documented it more clearly.
Quoted 28 lines
lib/string.c EXPORT_SYMBOL(memmove); · line 855
Quoted 9 lines
lib/string.c +1
-1@@ -852,7 +852,7 @@ EXPORT_SYMBOL(memmove);* @count: The size of the area. */ #undef memcmp -__visible int memcmp(const void *cs, const void *ct, size_t count) +static inline int __memcmp (const void *cs,const void *ct,size_t count)What the heck does
__visibledo?Quoted 16 lines
lib/string.c +9
-0{const unsigned char *su1, *su2;int res =0 ;@@ -862,6 +862,20 @@ __visible int memcmp(const void *cs, const void *ct, size_t count)break ;return res; } +__visible int memcmp(const void *cs, const void *ct, size_t count) +{ +const uint64_t *l1p = cs; +const uint64_t *l2p = ct; + +while (count >=sizeof (*l1p)) { +if (*l1p != *l2p) +return __memcmp (l1p, l2p,sizeof (*l1p)); + count -=sizeof (*l1p);
Quoted 9 lines
lib/string.c __visible int memcmp(const void *cs, const void *ct, size_t count) · line 878
Quoted 5 lines
lib/string.c +5
-0+ ++l1p; + ++l2p; + } +return __memcmp (l1p, l2p, count); +}This is going to do bad things if the incoming addresses aren't suitably aligned.
Right, as you said, this is easy to fix.
Quoted 6 lines
Certainly, byte-at-a-time is a pretty lame implementation when the addresses are suitably aligned. A fallback to the lame version when there is misalignment will be simple to do. And presumably there will be decent benefits to whoever is actually using this code. But I'm wondering who is actually using this code!
I found recent discussion about why x86-64 is using generic string function here:
+cc John, x86
https://lkml.org/lkml/2018/10/3/818
Signature · 15 lines
Jack Wang Linux Kernel Developer ProfitBricks GmbH Greifswalder Str. 207 D - 10405 Berlin Tel: +49 30 577 008 042 Fax: +49 30 577 008 299 Email: [email protected] URL: https://www.profitbricks.de Sitz der Gesellschaft: Berlin Registergericht: Amtsgericht Charlottenburg, HRB 125506 B Geschäftsführer: Achim Weiss, Matthias Steinberg, Christoph Steffens