lib/smp_processor_id: Don't use cpumask_equal()
PATCH 1/1 lib/smp_processor_id: Don't use cpumask_equal()
The check_preemption_disabled() function uses cpumask_equal() to see if the task is bounded to the current CPU only. cpumask_equal() calls memcmp() to do the comparison. As x86 doesn't have __HAVE_ARCH_MEMCMP, the slow memcmp() function in lib/string.c is used.
On a RT kernel that call check_preemption_disabled() very frequently, below is the perf-record output of a certain microbenchmark:
42.75% 2.45% testpmd [kernel.kallsyms] [k] check_preemption_disabled 40.01% 39.97% testpmd [kernel.kallsyms] [k] memcmp
We should avoid calling memcmp() in performance critical path. So the cpumask_equal() call is now replaced with an equivalent check that makes no external function call.
--- lib/smp_processor_id.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/smp_processor_id.c +2 -1
@@ -23,7 +23,8 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)* Kernel threads bound to a single CPU can safely use * smp_processor_id(): */ -if (cpumask_equal (current->cpus_ptr,cpumask_of (this_cpu))) +if ((current->nr_cpus_allowed ==1 ) && +cpumask_test_cpu (this_cpu, current->cpus_ptr)) goto out;/*
Signature · 1 line
2.18.1
Quoted 25 lines · Waiman Long
The
check_preemption_disabled()function usescpumask_equal()to see if the task is bounded to the current CPU only.cpumask_equal()callsmemcmp()to do the comparison. As x86 doesn't have__HAVE_ARCH_MEMCMP, the slowmemcmp()function inlib/string.cis used.On a RT kernel that call
check_preemption_disabled()very frequently, below is the perf-record output of a certain microbenchmark:42.75% 2.45% testpmd [kernel.kallsyms] [k]
check_preemption_disabled40.01% 39.97% testpmd [kernel.kallsyms] [k] memcmpWe should avoid calling
memcmp()in performance critical path. So thecpumask_equal()call is now replaced with an equivalent check that makes no external function call.Signed-off-by Waiman Long <[email protected]>--- lib/smp_processor_id.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)lib/smp_processor_id.c +0
-0
lib/smp_processor_id.c unsigned int check_preemption_disabled(const char *what1, const char *what2) · line 30
Quoted 9 lines
lib/smp_processor_id.c +2
-1@@ -23,7 +23,8 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)* Kernel threads bound to a single CPU can safely use * smp_processor_id(): */ -if (cpumask_equal (current->cpus_ptr,cpumask_of (this_cpu))) +if ((current->nr_cpus_allowed ==1 ) && +cpumask_test_cpu (this_cpu, current->cpus_ptr)) goto out;/*
My bad. The second check isn't really necessary.
Cheers,
Longman