Improve Coccinelle Parallelisation
cover Improve Coccinelle Parallelisation
Coccinelle utilises all available threads to implement parallelisation. However, this results in a decrease in performance.
This patchset aims to improve performance by modifying cocciccheck to use at most one thread per core by default.
Sumera Priyadarsini (2):
scripts: coccicheck: Change default value for parallelism
Documentation: Coccinelle: Modify parallelisation information in docs
Documentation/dev-tools/coccinelle.rst | 4 ++-- scripts/coccicheck | 5 +++++ 2 files changed, 7 insertions(+), 2 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/2 scripts: coccicheck: Change default value for parallelism
By default, coccicheck utilizes all available threads to implement parallelisation. However, when all available threads are used, a decrease in performance is noted. The elapsed time is minimum when at most one thread per core is used.
For example, on benchmarking the semantic patch kfree.cocci for usb/serial using hyperfine, the outputs obtained for J=5 and J=2 are 1.32 and 1.90 times faster than those for J=10 and J=9 respectively for two separate runs. For the larger drivers/staging directory, minimium elapsed time is obtained for J=3 which is 1.86 times faster than that for J=12. The optimal J value does not exceed 6 in any of the test runs. The benchmarks are run on a machine with 6 cores, with 2 threads per core, i.e, 12 hyperthreads in all.
To improve performance, modify coccicheck to use at most only one thread per core by default.
---
Changes in V2:
- Change commit message as suggested by Julia Lawall
Changes in V3:
- Use J/2 as optimal value for machines with more
than 8 hyperthreads as well.
--- scripts/coccicheck | 5 +++++ 1 file changed, 5 insertions(+)
scripts/coccicheck +5 -0
@@ -75,8 +75,13 @@ else OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" fi + # Use only one thread per core by default if hyperthreading is enabled + THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:]) if [ -z "$J" ]; then NPROC=$(getconf _NPROCESSORS_ONLN) + if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then + NPROC=$((NPROC/2)) + fi else NPROC="$J" fi
Signature · 1 line
2.25.1
Julia Lawall
maintainerRe: [Cocci] [PATCH 1/2] scripts: coccicheck: Change default value for parallelism
Quoted 14 lines · Sumera Priyadarsini
By default, coccicheck utilizes all available threads to implement parallelisation. However, when all available threads are used, a decrease in performance is noted. The elapsed time is minimum when at most one thread per core is used.
For example, on benchmarking the semantic patch
kfree.coccifor usb/serial using hyperfine, the outputs obtained for J=5 and J=2 are 1.32 and 1.90 times faster than those for J=10 and J=9 respectively for two separate runs. For the largerdrivers/stagingdirectory, minimium elapsed time is obtained for J=3 which is 1.86 times faster than that for J=12. The optimal J value does not exceed 6 in any of the test runs. The benchmarks are run on a machine with 6 cores, with 2 threads per core, i.e, 12 hyperthreads in all.
Quoted 4 lines
To improve performance, modify coccicheck to use at most only one thread per core by default.
Signed-off-by Sumera Priyadarsini <[email protected]>
I have applied this one, so just the patch on the docuemtnation needs to be improved.
julia
Quoted 37 lines
---
Changes in V2:
- Change commit message as suggested by Julia Lawall
Changes in V3:
- Use J/2 as optimal value for machines with more
than 8 hyperthreads as well.--- scripts/coccicheck | 5 +++++ 1 file changed, 5 insertions(+)scripts/coccicheck +5
-1@@ -75,8 +75,13 @@ else OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" fi + # Use only one thread per core by default if hyperthreading is enabled + THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:]) if [ -z "$J" ]; then NPROC=$(getconf _NPROCESSORS_ONLN) + if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then + NPROC=$((NPROC/2)) + fi else NPROC="$J" fi --2.25.1
_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci
PATCH 2/2 Documentation: Coccinelle: Modify parallelisation information in docs
In reply to Sumera PriyadarsiniThis patchset modifies coccicheck to use at most one thread per core by default for optimal performance. Modify documentation in coccinelle.rst to reflect the same.
--- Documentation/dev-tools/coccinelle.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/dev-tools/coccinelle.rst +2 -2
@@ -130,8 +130,8 @@ To enable verbose messages set the V= variable, for example:: Coccinelle parallelization -------------------------- -By default, coccicheck tries to run as parallel as possible. To change -the parallelism, set the J= variable. For example, to run across 4 CPUs:: +By default, coccicheck uses at most only one thread per core of the system. +To change the parallelism, set the J= variable. For example, to run across 4 CPUs:: make coccicheck MODE=report J=4
Signature · 1 line
2.25.1
Julia Lawall
maintainerRe: [Cocci] [PATCH 2/2] Documentation: Coccinelle: Modify parallelisation information in docs
Quoted 3 lines · Sumera Priyadarsini
This patchset modifies coccicheck to use at most one thread per core by default for optimal performance. Modify documentation in
coccinelle.rstto reflect the same.
It would be good for the documentation to mention that this only occurs if the machine has more than two cores (and more than 4 hardware threads).
julia
Quoted 29 lines
Signed-off-by Sumera Priyadarsini <[email protected]>--- Documentation/dev-tools/coccinelle.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Documentation/dev-tools/coccinelle.rst +2
-3@@ -130,8 +130,8 @@ To enable verbose messages set the V= variable, for example:: Coccinelle parallelization -------------------------- -By default, coccicheck tries to run as parallel as possible. To change -the parallelism, set the J= variable. For example, to run across 4 CPUs:: +By default, coccicheck uses at most only one thread per core of the system. +To change the parallelism, set the J= variable. For example, to run across 4 CPUs:: make coccicheck MODE=report J=4 --2.25.1
_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci