Skip to contentSkip to conversation

scripts: coccicheck: Change default value for parallelism

PATCH Superseded Sumera Priyadarsini 2 participants 4 messages Aug 12, 2020 – Aug 13, 2020 Coccinelle/Semantic Patches (SmPL)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scripts: coccicheck: Change default value for parallelism

By default, coccicheck utilizes all available threads to implement parallelisation. However, when hyperthreading is enabled, this leads to all threads per core being occupied resulting in longer wall-clock times and higher power consumption. Hence, to improve performance, modify coccicheck to use only one thread per core atmost.

In the cases where the total number of threads is more than 8 and hyperthreading is enabled, it was observed that optimum performance is achieved around one-fourth of the total number of cores. Modify the script further to accommodate this use case.

---
 scripts/coccicheck | 9 +++++++++
 1 file changed, 9 insertions(+)
scripts/coccicheck +9 -0
index e04d328210ac..dd228dcc915e 100755
@@ -75,8 +75,17 @@ else
         OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
     fi
 
+    # Use only one thread per core by default if hyperthreading is enabled
+    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
     if [ -z "$J" ]; then
         NPROC=$(getconf _NPROCESSORS_ONLN)
+	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
+		if [ $NPROC -gt 8 ] ; then
+			NPROC=$((NPROC/4))
+		else
+			NPROC=$((NPROC/2))
+		fi
+	fi
     else
         NPROC="$J"
     fi
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 4 lines · Sumera Priyadarsini

By default, coccicheck utilizes all available threads to implement parallelisation. However, when hyperthreading is enabled, this leads to all threads per core being occupied resulting in longer wall-clock times and higher power consumption.

I have the feeling that the above sentence is not quite optimal. Actually, using all of the available hardware threads would not be a bad thing, if it was giving a benefit. The point is that it doesn't. It makes the performance worse instead.

Quoted 2 lines

Hence, to improve performance,
modify coccicheck to use only one thread per core atmost.

"atmost" is not a word. It would be clearer to say "to use at most one thread per core".

Quoted 4 lines

In the cases where the total number of threads is more than 8 and hyperthreading is enabled, it was observed that optimum performance is achieved around one-fourth of the total number of cores. Modify the script further to accommodate this use case.

It would be nice to give some performance numbers and some information about the machine used.

thanks,
julia

Quoted 36 lines
---
 scripts/coccicheck | 9 +++++++++
 1 file changed, 9 insertions(+)
scripts/coccicheck +9 -1
index e04d328210ac..dd228dcc915e 100755
@@ -75,8 +75,17 @@ else
         OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
     fi
 
+    # Use only one thread per core by default if hyperthreading is enabled
+    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
     if [ -z "$J" ]; then
         NPROC=$(getconf _NPROCESSORS_ONLN)
+	if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
+		if [ $NPROC -gt 8 ] ; then
+			NPROC=$((NPROC/4))
+		else
+			NPROC=$((NPROC/2))
+		fi
+	fi
     else
         NPROC="$J"
     fi
--

2.17.1

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

#2 lore raw
Quoted 9 lines · Julia Lawall,
Quoted 4 lines · Sumera Priyadarsini

By default, coccicheck utilizes all available threads to implement parallelisation. However, when hyperthreading is enabled, this leads to all threads per core being occupied resulting in longer wall-clock times and higher power consumption.

Quoted 5 lines

I have the feeling that the above sentence is not quite optimal. Actually, using all of the available hardware threads would not be a bad thing, if it was giving a benefit. The point is that it doesn't. It makes the performance worse instead.

How does this sound?

However, when all available threads are used, a decrease in performance was noted. The elapsed time was minimum when at most one thread per core was used.

Quoted 6 lines
Quoted 2 lines

Hence, to improve performance,
modify coccicheck to use only one thread per core atmost.

"atmost" is not a word. It would be clearer to say "to use at most one thread per core".

Quoted 8 lines
Quoted 4 lines

In the cases where the total number of threads is more than 8 and hyperthreading is enabled, it was observed that optimum performance is achieved around one-fourth of the total number of cores. Modify the script further to accommodate this use case.

It would be nice to give some performance numbers and some information about the machine used.

Alright, will add this.

Quoted 17 lines

thanks,
julia

Quoted 14 lines
---
 scripts/coccicheck | 9 +++++++++
 1 file changed, 9 insertions(+)
scripts/coccicheck +0 -0
index e04d328210ac..dd228dcc915e 100755
@@ -75,8 +75,17 @@ else
         OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
     fi
 
Quoted 25 lines

scripts/coccicheck else · line 78

Quoted 1 line
scripts/coccicheck +1 -0
+    # Use only one thread per core by default if hyperthreading is

enabled

scripts/coccicheck else · line 79

Quoted 1 line
scripts/coccicheck +1 -0
+    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd

[:digit:])

Quoted 20 lines
scripts/coccicheck +7 -1
     if [ -z "$J" ]; then
         NPROC=$(getconf _NPROCESSORS_ONLN)
+     if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
+             if [ $NPROC -gt 8 ] ; then
+                     NPROC=$((NPROC/4))
+             else
+                     NPROC=$((NPROC/2))
+             fi
+     fi
     else
         NPROC="$J"
     fi
--

2.17.1

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

_______________________________________________
Cocci mailing list
[email protected]
https://systeme.lip6.fr/mailman/listinfo/cocci

maintainer
#3 lore raw
Quoted 18 lines · Sumera Priyadarsini

On Thu, 13 Aug, 2020, 1:45 AM Julia Lawall, <[email protected]> wrote:

On Wed, 12 Aug 2020, Sumera Priyadarsini wrote:

> By default, coccicheck utilizes all available threads to implement > parallelisation. However, when hyperthreading is enabled, this leads > to all threads per core being occupied resulting in longer wall-clock > times and higher power consumption.

I have the feeling that the above sentence is not quite optimal. Actually, using all of the available hardware threads would not be a bad thing, if it was giving a benefit.  The point is that it doesn't.  It makes the performance worse instead.

Quoted 3 lines

How does this sound?

However, when all available threads are used, a decrease in performance was noted. The elapsed time was minimum when at most one thread per core was used.

This looks better, thanks.

julia

Quoted 64 lines

> Hence, to improve performance,
> modify coccicheck to use only one thread per core atmost.

"atmost" is not a word.  It would be clearer to say "to use at most one thread per core".

> In the cases where the total number of threads is more than 8 and > hyperthreading is enabled, it was observed that optimum performance > is achieved around one-fourth of the total number of cores. > Modify the script further to accommodate this use case.

It would be nice to give some performance numbers and some information about the machine used.

Alright, will add this.

      thanks,
      julia

      >
      > Signed-off-by: Sumera Priyadarsini <[email protected]>
      > ---
      >  scripts/coccicheck | 9 +++++++++
      >  1 file changed, 9 insertions(+)
      >
      > diff --git a/scripts/coccicheck b/scripts/coccicheck
      > index e04d328210ac..dd228dcc915e 100755
      > --- a/scripts/coccicheck
      > +++ b/scripts/coccicheck
      > @@ -75,8 +75,17 @@ else
      >          OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
      >      fi
      >
      > +    # Use only one thread per core by default if hyperthreading is enabled
      > +    THREADS_PER_CORE=$(lscpu | grep "Thread(s) per core: " | tr -cd [:digit:])
      >      if [ -z "$J" ]; then
      >          NPROC=$(getconf _NPROCESSORS_ONLN)
      > +     if [ $THREADS_PER_CORE -gt 1 -a $NPROC -gt 2 ] ; then
      > +             if [ $NPROC -gt 8 ] ; then
      > +                     NPROC=$((NPROC/4))
      > +             else
      > +                     NPROC=$((NPROC/2))
      > +             fi
      > +     fi
      >      else
      >          NPROC="$J"
      >      fi
      > --
      > 2.17.1
      >
      > _______________________________________________
      > Cocci mailing list
      > [email protected]
      > https://systeme.lip6.fr/mailman/listinfo/cocci
      >