Skip to contentSkip to conversation

perf test lock_contention.sh: Skip test if the number of CPUs is low

PATCH In review [email protected] 3 participants 6 messages Sep 12, 2023 perf events

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 perf test lock_contention.sh: Skip test if the number of CPUs is low

From: Veronika Molnarova <[email protected]>

Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.

Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.

---
 tools/perf/tests/shell/lock_contention.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/lock_contention.sh +6 -0
index f2cc187b618..1d3598ff6a0 100755
@@ -31,6 +31,12 @@ check() {
 		echo "[Skip] No lock contention tracepoints"
 		err=2
 		exit
+	fi 
+
+	if [ `nproc` -lt 4 ]; then
+		echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+		err=2
+		exit
 	fi
 }
 
Signature · 1 line
2.41.0
#1 lore raw

PATCH 2/2 perf test stat+shadow_stat.sh: Add threshold for rounding errors

From: Veronika Molnarova <[email protected]>

The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.

Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.

---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +24 -6
index 0e9cba84e75..73c990fe5ee 100755
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015
+
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
 
@@ -33,10 +35,18 @@ test_global_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check the difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
@@ -67,10 +77,18 @@ test_no_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different for $cpu: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
Signature · 1 line
2.41.0
Acked-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors

Quoted 10 lines · On Tue, 12 Sep 2023, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.

Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.

Quoted 65 lines
---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +24 -6
index 0e9cba84e75..73c990fe5ee 100755
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015
+
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
 
@@ -33,10 +35,18 @@ test_global_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check the difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
@@ -67,10 +77,18 @@ test_no_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different for $cpu: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
Signature · 1 line
2.41.0
#3 lore raw

Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors

In reply to [email protected]

Em Tue, Sep 12, 2023 at 02:19:26PM +0200, [email protected] escreveu:

Quoted 19 lines

From: Veronika Molnarova <[email protected]>

The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.

Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.

---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +0 -0
index 0e9cba84e75..73c990fe5ee 100755

tools/perf/tests/shell/stat+shadow_stat.sh line 7

Quoted 4 lines
tools/perf/tests/shell/stat+shadow_stat.sh +1 -0
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015

You got it right on the patch description, I fixed it here as:

⬢[acme@toolbox perf-tools-next]$ git diff

tools/perf/tests/shell/stat+shadow_stat.sh +3 -3
index 626d669158d2d089..a7e4a8f2ad186458 100755
@@ -4,7 +4,7 @@
 
 set -e
 
-TRESHOLD=0.015
+THRESHOLD=0.015
 
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
@@ -38,7 +38,7 @@ test_global_aggr()
                res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
                if [ "$ipc" != "$res" ]; then
                        # check the difference from the real result for FP imperfections
-                       diff=`echo $ipc $res $TRESHOLD | \
+                       diff=`echo $ipc $res $THRESHOLD | \
                        awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
 
                        if [ $diff -eq 1 ]; then
@@ -80,7 +80,7 @@ test_no_aggr()
                res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
                if [ "$ipc" != "$res" ]; then
                        # check difference from the real result for FP imperfections
-                       diff=`echo $ipc $res $TRESHOLD | \
+                       diff=`echo $ipc $res $THRESHOLD | \
                        awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
 
                        if [ $diff -eq 1 ]; then

⬢[acme@toolbox perf-tools-next]$

Acked-by on v1 1/2
#4 lore raw
In reply to [email protected]
Quoted 7 lines · On Tue, 12 Sep 2023, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.

Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.

Quoted 25 lines
---
 tools/perf/tests/shell/lock_contention.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/lock_contention.sh +6 -0
index f2cc187b618..1d3598ff6a0 100755
@@ -31,6 +31,12 @@ check() {
 		echo "[Skip] No lock contention tracepoints"
 		err=2
 		exit
+	fi 
+
+	if [ `nproc` -lt 4 ]; then
+		echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+		err=2
+		exit
 	fi
 }
 
Signature · 1 line
2.41.0
#5 lore raw
In reply to [email protected]

Em Tue, Sep 12, 2023 at 02:19:25PM +0200, [email protected] escreveu:

Quoted 7 lines

From: Veronika Molnarova <[email protected]>

Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.

Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.

Better, but still some issues, fixing manually.

⬢[acme@toolbox perf-tools-next]$ b4 am -ctsl --cc-trailers [email protected] Grabbing thread from lore.kernel.org/all/20230912121926.12111-1-vmolnaro%40redhat.com/t.mbox.gz Checking for newer revisions
Grabbing search results from lore.kernel.org
Analyzing 4 messages in the thread
Checking attestation on all messages, may take a moment...
---
✓ [PATCH 1/2] perf test lock_contention.sh: Skip test if the number of CPUs is low
+ Acked-by: Michael Petlan <[email protected]> (✓ DKIM/redhat.com)
+ Link: https://lore.kernel.org/r/[email protected]
+ Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
✓ [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors
+ Acked-by: Michael Petlan <[email protected]> (✓ DKIM/redhat.com)
+ Link: https://lore.kernel.org/r/[email protected]
+ Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
---
✓ Signed: DKIM/redhat.com
---
Total patches: 2
---

Base: not specified

       git am ./20230912_vmolnaro_perf_test_lock_contention_sh_skip_test_if_the_number_of_cpus_is_low.mbx

⬢[acme@toolbox perf-tools-next]$ git diff
⬢[acme@toolbox perf-tools-next]$ git am ./20230912_vmolnaro_perf_test_lock_contention_sh_skip_test_if_the_number_of_cpus_is_low.mbx
Applying: perf test lock_contention.sh: Skip test if the number of CPUs is low .git/rebase-apply/patch:13: trailing whitespace.
fi
warning: 1 line adds whitespace errors.
Applying: perf test stat+shadow_stat.sh: Add threshold for rounding errors ⬢[acme@toolbox perf-tools-next]$

Quoted 24 lines
---
 tools/perf/tests/shell/lock_contention.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/lock_contention.sh +6 -0
index f2cc187b618..1d3598ff6a0 100755
@@ -31,6 +31,12 @@ check() {
 		echo "[Skip] No lock contention tracepoints"
 		err=2
 		exit
+	fi 
+
+	if [ `nproc` -lt 4 ]; then
+		echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+		err=2
+		exit
 	fi
 }
 
Signature · 1 line
2.41.0
Signature · 1 line
- Arnaldo