Skip to contentSkip to conversation

perf test lock_contention.sh: Skip test if the number of CPUs is low

PATCH In review [email protected] 3 participants 5 messages Aug 21, 2023 – Sep 6, 2023 perf events

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 perf test lock_contention.sh: Skip test if the number of CPUs is low

From: notValord <[email protected]>

Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.

Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.

---
 tools/perf/tests/shell/lock_contention.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/lock_contention.sh +6 -0
index f2cc187b618..1d3598ff6a0 100755
@@ -31,6 +31,12 @@ check() {
 		echo "[Skip] No lock contention tracepoints"
 		err=2
 		exit
+	fi 
+
+	if [ `nproc` -lt 4 ]; then
+		echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+		err=2
+		exit
 	fi
 }
 
Signature · 1 line
2.41.0
#1 lore raw

PATCH 2/2 perf test stat+shadow_stat.sh: Add threshold for rounding errors

From: notValord <[email protected]>

The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.

Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.

---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +24 -6
index 0e9cba84e75..73c990fe5ee 100755
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015
+
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
 
@@ -33,10 +35,18 @@ test_global_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check the difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
@@ -67,10 +77,18 @@ test_no_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different for $cpu: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
Signature · 1 line
2.41.0
Acked-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors

Quoted 10 lines · On Mon, 21 Aug 2023, [email protected] wrote:

From: notValord <[email protected]>

The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.

Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.

Quoted 65 lines
---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +24 -6
index 0e9cba84e75..73c990fe5ee 100755
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015
+
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
 
@@ -33,10 +35,18 @@ test_global_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check the difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
@@ -67,10 +77,18 @@ test_no_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different for $cpu: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
Signature · 1 line
2.41.0
#3 lore raw

Re: [PATCH 2/2] perf test stat+shadow_stat.sh: Add threshold for rounding errors

In reply to [email protected]

Em Mon, Aug 21, 2023 at 11:00:53AM +0200, [email protected] escreveu:

Quoted 1 line

From: notValord <[email protected]>

Hi, is the above the same person as [email protected]? Can we please have the full name and use that as the From and also the Signed-off-by?

Thanks,

- Arnaldo

Quoted 72 lines

The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.

Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.

---
 tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++-----
 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +24 -6
index 0e9cba84e75..73c990fe5ee 100755
@@ -4,6 +4,8 @@
 
 set -e
 
+TRESHOLD=0.015
+
 # skip if system-wide mode is forbidden
 perf stat -a true > /dev/null 2>&1 || exit 2
 
@@ -33,10 +35,18 @@ test_global_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check the difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
@@ -67,10 +77,18 @@ test_no_aggr()
 		fi
 
 		# use printf for rounding and a leading zero
-		res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"`
+		res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'`
 		if [ "$ipc" != "$res" ]; then
-			echo "IPC is different for $cpu: $res != $ipc  ($num / $cyc)"
-			exit 1
+			# check difference from the real result for FP imperfections
+			diff=`echo $ipc $res $TRESHOLD | \
+			awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'`
+
+			if [ $diff -eq 1 ]; then
+				echo "IPC is different: $res != $ipc  ($num / $cyc)"
+				exit 1
+			fi
+
+			echo "Warning: Difference of IPC is under the treshold"
 		fi
 	done
 }
Signature · 1 line
2.41.0
Signature · 1 line
- Arnaldo
Acked-by on v1 1/2
#4 lore raw
In reply to [email protected]
Quoted 7 lines · On Mon, 21 Aug 2023, [email protected] wrote:

From: notValord <[email protected]>

Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.

Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.

Quoted 25 lines
---
 tools/perf/tests/shell/lock_contention.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/lock_contention.sh +6 -0
index f2cc187b618..1d3598ff6a0 100755
@@ -31,6 +31,12 @@ check() {
 		echo "[Skip] No lock contention tracepoints"
 		err=2
 		exit
+	fi 
+
+	if [ `nproc` -lt 4 ]; then
+		echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly"
+		err=2
+		exit
 	fi
 }
 
Signature · 1 line
2.41.0