perf test stat+shadow_stat.sh: Add threshold for rounding errors
PATCH 1/2 perf test stat+shadow_stat.sh: Add threshold for rounding errors
From: Veronika Molnarova <[email protected]>
The test was failing in specific scenarios due to imperfection of FP arithmetics. The bc command wasn't correctly rounding the result of division causing the failure.
Replace the bc with awk which should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.
---
Resending the patches with sign-off as it didn't go through last time, also tried to fix the found issues.
tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++----- 1 file changed, 24 insertions(+), 6 deletions(-)
tools/perf/tests/shell/stat+shadow_stat.sh +24 -6
@@ -4,6 +4,8 @@ set -e +THRESHOLD=0.015 + # skip if system-wide mode is forbidden perf stat -a true > /dev/null 2>&1 || exit 2
@@ -33,10 +35,18 @@ test_global_aggr() fi # use printf for rounding and a leading zero - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"` + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'` if [ "$ipc" != "$res" ]; then - echo "IPC is different: $res != $ipc ($num / $cyc)" - exit 1 + # check the difference from the real result for FP imperfections + diff=`echo $ipc $res $THRESHOLD | \ + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'` + + if [ $diff -eq 1 ]; then + echo "IPC is different: $res != $ipc ($num / $cyc)" + exit 1 + fi + + echo "Warning: Difference of IPC is under the threshold" fi done }
@@ -67,10 +77,18 @@ test_no_aggr() fi # use printf for rounding and a leading zero - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"` + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'` if [ "$ipc" != "$res" ]; then - echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)" - exit 1 + # check difference from the real result for FP imperfections + diff=`echo $ipc $res $THRESHOLD | \ + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'` + + if [ $diff -eq 1 ]; then + echo "IPC is different: $res != $ipc ($num / $cyc)" + exit 1 + fi + + echo "Warning: Difference of IPC is under the threshold" fi done }
Signature · 1 line
2.41.0
PATCH 2/2 perf test lock_contention.sh: Skip test if not enough CPUs
From: Veronika Molnarova <[email protected]>
Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.
Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.
---
Resending the patches with sign-off as it didn't go through last time, also tried to fix the found issues.
tools/perf/tests/shell/lock_contention.sh | 6 ++++++ 1 file changed, 6 insertions(+)
tools/perf/tests/shell/lock_contention.sh +6 -0
@@ -32,6 +32,12 @@ check() { err=2 exit fi + + if [ `nproc` -lt 4 ]; then + echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly" + err=2 + exit + fi } test_record()
Signature · 1 line
2.41.0
Re: [PATCH 2/2] perf test lock_contention.sh: Skip test if not enough CPUs
Quoted 9 lines · On Tue, 19 Sep 2023, [email protected] wrote:
From: Veronika Molnarova <[email protected]>
Machines with less then 4 CPUs weren't consistently triggering lock events required for the test.
Skip the test on those machines. The limit of 4 CPUs is set as it generates around 100 lock events for a test.
Signed-off-by Veronika Molnarova <[email protected]>
Quoted 28 lines
---
Resending the patches with sign-off as it didn't go through last time, also tried to fix the found issues.tools/perf/tests/shell/lock_contention.sh | 6 ++++++ 1 file changed, 6 insertions(+)tools/perf/tests/shell/lock_contention.sh +6
-0@@ -32,6 +32,12 @@ check() { err=2 exit fi + + if [ `nproc` -lt 4 ]; then + echo "[Skip] Low number of CPUs (`nproc`), lock event cannot be triggered certainly" + err=2 + exit + fi } test_record()Signature · 1 line
2.41.0
Quoted 6 lines · On Tue, 19 Sep 2023, [email protected] wrote:
From: Veronika Molnarova <[email protected]>
The test was failing in specific scenarios due to imperfection of FP arithmetics. The
bccommand wasn't correctly rounding the result of division causing the failure.
Quoted 6 lines
Replace the
bcwithawkwhich should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.Signed-off-by Veronika Molnarova <[email protected]>
Quoted 68 lines
---
Resending the patches with sign-off as it didn't go through last time, also tried to fix the found issues.tools/perf/tests/shell/stat+shadow_stat.sh | 30 +++++++++++++++++----- 1 file changed, 24 insertions(+), 6 deletions(-)tools/perf/tests/shell/stat+shadow_stat.sh +24
-6@@ -4,6 +4,8 @@ set -e +THRESHOLD=0.015 + # skip if system-wide mode is forbidden perf stat -a true > /dev/null 2>&1 || exit 2@@ -33,10 +35,18 @@ test_global_aggr() fi # use printf for rounding and a leading zero - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"` + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'` if [ "$ipc" != "$res" ]; then - echo "IPC is different: $res != $ipc ($num / $cyc)" - exit 1 + # check the difference from the real result for FP imperfections + diff=`echo $ipc $res $THRESHOLD | \ + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'` + + if [ $diff -eq 1 ]; then + echo "IPC is different: $res != $ipc ($num / $cyc)" + exit 1 + fi + + echo "Warning: Difference of IPC is under the threshold" fi done }@@ -67,10 +77,18 @@ test_no_aggr() fi # use printf for rounding and a leading zero - res=`printf "%.2f" "$(echo "scale=6; $num / $cyc" | bc -q)"` + res=`echo $num $cyc | awk '{printf "%.2f", $1 / $2}'` if [ "$ipc" != "$res" ]; then - echo "IPC is different for $cpu: $res != $ipc ($num / $cyc)" - exit 1 + # check difference from the real result for FP imperfections + diff=`echo $ipc $res $THRESHOLD | \ + awk '{x = ($1 - $2) < 0 ? ($2 - $1) : ($1 - $2); print (x > $3)}'` + + if [ $diff -eq 1 ]; then + echo "IPC is different: $res != $ipc ($num / $cyc)" + exit 1 + fi + + echo "Warning: Difference of IPC is under the threshold" fi done }Signature · 1 line
2.41.0
Namhyung Kim
AppliedQuoted 8 lines · Michael Petlan
Quoted 6 lines · On Tue, 19 Sep 2023, [email protected] wrote:
From: Veronika Molnarova <[email protected]>
The test was failing in specific scenarios due to imperfection of FP arithmetics. The
bccommand wasn't correctly rounding the result of division causing the failure.
Quoted 8 lines
Quoted 6 lines
Replace the
bcwithawkwhich should work with more decimal places and add a threshold to catch any possible rounding errors. The acceptable rounding error is set to 0.01 when the test passes with a warning message.Signed-off-by Veronika Molnarova <[email protected]>Acked-by Michael Petlan <[email protected]>
Applied both to perf-tools-next, thanks!