cpuhotplug05.sh: Rewrite test case
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
1) Replace sar command with /proc/stat
2) Convert to new API
3) Remove unused/duplicated code
--- runtest/cpuhotplug | 2 +- .../cpu_hotplug/functional/cpuhotplug05.sh | 186 ++++++------------ 2 files changed, 63 insertions(+), 125 deletions(-)
runtest/cpuhotplug +1 -1
@@ -4,6 +4,6 @@ cpuhotplug02 cpuhotplug02.sh -c 1 -l 1 cpuhotplug03 cpuhotplug03.sh -c 1 -l 1 cpuhotplug04 cpuhotplug04.sh -l 1 -cpuhotplug05 cpuhotplug05.sh -c 1 -l 1 -d /tmp +cpuhotplug05 cpuhotplug05.sh -c 1 cpuhotplug06 cpuhotplug06.sh -c 1 -l 1 cpuhotplug07 cpuhotplug07.sh -c 1 -l 1 -d /usr/src/linux
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh +62 -124
219 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh line 59
Quoted 2 lines · Xiao Yang
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh +2
-0+do_test() +{
First of all, your patch changes the test scenario. The original scenario is this:
(setup) Deactivate selected CPU
1. Check that load statistics are zero or empty for selected CPU
2. Activate selected CPU
3. Check that load statistics are valid for selected CPU
4. Deactivate selected CPU
(cleanup) Activate selected CPU
I have a few kernel builds where step 3 correctly detects regression.
Your changed scenario looks like this:
1. Activate selected CPU
2. Check that load statistics are non-zero for selected CPU
3. Deactivate selected CPU
4. Check that load statistics are empty for selected CPU (cleanup) Return selected CPU back to original state
Your new scenario has two problems:
- You have to run at least two test loops to verify that reactivating a CPU doesn't break /proc/stat entries.
- You only check that load statistics are non-zero. This works with sar because the program already normalizes load statistics for you. But it won't detect the regression mentioned above because the bug sets one of the entries to something absurdly high. This will confuse sar enough to print all load values as 0 but it'll pass your checks.
In addition to checking that at least some values are non-zero, you need to check a few more things:
- Call getconf CLK_TCK to find /proc/stat timer resolution
- Calculate system uptime in seconds
- Check that sum of all /proc/stat values for selected CPU is less than or equal to (timer resolution * (uptime+1))
- Also consider doing the above check for all CPUs to increase test coverage.
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh line 81
Quoted 13 lines
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh +7
-3+ for field in $(seq 2 11); do + field_value=$(grep "^cpu${cpu_num}" /proc/stat | awk "{print \$$field}") + [ "$field_value" != "0" ] && passed=1 done - if [ $check_passed -eq 0 ]; then - tst_resm TFAIL "No change in the CPU statistics" - tst_exit + if [ $passed -eq 0 ]; then + tst_res TFAIL \ + "all field of online cpu{cpu_num} shows zero in /proc/stat" + return 1 fi
It'd be better to write this check (including the upper bound check explained above) entirely in Awk.
if ! awk "[test script]" /proc/stat; then
tst_res TFAIL ...
return 1
fi
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
Hi Martin,
Thanks for your comment.
Also sorry for the late reply.
Quoted 21 lines · Martin Doucha
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh line 59
Quoted 2 lines · Xiao Yang
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh +2
-0+do_test() +{First of all, your patch changes the test scenario. The original scenario is this:
(setup) Deactivate selected CPU
1. Check that load statistics are zero or empty for selected CPU
2. Activate selected CPU
3. Check that load statistics are valid for selected CPU
4. Deactivate selected CPU
(cleanup) Activate selected CPUI have a few kernel builds where step 3 correctly detects regression.
Your changed scenario looks like this:
1. Activate selected CPU
2. Check that load statistics are non-zero for selected CPU
3. Deactivate selected CPU
4. Check that load statistics are empty for selected CPU (cleanup) Return selected CPU back to original state
Quoted 4 lines
Your new scenario has two problems:
- You have to run at least two test loops to verify that reactivating a CPU doesn't break
/proc/statentries.
I think running cpuhotplug05.sh with -i 2 can verfiy this point.
Perhaps we can add -i option to runtest/cpuhotplug, or do you prefer to keep the original scenario?
Quoted 6 lines
- You only check that load statistics are non-zero. This works with sar because the program already normalizes load statistics for you. But it won't detect the regression mentioned above because the bug sets one of the entries to something absurdly high. This will confuse sar enough to print all load values as 0 but it'll pass your checks.
Good point.
I will looking into the code of sar.
Quoted 7 lines
In addition to checking that at least some values are non-zero, you need to check a few more things:
- Callgetconf CLK_TCKto find/proc/stattimer resolution
- Calculate system uptime in seconds
- Check that sum of all/proc/statvalues for selected CPU is less than or equal to (timer resolution * (uptime+1))
I will try to add them as you suggested.
Quoted 2 lines
- Also consider doing the above check for all CPUs to increase test coverage.
Is it necessary to add above check for all CPUs?
This test is designed to test specified CPU so you can test each CPU by using -c option.
Thanks,
Xiao Yang
Quoted 22 lines
Quoted 4 lines
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh +3
-0+ for field in $(seq 2 11); do + field_value=$(grep "^cpu${cpu_num}" /proc/stat | awk "{print \$$field}") + [ "$field_value" != "0" ] && passed=1testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh line 81
Quoted 10 lines
testcases/kernel/hotplug/cpu_hotplug/functional/cpuhotplug05.sh +4
-3done - if [ $check_passed -eq 0 ]; then - tst_resm TFAIL "No change in the CPU statistics" - tst_exit + if [ $passed -eq 0 ]; then + tst_res TFAIL \ + "all field of online cpu{cpu_num} shows zero in /proc/stat" + return 1 fiIt'd be better to write this check (including the upper bound check explained above) entirely in Awk.
if ! awk "[test script]"
/proc/stat; then
tst_resTFAIL ...
return 1
fi
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
Quoted 9 lines · Xiao Yang
Quoted 4 lines
Your new scenario has two problems:
- You have to run at least two test loops to verify that reactivating a CPU doesn't break
/proc/statentries.I think running
cpuhotplug05.shwith -i 2 can verfiy this point.Perhaps we can add -i option to runtest/cpuhotplug, or do you prefer to keep the original scenario?
I'd prefer keeping the original scenario (with your version of cleanup). Running extra iterations of the same test should not be required to get the full intended test coverage.
Quoted 7 lines
Quoted 2 lines
- Also consider doing the above check for all CPUs to increase test coverage.
Is it necessary to add above check for all CPUs?
This test is designed to test specified CPU so you can test each CPU by using -c option.
It's not necessary but it'd be a nice addition. Note that running the test script multiple times with different -c arguments does not fully cover this extra check. Turning a CPU core off and on again might reset its /proc/stat line into a sane state while breaking all the others.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
Hi,
Quoted 6 lines · Xiao Yang
Quoted 4 lines
Quoted 4 lines
Your new scenario has two problems:
- You have to run at least two test loops to verify that reactivating a CPU doesn't break
/proc/statentries.
Quoted 8 lines
Quoted 4 lines
I think running
cpuhotplug05.shwith -i 2 can verfiy this point.Perhaps we can add -i option to runtest/cpuhotplug, or do you prefer to keep the original scenario?
I'd prefer keeping the original scenario (with your version of cleanup). Running extra iterations of the same test should not be required to get the full intended test coverage.
Agree with Martin. Yang, could you, please, send new version, where you keep it?
Kind regards,
Petr
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
Quoted 1 line · Petr Vorel
Agree with Martin. Yang, could you, please, send new version, where you keep it?
Hi Petr, Martin.
Sorry, I am busy with other things recently.
Don't worry, I will send new version this week or next week. :-)
Thanks,
Xiao Yang
Quoted 3 lines
Kind regards,
Petr
PATCH 1/1 cpuhotplug05.sh: Rewrite test case
Hi,
Quoted 2 lines
Sorry, I am busy with other things recently.
Don't worry, I will send new version this week or next week. :-)
Yang, sure, no rush.
Martin, thanks a lot for very good review!
Kind regards,
Petr