nvme/033: add test case for nvme update hardware queue count
PATCH 1/1 nvme/033: add test case for nvme update hardware queue count
Modify nvme module parameter write_queues to change hardware queue count, then reset nvme controller to reinitialize nvme with different queue count.
Attention, this test case may trigger a kernel panic.
--- tests/nvme/033 | 87 ++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/033.out | 2 ++ 2 files changed, 89 insertions(+) create mode 100755 tests/nvme/033 create mode 100644 tests/nvme/033.out
tests/nvme/033 new file +87 -0
@@ -0,0 +1,87 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0+ +# Copyright (C) 2020 Weiping Zhang <[email protected]> +# +# Test nvme update hardware queue count larger than system cpu count +# + +. tests/nvme/rc + +DESCRIPTION="test nvme update hardware queue count larger than system cpu count" +QUICK=1 + +requires() { + _have_program dd +} + +device_requires() { + _test_dev_is_nvme +} + +test_device() { + echo "Running ${TEST_NAME}" + + local old_write_queues + local cur_hw_io_queues + local file + local sys_dev=$TEST_DEV_SYSFS/device + + # backup old module parameter: write_queues + file=/sys/module/nvme/parameters/write_queues + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi + old_write_queues="$(cat $file)" + + # get current hardware queue count + file="$sys_dev/queue_count" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi + cur_hw_io_queues="$(cat "$file")" + # minus admin queue + cur_hw_io_queues=$((cur_hw_io_queues - 1)) + + # set write queues count to increase more hardware queues + file=/sys/module/nvme/parameters/write_queues + echo "$cur_hw_io_queues" > "$file" + + # reset controller, make it effective + file="$sys_dev/reset_controller" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi + echo 1 > "$file" + + # wait nvme reinitialized + for ((m = 0; m < 10; m++)); do + if [[ -b "${TEST_DEV}" ]]; then + break + fi + sleep 0.5 + done + if (( m > 9 )); then + echo "nvme still not reinitialized after 5 seconds!" + return 1 + fi + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 status=none + + # If all work well restore hardware queue to default + file=/sys/module/nvme/parameters/write_queues + echo "$old_write_queues" > "$file" + + # reset controller + file="$sys_dev/reset_controller" + echo 1 > "$file" + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 iflag=direct status=none + dd if=/dev/zero of="${TEST_DEV}" bs=4096 count=1 oflag=direct status=none + + echo "Test complete" +}
tests/nvme/033.out new file +2 -0
@@ -0,0 +1,2 @@ +Running nvme/033 +Test complete
Signature · 1 line
2.18.1
Quoted 44 lines · Weiping Zhang
Modify nvme module parameter
write_queuesto change hardware queue count, then reset nvme controller to reinitialize nvme with different queue count.Attention, this test case may trigger a kernel panic.
Signed-off-by Weiping Zhang <[email protected]>--- tests/nvme/033 | 87 ++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/033.out | 2 ++ 2 files changed, 89 insertions(+)create mode 100755 tests/nvme/033
create mode 100644 tests/nvme/033.outtests/nvme/033 new file +24
-0@@ -0,0 +1,87 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0+ +# Copyright (C) 2020 Weiping Zhang <[email protected]> +# +# Test nvme update hardware queue count larger than system cpu count +# + +. tests/nvme/rc + +DESCRIPTION="test nvme update hardware queue count larger than system cpu count" +QUICK=1 + +requires() { + _have_program dd +} + +device_requires() { + _test_dev_is_nvme +} + +test_device() { + echo "Running ${TEST_NAME}" + + local old_write_queues
tests/nvme/033 line 34
Quoted 10 lines
tests/nvme/033 new file +10
-0+ local cur_hw_io_queues + local file + local sys_dev=$TEST_DEV_SYSFS/device + + # backup old module parameter: write_queues + file=/sys/module/nvme/parameters/write_queues + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi
can we skip the test ? I think Omar added a feature to skip the test.
tests/nvme/033 line 42
Quoted 8 lines
tests/nvme/033 new file +8
-0+ old_write_queues="$(cat $file)" + + # get current hardware queue count + file="$sys_dev/queue_count" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi
Same here.
tests/nvme/033 line 50
Quoted 8 lines
tests/nvme/033 new file +8
-0+ cur_hw_io_queues="$(cat "$file")" + # minus admin queue + cur_hw_io_queues=$((cur_hw_io_queues - 1)) + + # set write queues count to increase more hardware queues + file=/sys/module/nvme/parameters/write_queues + echo "$cur_hw_io_queues" > "$file" +
Shouldn't we check if new queue count is set here by reading write_queues ?
tests/nvme/033 line 56
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # reset controller, make it effective + file="$sys_dev/reset_controller" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi
I think we need to add a helper to verify all the files and skip the test required file doesn't exists. Also, please use different variables representing different files.
Quoted 7 lines
tests/nvme/033 new file +7
-0+ echo 1 > "$file" + + # wait nvme reinitialized + for ((m = 0; m < 10; m++)); do + if [[ -b "${TEST_DEV}" ]]; then + break + fi
tests/nvme/033 line 73
Quoted 10 lines
tests/nvme/033 new file +10
-0+ sleep 0.5 + done + if (( m > 9 )); then + echo "nvme still not reinitialized after 5 seconds!" + return 1 + fi + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 status=none +
This should not lead to the kernel panic. Do you have a patch to fix the panic ? If not can you please provide information so that we can fix the panic and make test a test which will not result in panic ?
tests/nvme/033 line 84
Quoted 11 lines
tests/nvme/033 new file +11
-0+ # If all work well restore hardware queue to default + file=/sys/module/nvme/parameters/write_queues + echo "$old_write_queues" > "$file" + + # reset controller + file="$sys_dev/reset_controller" + echo 1 > "$file" + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 iflag=direct status=none + dd if=/dev/zero of="${TEST_DEV}" bs=4096 count=1 oflag=direct status=none
Just a write a helper for dd command instead of hard-coding it.
Quoted 12 lines
tests/nvme/033 new file +3
-0+ + echo "Test complete" +}tests/nvme/033.out new file +2
-0@@ -0,0 +1,2 @@ +Running nvme/033 +Test complete
Quoted 1 line · Chaitanya Kulkarni <[email protected]> 于2020年4月7日周二 下午11:32写道:
Hi Chaitanya,
Thanks your review
Quoted 7 lines
tests/nvme/033 line 34
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # backup old module parameter: write_queues + file=/sys/module/nvme/parameters/write_queues + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fican we skip the test ? I think Omar added a feature to skip the test.
What feature can be used here, I don't see any rc file "set -e".
Quoted 9 lines
tests/nvme/033 line 42
Quoted 8 lines
tests/nvme/033 new file +8
-0+ old_write_queues="$(cat $file)" + + # get current hardware queue count + file="$sys_dev/queue_count" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fiSame here.
Quoted 10 lines
tests/nvme/033 line 50
Quoted 8 lines
tests/nvme/033 new file +8
-0+ cur_hw_io_queues="$(cat "$file")" + # minus admin queue + cur_hw_io_queues=$((cur_hw_io_queues - 1)) + + # set write queues count to increase more hardware queues + file=/sys/module/nvme/parameters/write_queues + echo "$cur_hw_io_queues" > "$file" +Shouldn't we check if new queue count is set here by reading
write_queues?
Most of time, this parameter will not equal to io queue_count, if really so, it will makes this test case be more complicated.
Quoted 9 lines
tests/nvme/033 line 56
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # reset controller, make it effective + file="$sys_dev/reset_controller" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fiI think we need to add a helper to verify all the files and skip the test required file doesn't exists. Also, please use different variables representing different files.
The reason why use same variable name $file, is that copy and paste code(check file exist or not).
If common/rc include "set -e", all these checks can be removed.
Quoted 12 lines
Quoted 12 lines
tests/nvme/033 new file +12
-0+ echo 1 > "$file" + + # wait nvme reinitialized + for ((m = 0; m < 10; m++)); do + if [[ -b "${TEST_DEV}" ]]; then + break + fi + sleep 0.5 + done + if (( m > 9 )); then + echo "nvme still not reinitialized after 5 seconds!" + return 1
Quoted 9 lines
tests/nvme/033 line 73
Quoted 5 lines
tests/nvme/033 new file +5
-0+ fi + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 status=none +This should not lead to the kernel panic. Do you have a patch to fix the panic ? If not can you please provide information so that we can fix the panic and make test a test which will not result in panic ?
The patch is under the review, for more detail please visit:
https://patchwork.kernel.org/patch/11476409/
Quoted 4 lines
Quoted 4 lines
tests/nvme/033 new file +4
-0+ # If all work well restore hardware queue to default + file=/sys/module/nvme/parameters/write_queues + echo "$old_write_queues" > "$file" +
Quoted 8 lines
tests/nvme/033 line 84
Quoted 7 lines
tests/nvme/033 new file +7
-0+ # reset controller + file="$sys_dev/reset_controller" + echo 1 > "$file" + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 iflag=direct status=none + dd if=/dev/zero of="${TEST_DEV}" bs=4096 count=1 oflag=direct status=noneJust a write a helper for dd command instead of hard-coding it.
I think dd is simple enough.
Quoted 4 lines
Quoted 4 lines
tests/nvme/033 new file +3
-0+ + echo "Test complete" +}tests/nvme/033.out +0
-0
Quoted 9 lines
tests/nvme/033.out line 2
Quoted 8 lines
new file mode 100644
index 0000000..9648c73tests/nvme/033.out new file +2
-0@@ -0,0 +1,2 @@ +Running nvme/033 +Test complete
Thanks
Quoted 13 lines · Weiping Zhang
Quoted 1 line · Chaitanya Kulkarni <[email protected]> 于2020年4月7日周二 下午11:32写道:
Hi Chaitanya,
Thanks your reviewQuoted 7 lines
tests/nvme/033 line 34
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # backup old module parameter: write_queues + file=/sys/module/nvme/parameters/write_queues + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fican we skip the test ? I think Omar added a feature to skip the test.
Please have a look this discussion :-
https://www.spinics.net/lists/linux-block/msg50491.html
Quoted 58 lines
What feature can be used here, I don't see any rc file "set -e".
Quoted 9 lines
tests/nvme/033 line 42
Quoted 8 lines
tests/nvme/033 new file +8
-0+ old_write_queues="$(cat $file)" + + # get current hardware queue count + file="$sys_dev/queue_count" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fiSame here.
Quoted 10 lines
tests/nvme/033 line 50
Quoted 8 lines
tests/nvme/033 new file +8
-0+ cur_hw_io_queues="$(cat "$file")" + # minus admin queue + cur_hw_io_queues=$((cur_hw_io_queues - 1)) + + # set write queues count to increase more hardware queues + file=/sys/module/nvme/parameters/write_queues + echo "$cur_hw_io_queues" > "$file" +Shouldn't we check if new queue count is set here by reading
write_queues?Most of time, this parameter will not equal to io
queue_count, if really so, it will makes this test case be more complicated.Quoted 9 lines
tests/nvme/033 line 56
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # reset controller, make it effective + file="$sys_dev/reset_controller" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fiI think we need to add a helper to verify all the files and skip the test required file doesn't exists. Also, please use different variables representing different files.
The reason why use same variable name $file, is that copy and paste code(check file exist or not).
If common/rc include "set -e", all these checks can be removed.
Quoted 12 lines
Quoted 12 lines
tests/nvme/033 new file +12
-0+ echo 1 > "$file" + + # wait nvme reinitialized + for ((m = 0; m < 10; m++)); do + if [[ -b "${TEST_DEV}" ]]; then + break + fi + sleep 0.5 + done + if (( m > 9 )); then + echo "nvme still not reinitialized after 5 seconds!" + return 1Quoted 9 lines
tests/nvme/033 line 73
Quoted 5 lines
tests/nvme/033 new file +5
-0+ fi + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 status=none +This should not lead to the kernel panic. Do you have a patch to fix the panic ? If not can you please provide information so that we can fix the panic and make test a test which will not result in panic ?
Quoted 2 lines
The patch is under the review, for more detail please visit:
https://patchwork.kernel.org/patch/11476409/
We need to wait for the patch to get in first before we add test with fixes tag. I'm not sure is it a good idea to have a test which results in panic when patch in discussion is not in the tree, in that case are testing the known failure.
We need to add a test to validate and pass the fix and make sure as development goes on fixed code is stable.
Tests in blktests should always pass based on the feature or a fix, unless further development adds a regression or has an indirect effect.
Omar any thoughts ?
Quoted 12 lines
Quoted 4 lines
Quoted 4 lines
tests/nvme/033 new file +4
-0+ # If all work well restore hardware queue to default + file=/sys/module/nvme/parameters/write_queues + echo "$old_write_queues" > "$file" +Quoted 8 lines
tests/nvme/033 line 84
Quoted 7 lines
tests/nvme/033 new file +7
-0+ # reset controller + file="$sys_dev/reset_controller" + echo 1 > "$file" + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 iflag=direct status=none + dd if=/dev/zero of="${TEST_DEV}" bs=4096 count=1 oflag=direct status=noneJust a write a helper for dd command instead of hard-coding it.
Quoted 1 line
I think dd is simple enough.
Fine.
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
tests/nvme/033 new file +3
-0+ + echo "Test complete" +}tests/nvme/033.out +0
-0Quoted 9 lines
tests/nvme/033.out line 2
Quoted 8 lines
new file mode 100644
index 0000000..9648c73tests/nvme/033.out new file +2
-0@@ -0,0 +1,2 @@ +Running nvme/033 +Test completeThanks
Quoted 82 lines · Chaitanya Kulkarni
Quoted 13 lines · Weiping Zhang
Quoted 1 line · Chaitanya Kulkarni <[email protected]> 于2020年4月7日周二 下午11:32写道:
Hi Chaitanya,
Thanks your reviewQuoted 7 lines
tests/nvme/033 line 34
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # backup old module parameter: write_queues + file=/sys/module/nvme/parameters/write_queues + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fican we skip the test ? I think Omar added a feature to skip the test.
Please have a look this discussion :-
https://www.spinics.net/lists/linux-block/msg50491.htmlQuoted 58 lines
What feature can be used here, I don't see any rc file "set -e".
Quoted 9 lines
tests/nvme/033 line 42
Quoted 8 lines
tests/nvme/033 new file +8
-0+ old_write_queues="$(cat $file)" + + # get current hardware queue count + file="$sys_dev/queue_count" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fiSame here.
Quoted 10 lines
tests/nvme/033 line 50
Quoted 8 lines
tests/nvme/033 new file +8
-0+ cur_hw_io_queues="$(cat "$file")" + # minus admin queue + cur_hw_io_queues=$((cur_hw_io_queues - 1)) + + # set write queues count to increase more hardware queues + file=/sys/module/nvme/parameters/write_queues + echo "$cur_hw_io_queues" > "$file" +Shouldn't we check if new queue count is set here by reading
write_queues?Most of time, this parameter will not equal to io
queue_count, if really so, it will makes this test case be more complicated.Quoted 9 lines
tests/nvme/033 line 56
Quoted 6 lines
tests/nvme/033 new file +6
-0+ # reset controller, make it effective + file="$sys_dev/reset_controller" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fiI think we need to add a helper to verify all the files and skip the test required file doesn't exists. Also, please use different variables representing different files.
The reason why use same variable name $file, is that copy and paste code(check file exist or not).
If common/rc include "set -e", all these checks can be removed.
Quoted 12 lines
Quoted 12 lines
tests/nvme/033 new file +12
-0+ echo 1 > "$file" + + # wait nvme reinitialized + for ((m = 0; m < 10; m++)); do + if [[ -b "${TEST_DEV}" ]]; then + break + fi + sleep 0.5 + done + if (( m > 9 )); then + echo "nvme still not reinitialized after 5 seconds!" + return 1Quoted 9 lines
tests/nvme/033 line 73
Quoted 5 lines
tests/nvme/033 new file +5
-0+ fi + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 status=none +This should not lead to the kernel panic. Do you have a patch to fix the panic ? If not can you please provide information so that we can fix the panic and make test a test which will not result in panic ?
Quoted 2 lines
The patch is under the review, for more detail please visit:
https://patchwork.kernel.org/patch/11476409/We need to wait for the patch to get in first before we add test with fixes tag. I'm not sure is it a good idea to have a test which results in panic when patch in discussion is not in the tree, in that case are testing the known failure.
Quoted 7 lines
We need to add a test to validate and pass the fix and make sure as development goes on fixed code is stable.
Tests in blktests should always pass based on the feature or a fix, unless further development adds a regression or has an indirect effect.
Omar any thoughts ?
Yeah, it's probably a good idea to wait to merge this until the commit has hit mainline.
Quoted 44 lines · Weiping Zhang
Modify nvme module parameter
write_queuesto change hardware queue count, then reset nvme controller to reinitialize nvme with different queue count.Attention, this test case may trigger a kernel panic.
Signed-off-by Weiping Zhang <[email protected]>--- tests/nvme/033 | 87 ++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/033.out | 2 ++ 2 files changed, 89 insertions(+) create mode 100755 tests/nvme/033 create mode 100644 tests/nvme/033.outtests/nvme/033 new file +24
-0@@ -0,0 +1,87 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0+ +# Copyright (C) 2020 Weiping Zhang <[email protected]> +# +# Test nvme update hardware queue count larger than system cpu count +# + +. tests/nvme/rc + +DESCRIPTION="test nvme update hardware queue count larger than system cpu count" +QUICK=1 + +requires() { + _have_program dd +} + +device_requires() { + _test_dev_is_nvme +} + +test_device() { + echo "Running ${TEST_NAME}" + + local old_write_queues
tests/nvme/033 line 34
Quoted 10 lines
tests/nvme/033 new file +10
-0+ local cur_hw_io_queues + local file + local sys_dev=$TEST_DEV_SYSFS/device + + # backup old module parameter: write_queues + file=/sys/module/nvme/parameters/write_queues + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi
Tests shouldn't fail if kernel support is missing. You can check this with _have_module_param nvme write_queues in requires().
tests/nvme/033 line 42
Quoted 8 lines
tests/nvme/033 new file +8
-0+ old_write_queues="$(cat $file)" + + # get current hardware queue count + file="$sys_dev/queue_count" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi
Again, this should be checked before the test is run (device_requires(), in this case). Something like
if [[ ! -e "$TEST_DEV_SYSFS/device/queue_count" ]]; then
SKIP_REASON="device does have queue_count sysfs attribute" return
fi
Quoted 4 lines
tests/nvme/033 new file +4
-0+ cur_hw_io_queues="$(cat "$file")" + # minus admin queue + cur_hw_io_queues=$((cur_hw_io_queues - 1)) +
tests/nvme/033 line 56
Quoted 10 lines
tests/nvme/033 new file +10
-0+ # set write queues count to increase more hardware queues + file=/sys/module/nvme/parameters/write_queues + echo "$cur_hw_io_queues" > "$file" + + # reset controller, make it effective + file="$sys_dev/reset_controller" + if [[ ! -e "$file" ]]; then + echo "$file does not exist" + return 1 + fi
Same thing here.
tests/nvme/033 line 68
Quoted 12 lines
tests/nvme/033 new file +12
-0+ echo 1 > "$file" + + # wait nvme reinitialized + for ((m = 0; m < 10; m++)); do + if [[ -b "${TEST_DEV}" ]]; then + break + fi + sleep 0.5 + done + if (( m > 9 )); then + echo "nvme still not reinitialized after 5 seconds!" + return 1
You're not resetting the write_queues parameter here.
tests/nvme/033 line 80
Quoted 12 lines
tests/nvme/033 new file +12
-0+ fi + + # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 status=none + + # If all work well restore hardware queue to default + file=/sys/module/nvme/parameters/write_queues + echo "$old_write_queues" > "$file" + + # reset controller + file="$sys_dev/reset_controller" + echo 1 > "$file"
Does this need the same logic to wait for the device to reappear?
Quoted 17 lines
tests/nvme/033 new file +6
-0+ # read data from device (may kernel panic) + dd if="${TEST_DEV}" of=/dev/null bs=4096 count=1 iflag=direct status=none + dd if=/dev/zero of="${TEST_DEV}" bs=4096 count=1 oflag=direct status=none + + echo "Test complete" +}tests/nvme/033.out new file +2
-0@@ -0,0 +1,2 @@ +Running nvme/033 +Test completeSignature · 1 line
2.18.1