Skip to contentSkip to conversation

nvme: Add new test case about nvme rescan/reset/remove during IO

PATCH Superseded Yi Zhang 2 participants 3 messages Sep 9, 2019 – Sep 11, 2019 Block layer

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Yi Zhang

author
#0 lore raw

PATCH v2 1/1 nvme: Add new test case about nvme rescan/reset/remove during IO

Add one test to cover NVMe SSD rescan/reset/remove operation during IO, the steps found several issues during my previous testing, check them here:
http://lists.infradead.org/pipermail/linux-nvme/2017-February/008358.html
http://lists.infradead.org/pipermail/linux-nvme/2017-May/010259.html

---

changes from v1:
- add variable for "/sys/bus/pci/devices/${pdev}"
- add kill $!; wait; for background fio
- add rescan/reset/remove sysfs node check
- add loop checking for nvme reinitialized

---

---
 tests/nvme/031     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/031.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/nvme/031
 create mode 100644 tests/nvme/031.out
tests/nvme/031 new file +71 -0
new file mode 100755
index 0000000..db163a2
@@ -0,0 +1,71 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Yi Zhang <[email protected]>
+#
+# Test nvme pci adapter rescan/reset/remove operation during I/O
+#
+# Regression test for bellow two commits:
+# http://lists.infradead.org/pipermail/linux-nvme/2017-May/010367.html
+# 986f75c876db nvme: avoid to use blk_mq_abort_requeue_list()
+# 806f026f9b90 nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
+
+. tests/nvme/rc
+
+DESCRIPTION="test nvme pci adapter rescan/reset/remove during I/O"
+TIMED=1
+
+requires() {
+	_have_fio
+}
+
+device_requires() {
+	_test_dev_is_nvme
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+
+	pdev="$(_get_pci_dev_from_blkdev)"
+	local sysfs="/sys/bus/pci/devices/${pdev}"
+
+	# start fio job
+	_run_fio_rand_io --filename="$TEST_DEV" --size=1g \
+		--group_reporting  &> /dev/null &
+
+	sleep 5
+
+	# do rescan/reset/remove operation
+	if [[ -f "${sysfs}"/rescan ]]; then
+		echo 1 > "${sysfs}"/rescan
+	else
+		echo "${sysfs}/rescan doesn't exist!"
+	fi
+	# QEMU VM doesn't have the "reset" attribute, skip it
+	if [[ -f "${sysfs}"/reset ]]; then
+		echo 1 > "${sysfs}"/reset
+	fi
+	if [[ -f "${sysfs}"/remove ]]; then
+		echo 1 > "${sysfs}"/remove
+	else
+		echo "${sysfs}/remove doesn't exist!"
+
+	fi
+
+	{ kill $!; wait; } &> /dev/null
+
+	echo 1 > /sys/bus/pci/rescan
+
+	# wait nvme reinitialized
+	local m
+	for ((m = 0; m < 10; m++)); do
+		if [[ -b "${TEST_DEV}" ]]; then
+			break
+		fi
+		sleep 0.5
+	done
+        if (( m > 9 )); then
+                echo "nvme still not reinitialized after 5 seconds!"
+        fi
+
+	echo "Test complete"
+}
tests/nvme/031.out new file +2 -0
new file mode 100644
index 0000000..ae902bd
@@ -0,0 +1,2 @@
+Running nvme/031
+Test complete
Signature · 1 line
2.17.2
#1 lore raw
Quoted 72 lines · Yi Zhang

Add one test to cover NVMe SSD rescan/reset/remove operation during IO, the steps found several issues during my previous testing, check them here:
http://lists.infradead.org/pipermail/linux-nvme/2017-February/008358.html
http://lists.infradead.org/pipermail/linux-nvme/2017-May/010259.html

---

changes from v1:
- add variable for "/sys/bus/pci/devices/${pdev}"
- add kill $!; wait; for background fio
- add rescan/reset/remove sysfs node check
- add loop checking for nvme reinitialized

---

---
  tests/nvme/031     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
  tests/nvme/031.out |  2 ++
  2 files changed, 73 insertions(+)

create mode 100755 tests/nvme/031
create mode 100644 tests/nvme/031.out

tests/nvme/031 new file +43 -0
new file mode 100755
index 0000000..db163a2
@@ -0,0 +1,71 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Yi Zhang <[email protected]>
+#
+# Test nvme pci adapter rescan/reset/remove operation during I/O
+#
+# Regression test for bellow two commits:
+# http://lists.infradead.org/pipermail/linux-nvme/2017-May/010367.html
+# 986f75c876db nvme: avoid to use blk_mq_abort_requeue_list()
+# 806f026f9b90 nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
+
+. tests/nvme/rc
+
+DESCRIPTION="test nvme pci adapter rescan/reset/remove during I/O"
+TIMED=1
+
+requires() {
+	_have_fio
+}
+
+device_requires() {
+	_test_dev_is_nvme
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+
+	pdev="$(_get_pci_dev_from_blkdev)"
+	local sysfs="/sys/bus/pci/devices/${pdev}"
+
+	# start fio job
+	_run_fio_rand_io --filename="$TEST_DEV" --size=1g \
+		--group_reporting  &> /dev/null &
+
+	sleep 5
+
+	# do rescan/reset/remove operation
+	if [[ -f "${sysfs}"/rescan ]]; then
+		echo 1 > "${sysfs}"/rescan
+	else
+		echo "${sysfs}/rescan doesn't exist!"
+	fi
+	# QEMU VM doesn't have the "reset" attribute, skip it

tests/nvme/031 line 52

Quoted 10 lines
tests/nvme/031 new file +9 -0
+	if [[ -f "${sysfs}"/reset ]]; then
+		echo 1 > "${sysfs}"/reset
+	fi
 
+	if [[ -f "${sysfs}"/remove ]]; then
+		echo 1 > "${sysfs}"/remove
+	else
+		echo "${sysfs}/remove doesn't exist!"
+
+	fi

This is a lot of code repetition. You should be creating one helper and passing just file names. (No need to check the return value). something like this :-

check_sysfs()

{
	local sysfs_attr=$1

	if [[ -f "${sysfs_attr}" ]]; then
		echo 1 > "${sysfs_attr}"
	else
		#TODO : add a check to not print if sysfs_attr is not
                 #reset
		echo "${sysfs_attr] doesn't exist!"
	fi
}

and call above function here :-

for i in rescan remove reset; do
check_sysfs_attr $i
done

tests/nvme/031 line 59

Quoted 7 lines
tests/nvme/031 new file +7 -0
+
+	{ kill $!; wait; } &> /dev/null
+
+	echo 1 > /sys/bus/pci/rescan
+
+	# wait nvme reinitialized
+	local m

Please declare all the local variables at the start of the function.

Do we need to call udevadm settle here ?

tests/nvme/031 line 68

Quoted 9 lines
tests/nvme/031 new file +9 -0
+	for ((m = 0; m < 10; m++)); do
+		if [[ -b "${TEST_DEV}" ]]; then
+			break
+		fi
+		sleep 0.5
+	done
+        if (( m > 9 )); then
+                echo "nvme still not reinitialized after 5 seconds!"
+        fi

Please recheck the alignment in the above if.

Quoted 12 lines
tests/nvme/031 new file +3 -0
+
+	echo "Test complete"
+}
tests/nvme/031.out new file +2 -0
new file mode 100644
index 0000000..ae902bd
@@ -0,0 +1,2 @@
+Running nvme/031
+Test complete
author
#2 lore raw

Hi Chaitanya
Thanks for your review.

Quoted 98 lines · Chaitanya Kulkarni
Quoted 71 lines · Yi Zhang

Add one test to cover NVMe SSD rescan/reset/remove operation during IO, the steps found several issues during my previous testing, check them here:
http://lists.infradead.org/pipermail/linux-nvme/2017-February/008358.html
http://lists.infradead.org/pipermail/linux-nvme/2017-May/010259.html

---

changes from v1:
- add variable for "/sys/bus/pci/devices/${pdev}"
- add kill $!; wait; for background fio
- add rescan/reset/remove sysfs node check
- add loop checking for nvme reinitialized

---

---
   tests/nvme/031     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
   tests/nvme/031.out |  2 ++
   2 files changed, 73 insertions(+)

create mode 100755 tests/nvme/031
create mode 100644 tests/nvme/031.out

tests/nvme/031 new file +42 -0
new file mode 100755
index 0000000..db163a2
@@ -0,0 +1,71 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Yi Zhang <[email protected]>
+#
+# Test nvme pci adapter rescan/reset/remove operation during I/O
+#
+# Regression test for bellow two commits:
+# http://lists.infradead.org/pipermail/linux-nvme/2017-May/010367.html
+# 986f75c876db nvme: avoid to use blk_mq_abort_requeue_list()
+# 806f026f9b90 nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
+
+. tests/nvme/rc
+
+DESCRIPTION="test nvme pci adapter rescan/reset/remove during I/O"
+TIMED=1
+
+requires() {
+	_have_fio
+}
+
+device_requires() {
+	_test_dev_is_nvme
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+
+	pdev="$(_get_pci_dev_from_blkdev)"
+	local sysfs="/sys/bus/pci/devices/${pdev}"
+
+	# start fio job
+	_run_fio_rand_io --filename="$TEST_DEV" --size=1g \
+		--group_reporting  &> /dev/null &
+
+	sleep 5
+
+	# do rescan/reset/remove operation
+	if [[ -f "${sysfs}"/rescan ]]; then
+		echo 1 > "${sysfs}"/rescan
+	else
+		echo "${sysfs}/rescan doesn't exist!"
+	fi

tests/nvme/031 line 52

Quoted 10 lines
tests/nvme/031 new file +10 -0
+	# QEMU VM doesn't have the "reset" attribute, skip it
+	if [[ -f "${sysfs}"/reset ]]; then
+		echo 1 > "${sysfs}"/reset
+	fi
+	if [[ -f "${sysfs}"/remove ]]; then
+		echo 1 > "${sysfs}"/remove
+	else
+		echo "${sysfs}/remove doesn't exist!"
+
+	fi

This is a lot of code repetition. You should be creating one helper and passing just file names. (No need to check the return value). something like this :-

check_sysfs()

{
	local sysfs_attr=$1

	if [[ -f "${sysfs_attr}" ]]; then
		echo 1 > "${sysfs_attr}"
	else
		#TODO : add a check to not print if sysfs_attr is not
                  #reset
		echo "${sysfs_attr] doesn't exist!"
	fi
}
Quoted 5 lines

and call above function here :-

for i in rescan remove reset; do
check_sysfs_attr $i
done

will fix it in next version

Quoted 10 lines

tests/nvme/031 line 59

Quoted 7 lines
tests/nvme/031 new file +7 -0
+
+	{ kill $!; wait; } &> /dev/null
+
+	echo 1 > /sys/bus/pci/rescan
+
+	# wait nvme reinitialized
+	local m

Please declare all the local variables at the start of the function.

Do we need to call udevadm settle here ?

agree, will add it.

Quoted 11 lines

tests/nvme/031 line 68

Quoted 10 lines
tests/nvme/031 new file +9 -0
+	for ((m = 0; m < 10; m++)); do
+		if [[ -b "${TEST_DEV}" ]]; then
+			break
+		fi
+		sleep 0.5
+	done
+        if (( m > 9 )); then
+                echo "nvme still not reinitialized after 5 seconds!"
+        fi

Please recheck the alignment in the above if.

Thanks.

Quoted 13 lines
Quoted 13 lines
tests/nvme/031 new file +3 -0
+
+	echo "Test complete"
+}
tests/nvme/031.out new file +2 -0
new file mode 100644
index 0000000..ae902bd
@@ -0,0 +1,2 @@
+Running nvme/031
+Test complete