More fixes for FC enabling
Subsystems (2)
v2 cover More fixes for FC enabling
I (think) address all the feedback I got from Sagi and Shinichiro except one. The _have_nvme_cli_context() (previously _nvme_cli_support_context()) function is still there. I didn't find any other good way to achieve this and I found in blktests another function doing the same: _have_fio_zbd_zonemode().
Daniel
changes:
v2:
- nvme/048:
- untaggle waiting for state change and queue count check
- make all variables local
- compare numbers not strings
- nvme/rc:
- rename _nvme_cli_supports_context to _have_nvme_cli_context
- only add --context for fc
- reordered setup/cleanup
- nvme/{041,042,043,044,045,048}:
- move all changes related to this patch
v1:
- https://lore.kernel.org/linux-nvme/[email protected]/
Daniel Wagner (3):
nvme/048: Check for queue count check directly nvme/rc: Avoid triggering host nvme-cli autoconnect nvme/{041,042,043,044,045,048}: Use default hostnqn and hostid
tests/nvme/041 | 8 ++--- tests/nvme/042 | 8 ++--- tests/nvme/043 | 8 ++--- tests/nvme/044 | 8 ++--- tests/nvme/045 | 8 ++--- tests/nvme/048 | 32 +++++++++++++------- tests/nvme/rc | 81 +++++++++++++++++++++++++++++++++++++++++++------- 7 files changed, 103 insertions(+), 50 deletions(-)
Signature · 1 line
2.41.0
PATCH v2 1/3 nvme/048: Check for queue count check directly
The test monitored the state changes live -> resetting -> connecting -> live, to figure out the queue count change was successful.
The fc transport is reconnecting very fast and the state transitions are not observed by the current approach.
So instead trying to monitor the state changes, let's just wait for the live state and the correct queue number.
As queue count is depending on the number of online CPUs we explicitly use 1 and 2 for the max_queue count. This means the queue_count value needs to reach either 2 or 3 (admin queue included).
--- tests/nvme/048 | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-)
tests/nvme/048 +20 -4
@@ -42,6 +42,24 @@ nvmf_wait_for_state() { return 0 } +nvmf_check_queue_count() { + local subsys_name="$1" + local queue_count="$2" + local nvmedev + local queue_count_file + + nvmedev=$(_find_nvme_dev "${subsys_name}") + queue_count_file=$(cat /sys/class/nvme-fabrics/ctl/"${nvmedev}"/queue_count) + + queue_count=$((queue_count + 1)) + if [[ "${queue_count}" -ne "${queue_count_file}" ]]; then + echo "expected queue count ${queue_count} not set" + return 1 + fi + + return 0 +} + set_nvmet_attr_qid_max() { local nvmet_subsystem="$1" local qid_max="$2"
@@ -56,10 +74,8 @@ set_qid_max() { local qid_max="$3" set_nvmet_attr_qid_max "${subsys_name}" "${qid_max}" - - # Setting qid_max forces a disconnect and the reconntect attempt starts - nvmf_wait_for_state "${subsys_name}" "connecting" || return 1 nvmf_wait_for_state "${subsys_name}" "live" || return 1 + nvmf_check_queue_count "${subsys_name}" "${qid_max}" || return 1 return 0 }
@@ -103,7 +119,7 @@ test() { echo FAIL else set_qid_max "${port}" "${subsys_name}" 1 || echo FAIL - set_qid_max "${port}" "${subsys_name}" 128 || echo FAIL + set_qid_max "${port}" "${subsys_name}" 2 || echo FAIL fi _nvme_disconnect_subsys "${subsys_name}"
Signature · 1 line
2.41.0
PATCH v2 2/3 nvme/rc: Avoid triggering host nvme-cli autoconnect
In reply to Daniel WagnerWhen the host has enabled the udev/systemd autoconnect services for the fc transport it interacts with blktests and make tests break.
nvme-cli learned to ignore connects attemps when using the --context command line option paired with a volatile configuration. Thus we can mark all the resources created by blktests and avoid any interaction with the systemd autoconnect scripts.
Only enabled this for the fc transport.
--- tests/nvme/rc | 81 ++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 71 insertions(+), 10 deletions(-)
tests/nvme/rc +71 -10
@@ -14,8 +14,8 @@ def_remote_wwnn="0x10001100aa000001" def_remote_wwpn="0x20001100aa000001" def_local_wwnn="0x10001100aa000002" def_local_wwpn="0x20001100aa000002" -def_hostnqn="$(cat /etc/nvme/hostnqn 2> /dev/null)" -def_hostid="$(cat /etc/nvme/hostid 2> /dev/null)" +def_hostnqn="nqn.2014-08.org.nvmexpress:uuid:242d4a24-2484-4a80-8234-d0169409c5e8" +def_hostid="242d4a24-2484-4a80-8234-d0169409c5e8" nvme_trtype=${nvme_trtype:-"loop"} nvme_img_size=${nvme_img_size:-"1G"} nvme_num_iter=${nvme_num_iter:-"1000"}
@@ -161,6 +161,57 @@ _nvme_calc_rand_io_size() { echo "${io_size_kb}k" } +_have_nvme_cli_context() { + # ignore all non-fc transports + if [[ "${nvme_trtype}" == "fc" ]] || + ! nvme connect --help 2>&1 | grep -q -- '--context=<STR>' > /dev/null; then + return 1 + fi + return 0 +} + +_setup_nvme_cli() { + local local_wwnn="${1}" + local local_wwpn="${2}" + local remote_wwnn="${3}" + local remote_wwpn="${4}" + + if ! _have_nvme_cli_context; then + return + fi + + mkdir -p /run/nvme + cat >> /run/nvme/blktests.json <<-EOF + [ + { + "hostnqn": "${def_hostnqn}", + "hostid": "${def_hostid}", + "subsystems": [ + { + "application": "blktests", + "nqn": "blktests-subsystem-1", + "ports": [ + { + "transport": "fc", + "traddr": "nn-${remote_wwnn}:pn-${remote_wwpn}", + "host_traddr": "nn-${local_wwnn}:pn-${local_wwpn}" + } + ] + } + ] + } + ] + EOF +} + +_cleanup_nvme_cli() { + if ! _have_nvme_cli_context; then + return + fi + + rm -f /run/nvme/blktests.json +} + _nvme_fcloop_add_rport() { local local_wwnn="$1" local local_wwpn="$2"
@@ -193,6 +244,9 @@ _setup_fcloop() { local remote_wwnn="${3:-$def_remote_wwnn}" local remote_wwpn="${4:-$def_remote_wwpn}" + _setup_nvme_cli "${local_wwnn}" "${local_wwpn}" \ + "${remote_wwnn}" "${remote_wwpn}" + _nvme_fcloop_add_tport "${remote_wwnn}" "${remote_wwpn}" _nvme_fcloop_add_lport "${local_wwnn}" "${local_wwpn}" _nvme_fcloop_add_rport "${local_wwnn}" "${local_wwpn}" \
@@ -235,6 +289,8 @@ _cleanup_fcloop() { _nvme_fcloop_del_lport "${local_wwnn}" "${local_wwpn}" _nvme_fcloop_del_rport "${local_wwnn}" "${local_wwpn}" \ "${remote_wwnn}" "${remote_wwpn}" + + _cleanup_nvme_cli } _cleanup_nvmet() {
@@ -436,18 +492,18 @@ _nvme_connect_subsys() { trtype="$1" subsysnqn="$2" - ARGS=(-t "${trtype}" -n "${subsysnqn}") + ARGS=(-t "${trtype}" + -n "${subsysnqn}" + --hostnqn="${hostnqn}" + --hostid="${hostid}") + if _have_nvme_cli_context; then + ARGS+=(--context="blktests") + fi if [[ "${trtype}" == "fc" ]] ; then ARGS+=(-a "${traddr}" -w "${host_traddr}") elif [[ "${trtype}" != "loop" ]]; then ARGS+=(-a "${traddr}" -s "${trsvcid}") fi - if [[ "${hostnqn}" != "$def_hostnqn" ]]; then - ARGS+=(--hostnqn="${hostnqn}") - fi - if [[ "${hostid}" != "$def_hostid" ]]; then - ARGS+=(--hostid="${hostid}") - fi if [[ -n "${hostkey}" ]]; then ARGS+=(--dhchap-secret="${hostkey}") fi
@@ -482,7 +538,12 @@ _nvme_discover() { local host_traddr="${3:-$def_host_traddr}" local trsvcid="${3:-$def_trsvcid}" - ARGS=(-t "${trtype}") + ARGS=(-t "${trtype}" + --hostnqn="${def_hostnqn}" + --hostid="${def_hostid}") + if _have_nvme_cli_context; then + ARGS+=(--context="blktests") + fi if [[ "${trtype}" = "fc" ]]; then ARGS+=(-a "${traddr}" -w "${host_traddr}") elif [[ "${trtype}" != "loop" ]]; then
Signature · 1 line
2.41.0
Re: [PATCH blktests v2 2/3] nvme/rc: Avoid triggering host nvme-cli autoconnect
Quoted 31 lines · Daniel Wagner
When the host has enabled the udev/systemd autoconnect services for the fc transport it interacts with blktests and make tests break.
nvme-cli learned to ignore connects attemps when using the --context command line option paired with a volatile configuration. Thus we can mark all the resources created by blktests and avoid any interaction with the systemd autoconnect scripts.
Only enabled this for the fc transport.
Signed-off-by Daniel Wagner <[email protected]>--- tests/nvme/rc | 81 ++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 71 insertions(+), 10 deletions(-)tests/nvme/rc +2
-2@@ -14,8 +14,8 @@ def_remote_wwnn="0x10001100aa000001" def_remote_wwpn="0x20001100aa000001" def_local_wwnn="0x10001100aa000002" def_local_wwpn="0x20001100aa000002" -def_hostnqn="$(cat /etc/nvme/hostnqn 2> /dev/null)" -def_hostid="$(cat /etc/nvme/hostid 2> /dev/null)" +def_hostnqn="nqn.2014-08.org.nvmexpress:uuid:242d4a24-2484-4a80-8234-d0169409c5e8" +def_hostid="242d4a24-2484-4a80-8234-d0169409c5e8" nvme_trtype=${nvme_trtype:-"loop"} nvme_img_size=${nvme_img_size:-"1G"} nvme_num_iter=${nvme_num_iter:-"1000"}
tests/nvme/rc _nvme_calc_rand_io_size() { · line 166
Quoted 6 lines
tests/nvme/rc +3
-0@@ -161,6 +161,57 @@ _nvme_calc_rand_io_size() { echo "${io_size_kb}k" } +_have_nvme_cli_context() { + # ignore all non-fc transports + if [[ "${nvme_trtype}" == "fc" ]] ||
This should be !=, forgot to merge a fixup to this patch.
Quoted 113 lines
tests/nvme/rc +66
-8+ ! nvme connect --help 2>&1 | grep -q -- '--context=<STR>' > /dev/null; then + return 1 + fi + return 0 +} + +_setup_nvme_cli() { + local local_wwnn="${1}" + local local_wwpn="${2}" + local remote_wwnn="${3}" + local remote_wwpn="${4}" + + if ! _have_nvme_cli_context; then + return + fi + + mkdir -p /run/nvme + cat >> /run/nvme/blktests.json <<-EOF + [ + { + "hostnqn": "${def_hostnqn}", + "hostid": "${def_hostid}", + "subsystems": [ + { + "application": "blktests", + "nqn": "blktests-subsystem-1", + "ports": [ + { + "transport": "fc", + "traddr": "nn-${remote_wwnn}:pn-${remote_wwpn}", + "host_traddr": "nn-${local_wwnn}:pn-${local_wwpn}" + } + ] + } + ] + } + ] + EOF +} + +_cleanup_nvme_cli() { + if ! _have_nvme_cli_context; then + return + fi + + rm -f /run/nvme/blktests.json +} + _nvme_fcloop_add_rport() { local local_wwnn="$1" local local_wwpn="$2"@@ -193,6 +244,9 @@ _setup_fcloop() { local remote_wwnn="${3:-$def_remote_wwnn}" local remote_wwpn="${4:-$def_remote_wwpn}" + _setup_nvme_cli "${local_wwnn}" "${local_wwpn}" \ + "${remote_wwnn}" "${remote_wwpn}" + _nvme_fcloop_add_tport "${remote_wwnn}" "${remote_wwpn}" _nvme_fcloop_add_lport "${local_wwnn}" "${local_wwpn}" _nvme_fcloop_add_rport "${local_wwnn}" "${local_wwpn}" \@@ -235,6 +289,8 @@ _cleanup_fcloop() { _nvme_fcloop_del_lport "${local_wwnn}" "${local_wwpn}" _nvme_fcloop_del_rport "${local_wwnn}" "${local_wwpn}" \ "${remote_wwnn}" "${remote_wwpn}" + + _cleanup_nvme_cli } _cleanup_nvmet() {@@ -436,18 +492,18 @@ _nvme_connect_subsys() { trtype="$1" subsysnqn="$2" - ARGS=(-t "${trtype}" -n "${subsysnqn}") + ARGS=(-t "${trtype}" + -n "${subsysnqn}" + --hostnqn="${hostnqn}" + --hostid="${hostid}") + if _have_nvme_cli_context; then + ARGS+=(--context="blktests") + fi if [[ "${trtype}" == "fc" ]] ; then ARGS+=(-a "${traddr}" -w "${host_traddr}") elif [[ "${trtype}" != "loop" ]]; then ARGS+=(-a "${traddr}" -s "${trsvcid}") fi - if [[ "${hostnqn}" != "$def_hostnqn" ]]; then - ARGS+=(--hostnqn="${hostnqn}") - fi - if [[ "${hostid}" != "$def_hostid" ]]; then - ARGS+=(--hostid="${hostid}") - fi if [[ -n "${hostkey}" ]]; then ARGS+=(--dhchap-secret="${hostkey}") fi@@ -482,7 +538,12 @@ _nvme_discover() { local host_traddr="${3:-$def_host_traddr}" local trsvcid="${3:-$def_trsvcid}" - ARGS=(-t "${trtype}") + ARGS=(-t "${trtype}" + --hostnqn="${def_hostnqn}" + --hostid="${def_hostid}") + if _have_nvme_cli_context; then + ARGS+=(--context="blktests") + fi if [[ "${trtype}" = "fc" ]]; then ARGS+=(-a "${traddr}" -w "${host_traddr}") elif [[ "${trtype}" != "loop" ]]; thenSignature · 1 line
2.41.0
PATCH v2 3/3 nvme/{041,042,043,044,045,048}: Use default hostnqn and hostid
In reply to Daniel WagnerThe host might have enabled the udev/systemd auto connect feature. This disturbs the blktests for the fc transport. nvme-cli is able to distinguish between the different invocations via the --context option. In order to get this working we have to use the default hostnqn and hostid and not random generated IDs for every single run.
--- tests/nvme/041 | 8 ++------ tests/nvme/042 | 8 ++------ tests/nvme/043 | 8 ++------ tests/nvme/044 | 8 ++------ tests/nvme/045 | 8 ++------ tests/nvme/048 | 8 ++------ 6 files changed, 12 insertions(+), 36 deletions(-)
tests/nvme/041 +2 -6
@@ -30,12 +30,8 @@ test() { echo "Running ${TEST_NAME}" - hostid="$(uuidgen)" - if [ -z "$hostid" ] ; then - echo "uuidgen failed" - return 1 - fi - hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}" + hostid="${def_hostid}" + hostnqn="${def_hostnqn}" hostkey="$(nvme gen-dhchap-key -n ${subsys_name} 2> /dev/null)" if [ -z "$hostkey" ] ; then echo "nvme gen-dhchap-key failed"
tests/nvme/042 +2 -6
@@ -32,12 +32,8 @@ test() { echo "Running ${TEST_NAME}" - hostid="$(uuidgen)" - if [ -z "$hostid" ] ; then - echo "uuidgen failed" - return 1 - fi - hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}" + hostid="${def_hostid}" + hostnqn="${def_hostnqn}" _setup_nvmet
tests/nvme/043 +2 -6
@@ -33,12 +33,8 @@ test() { echo "Running ${TEST_NAME}" - hostid="$(uuidgen)" - if [ -z "$hostid" ] ; then - echo "uuidgen failed" - return 1 - fi - hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}" + hostid="${def_hostid}" + hostnqn="${def_hostnqn}" _setup_nvmet
tests/nvme/044 +2 -6
@@ -32,12 +32,8 @@ test() { echo "Running ${TEST_NAME}" - hostid="$(uuidgen)" - if [ -z "$hostid" ] ; then - echo "uuidgen failed" - return 1 - fi - hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}" + hostid="${def_hostid}" + hostnqn="${def_hostnqn}" hostkey="$(nvme gen-dhchap-key -n ${subsys_name} 2> /dev/null)" if [ -z "$hostkey" ] ; then
tests/nvme/045 +2 -6
@@ -36,12 +36,8 @@ test() { echo "Running ${TEST_NAME}" - hostid="$(uuidgen)" - if [ -z "$hostid" ] ; then - echo "uuidgen failed" - return 1 - fi - hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}" + hostid="${def_hostid}" + hostnqn="${def_hostnqn}" hostkey="$(nvme gen-dhchap-key -n ${subsys_name} 2> /dev/null)" if [ -z "$hostkey" ] ; then
tests/nvme/048 +2 -6
@@ -93,12 +93,8 @@ test() { _setup_nvmet - hostid="$(uuidgen)" - if [ -z "$hostid" ] ; then - echo "uuidgen failed" - return 1 - fi - hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}" + hostid="${def_hostid}" + hostnqn="${def_hostnqn}" truncate -s "${nvme_img_size}" "${file_path}"
Signature · 1 line
2.41.0
Quoted 4 lines · Daniel Wagner
I (think) address all the feedback I got from Sagi and Shinichiro except one. The
_have_nvme_cli_context()(previously_nvme_cli_support_context()) function is still there. I didn't find any other good way to achieve this and I found in blktests another function doing the same:_have_fio_zbd_zonemode().
I found that the latest blktests fix for the hostnqn/hostid issue created conflict with the 2nd patch in this series. Sorry about that. (Actually, I guess this series could be enough for the issue...)
Daniel, I can fix the conflict and repost the series to the list. Or you can fix the conflict by yourself. Please let me know your preference.
Other than that, I think the three patches are good enough to apply.
Quoted 9 lines · Shinichiro Kawasaki
Quoted 4 lines · Daniel Wagner
I (think) address all the feedback I got from Sagi and Shinichiro except one. The
_have_nvme_cli_context()(previously_nvme_cli_support_context()) function is still there. I didn't find any other good way to achieve this and I found in blktests another function doing the same:_have_fio_zbd_zonemode().I found that the latest blktests fix for the hostnqn/hostid issue created conflict with the 2nd patch in this series. Sorry about that. (Actually, I guess this series could be enough for the issue...)
No problem.
Quoted 2 lines
Daniel, I can fix the conflict and repost the series to the list. Or you can fix the conflict by yourself. Please let me know your preference.
I'll rebase and fixup the conflict. Also give it another quick test run just to make sure all still works.
Quoted 1 line
Other than that, I think the three patches are good enough to apply.
Great!