ANA and fcloop tests
Subsystems (2)
cover ANA and fcloop tests
Hi Omar,
here are two small tests which have been lingering in my repository for far too long, so I figured it's time to finally post them. The first is a rather trivial one for testing ANA states, the second finally adds a test case for the FC Loop driver.
It also serves as a template on how to actually _use_ the FC Loop driver itself :-)
However, that one uncovered a use-after-free issue in the NVMe-FC stack; check the 'Fixme' line in the test. As it _might_ be a generic issue and not something specific to fcloop I'm posting it now to allow other people (Hi James!) to reproduce and possibly fix it.
As usual, comments and reviews are welcome.
Hannes Reinecke (3):
nvme: enable ANA support
nvme/033: add test for ANA state transition tests/nvme/034: Add a test for the fcloop driver
common/fcloop | 58 ++++++++++++++++++++++ tests/nvme/033 | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/033.out | 18 +++++++ tests/nvme/034 | 122 ++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/034.out | 3 ++ tests/nvme/rc | 88 +++++++++++++++++++++++++++++++-- 6 files changed, 425 insertions(+), 3 deletions(-) create mode 100644 common/fcloop create mode 100644 tests/nvme/033 create mode 100644 tests/nvme/033.out create mode 100644 tests/nvme/034 create mode 100644 tests/nvme/034.out
Signature · 1 line
2.13.7
PATCH 1/3 nvme: enable ANA support
Add ANA support functions and update _create_nvmet_ns() to accept an additional ANA grpid parameter.
--- tests/nvme/rc | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 69 insertions(+), 3 deletions(-)
tests/nvme/rc +69 -3
@@ -14,7 +14,7 @@ group_device_requires() { _test_dev_is_nvme } -NVMET_CFS="/sys/kernel/config/nvmet/" +NVMET_CFS="/sys/kernel/config/nvmet" _test_dev_is_nvme() { if ! readlink -f "$TEST_DEV_SYSFS/device" | grep -q nvme; then
@@ -90,6 +90,7 @@ _create_nvmet_port() { mkdir "${NVMET_CFS}/ports/${port}" echo "${trtype}" > "${NVMET_CFS}/ports/${port}/addr_trtype" + echo "${port}" > "${NVMET_CFS}/ports/${port}/addr_traddr" echo "${port}" }
@@ -99,6 +100,62 @@ _remove_nvmet_port() { rmdir "${NVMET_CFS}/ports/${port}" } +_create_nvmet_anagroup() { + local port="$1" + local port_cfs="${NVMET_CFS}/ports/${port}" + local grpid + + for ((grpid = 1; ; grpid++)); do + if [[ ! -e "${port_cfs}/ana_groups/${grpid}" ]]; then + break + fi + done + + mkdir "${port_cfs}/ana_groups/${grpid}" + + echo "${grpid}" +} + +_remove_nvmet_anagroup() { + local port="$1" + local grpid="$2" + local ana_cfs="${NVMET_CFS}/ports/${port}/ana_groups/${grpid}" + + rmdir "${ana_cfs}" +} + +_set_nvmet_anagroup_state() { + local port="$1" + local grpid="$2" + local state="$3" + local ana_cfs="${NVMET_CFS}/ports/${port}/ana_groups/${grpid}" + + echo "${state}" > "${ana_cfs}/ana_state" +} + +_display_ana_state() { + local nvmet_subsystem="$1" + local grpid state cntlid + + for nvme in /sys/class/nvme/* ; do + [ -f ${nvme}/subsysnqn ] || continue + subsys="$(cat "${nvme}/subsysnqn")" + if [ "${subsys}" != "${nvmet_subsystem}" ] ; then + continue + fi + cntlid="$(cat "${nvme}/cntlid")" + for c in ${nvme}/nvme* ; do + if [ ! -d ${c} ] ; then + echo "${cntlid}: ANA disabled" + continue + fi + grpid="$(cat "${c}/ana_grpid")" + state="$(cat "${c}/ana_state")" + echo "${cntlid}: grpid ${grpid} state ${state}" + done + done +} + _create_nvmet_ns() { local nvmet_subsystem="$1" local nsid="$2"
@@ -106,14 +163,22 @@ _create_nvmet_ns() { local uuid="00000000-0000-0000-0000-000000000000" local subsys_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}" local ns_path="${subsys_path}/namespaces/${nsid}" + local ana_grpid - if [[ $# -eq 4 ]]; then + if [[ $# -ge 4 ]]; then uuid="$4" fi + if [[ $# -eq 5 ]]; then + ana_grpid="$5" + fi + mkdir "${ns_path}" printf "%s" "${blkdev}" > "${ns_path}/device_path" printf "%s" "${uuid}" > "${ns_path}/device_uuid" + if [ -n "${ana_grpid}" ] ; then + printf "%s" "${ana_grpid}" > "${ns_path}/ana_grpid" + fi printf 1 > "${ns_path}/enable" }
@@ -121,11 +186,12 @@ _create_nvmet_subsystem() { local nvmet_subsystem="$1" local blkdev="$2" local uuid=$3 + local ana_grpid=$4 local cfs_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}" mkdir -p "${cfs_path}" echo 1 > "${cfs_path}/attr_allow_any_host" - _create_nvmet_ns "${nvmet_subsystem}" "1" "${blkdev}" "${uuid}" + _create_nvmet_ns "${nvmet_subsystem}" "1" "${blkdev}" "${uuid}" "${ana_grpid}" } _remove_nvmet_ns() {
Signature · 1 line
2.13.7
PATCH 2/3 nvme/033: add test for ANA state transition
In reply to Hannes ReineckeAdd a test to check ANA state transitions.
--- tests/nvme/033 | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/033.out | 18 +++++++ tests/nvme/rc | 16 ++++++ 3 files changed, 173 insertions(+) create mode 100644 tests/nvme/033 create mode 100644 tests/nvme/033.out
tests/nvme/033 new file +139 -0
@@ -0,0 +1,139 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Regression test for ANA base support +# +# Copyright (C) 2018 Hannes Reinecke, SUSE Linux GmbH +# + +. tests/nvme/rc + +DESCRIPTION="test ANA optimized/transitioning/inaccessible support" +QUICK=1 + +switch_nvmet_anagroup() { + local port1="$1" + local port2="$2" + local mode="$3" + + echo "ANA state ${mode}" + + if [ "${mode}" = "change" ] ; then + _set_nvmet_anagroup_state "${port1}" "1" "change" + _set_nvmet_anagroup_state "${port1}" "2" "change" + _set_nvmet_anagroup_state "${port2}" "1" "change" + _set_nvmet_anagroup_state "${port2}" "2" "change" + elif [ "${mode}" = "failover" ] ; then + _set_nvmet_anagroup_state "${port1}" "1" "inaccessible" + _set_nvmet_anagroup_state "${port1}" "2" "optimized" + _set_nvmet_anagroup_state "${port2}" "1" "optimized" + _set_nvmet_anagroup_state "${port2}" "2" "inaccessible" + else + _set_nvmet_anagroup_state "${port1}" "1" "optimized" + _set_nvmet_anagroup_state "${port1}" "2" "inaccessible" + _set_nvmet_anagroup_state "${port2}" "1" "inaccessible" + _set_nvmet_anagroup_state "${port2}" "2" "optimized" + fi +} + +requires() { + _have_program nvme && _have_modules nvme-loop && \ + _have_configfs && _have_fio +} + +test() { + local mount_dir="/mnt/blktests" + local subsys="blktests-subsystem-1" + local port1 + local port2 + local loop_dev1 + local loop_dev2 + local nvmedev + + echo "Running ${TEST_NAME}" + + modprobe nvmet + modprobe nvme-loop + + mkdir -p "${mount_dir}" > /dev/null 2>&1 + + port1="$(_create_nvmet_port "loop")" + ag1="$(_create_nvmet_anagroup "${port1}")" + + port2="$(_create_nvmet_port "loop")" + ag2="$(_create_nvmet_anagroup "${port2}")" + + truncate -s 1G "$TMPDIR/img1" + + loop_dev1="$(losetup -f --show "$TMPDIR/img1")" + + _create_nvmet_subsystem "${subsys}" "${loop_dev1}" \ + "91fdba0d-f87b-4c25-b80f-db7be1418b9e" "1" + + truncate -s 1G "$TMPDIR/img2" + + loop_dev2="$(losetup -f --show "$TMPDIR/img2")" + + _create_nvmet_ns "${subsys}" "2" "${loop_dev2}" \ + "9aed0138-bfd9-46f5-92ac-24c70377fd49" "2" + + _add_nvmet_subsys_to_port "${port1}" "${subsys}" + _add_nvmet_subsys_to_port "${port2}" "${subsys}" + + switch_nvmet_anagroup "${port1}" "${port2}" failback + + nvme connect -t loop -a "${port1}" -n "${subsys}" + nvme connect -t loop -a "${port2}" -n "${subsys}" + + sleep 1 + + _display_ana_state "${subsys}" + + nvmedev="$(_find_nvme_ns 91fdba0d-f87b-4c25-b80f-db7be1418b9e)" + + mkfs.xfs -f /dev/"${nvmedev}" > /dev/null 2>&1 + + mount /dev/"${nvmedev}" "${mount_dir}" + + _run_fio_rand_io --size=32m --directory="${mount_dir}" --runtime=20s --time_based & + trap "kill $!" EXIT + + sleep 10 + + switch_nvmet_anagroup "${port1}" "${port2}" "change" + + # Insert a delay to allow the AEN to be processed + sleep 1 + + _display_ana_state "${subsys}" + + sleep 6 + + switch_nvmet_anagroup "${port1}" "${port2}" "failover" + + # Insert a delay to allow the AEN to be processed + sleep 1 + + _display_ana_state "${subsys}" + + wait + trap - EXIT + + umount "${mount_dir}" > /dev/null 2>&1 + + nvme disconnect -n "${subsys}" + + _remove_nvmet_subsystem_from_port "${port1}" "${subsys}" + _remove_nvmet_subsystem_from_port "${port2}" "${subsys}" + _remove_nvmet_ns "${subsys}" "2" + _remove_nvmet_subsystem "${subsys}" + _remove_nvmet_anagroup "${port1}" "${ag1}" + _remove_nvmet_port "${port1}" + _remove_nvmet_anagroup "${port2}" "${ag2}" + _remove_nvmet_port "${port2}" + losetup -d "${loop_dev2}" + rm "$TMPDIR/img2" + losetup -d "${loop_dev1}" + rm "$TMPDIR/img1" + + echo "Test complete" +}
tests/nvme/033.out new file +18 -0
@@ -0,0 +1,18 @@ +Running nvme/033 +ANA state failback +1: grpid 1 state optimized +1: grpid 2 state inaccessible +2: grpid 1 state inaccessible +2: grpid 2 state optimized +ANA state change +1: grpid 1 state change +1: grpid 2 state change +2: grpid 1 state change +2: grpid 2 state change +ANA state failover +1: grpid 1 state inaccessible +1: grpid 2 state optimized +2: grpid 1 state optimized +2: grpid 2 state inaccessible +NQN:blktests-subsystem-1 disconnected 2 controller(s) +Test complete
tests/nvme/rc +16 -0
@@ -246,6 +246,22 @@ _find_nvme_loop_dev() { done } +_find_nvme_ns() { + local uuid=$1 + local dev + local hidden + + for dev in /sys/block/nvme*; do + hidden="$(cat "${dev}/hidden")" + [[ "$hidden" -eq 1 ]] && continue + u="$(cat "${dev}/uuid")" + if [[ "$uuid" = "$u" ]] ; then + echo "$(basename "${dev}")" + break + fi + done +} + _filter_discovery() { sed -n -r -e "s/Generation counter [0-9]+/Generation counter X/" \ -e '/Discovery Log Number|Log Entry|trtype|subnqn/p'
Signature · 1 line
2.13.7
PATCH 3/3 tests/nvme/034: Add a test for the fcloop driver
In reply to Hannes ReineckeAdd a test for the in-kernel fcloop driver. Despite being a loop driver it still requires an actual FC card to run this test.
--- common/fcloop | 58 +++++++++++++++++++++++++ tests/nvme/034 | 122 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/nvme/034.out | 3 ++ 3 files changed, 183 insertions(+) create mode 100644 common/fcloop create mode 100644 tests/nvme/034 create mode 100644 tests/nvme/034.out
common/fcloop new file +58 -0
@@ -0,0 +1,58 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0+ +# +# fcloop helper functions. + +. common/shellcheck + +_nvme_fcloop_add_rport() { + local local_wwnn="$1" + local local_wwpn="$2" + local remote_wwnn="$3" + local remote_wwpn="$4" + local loopctl=/sys/class/fcloop/ctl + + echo "wwnn=${remote_wwnn},wwpn=${remote_wwpn},lpwwnn=${local_wwnn},lpwwpn=${local_wwpn},roles=0x60" > ${loopctl}/add_remote_port +} + +_nvme_fcloop_del_rport() { + local local_wwnn="$1" + local local_wwpn="$2" + local remote_wwnn="$3" + local remote_wwpn="$4" + local loopctl=/sys/class/fcloop/ctl + + echo "wwnn=${remote_wwnn},wwpn=${remote_wwpn}" > ${loopctl}/del_remote_port +} + +_nvme_fcloop_add_lport() { + local wwnn="$1" + local wwpn="$2" + local loopctl=/sys/class/fcloop/ctl + + echo "wwnn=${wwnn},wwpn=${wwpn}" > ${loopctl}/add_local_port +} + +_nvme_fcloop_del_lport() { + local wwnn="$1" + local wwpn="$2" + local loopctl=/sys/class/fcloop/ctl + + echo "wwnn=${wwnn},wwpn=${wwpn}" > ${loopctl}/del_local_port +} + +_nvme_fcloop_add_tport() { + local wwnn="$1" + local wwpn="$2" + local loopctl=/sys/class/fcloop/ctl + + echo "wwnn=${wwnn},wwpn=${wwpn}" > ${loopctl}/add_target_port +} + +_nvme_fcloop_del_tport() { + local wwnn="$1" + local wwpn="$2" + local loopctl=/sys/class/fcloop/ctl + + echo "wwnn=${wwnn},wwpn=${wwpn}" > ${loopctl}/del_target_port +}
tests/nvme/034 new file +122 -0
122 lines not on this page · raw message
tests/nvme/034.out new file +3 -0
@@ -0,0 +1,3 @@ +Running nvme/034 +NQN:blktests-subsystem-1 disconnected 2 controller(s) +Test complete
Signature · 1 line
2.13.7