nvmeof-mp/srp/nvme-rdma misc fix and enhancement
Subsystems (2)
v3 cover nvmeof-mp/srp/nvme-rdma misc fix and enhancement
Hi
This patch series addressed some failures when I run nvmeof-mp/srp test and also add suport to use siw for nvme-rdma/nvmeof-mp/srp testing from cmdline, like this:
$ use_siw=1 nvme-trtype=rdma ./check nvme $ use_siw=1 ./check nvmeof-mp $ use_siw-1 ./check srp
V3:
Use sed to output the scheduler from sysfs directly in patch 3
V2:
Update the ib_srpt module path in patch 1, avoid to use ls Update the SKIP_REASON in patch 2
Introduce get_scheduler_list, fix nvmeof-mp/012 and srp/012 in patch 3 Typo fix in patch 4
Yi Zhang (5):
tests/srp/rc: update the ib_srpt module name tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N nvmeof-mp/012, srp/012: fix the scheduler list common/rc: _have_iproute2 fix for "ip -V" change common/multipath-over-rdma: allow to set use_siw
common/multipath-over-rdma | 13 ++++++++++++- common/rc | 2 +- tests/nvmeof-mp/012 | 10 ++++++---- tests/nvmeof-mp/rc | 8 +++++--- tests/srp/012 | 10 ++++++---- tests/srp/rc | 4 ++-- 6 files changed, 32 insertions(+), 15 deletions(-)
Signature · 7 lines
2.21.0 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
PATCH v3 1/5 tests/srp/rc: update the ib_srpt module name
Fix the ib_srpt module insmod failure as the module in some distros are end with .xz, like bellow on fedora:
/lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko.xz
--- tests/srp/rc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/srp/rc +2 -2
@@ -465,7 +465,7 @@ configure_target_ports() { # Load LIO and configure the SRP target driver and LUNs. start_lio_srpt() { - local b d gid i ini_ids=() opts p target_ids=() vdev + local b d gid i ini_ids=() opts p target_ids=() for gid in $(all_primary_gids); do if [ "${gid#fe8}" != "$gid" ]; then
@@ -500,7 +500,7 @@ start_lio_srpt() { if modinfo ib_srpt | grep -q '^parm:[[:blank:]]*rdma_cm_port:'; then opts+=("rdma_cm_port=${srp_rdma_cm_port}") fi - insmod "/lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko" "${opts[@]}" || return $? + insmod "/lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt."* "${opts[@]}" || return $? i=0 for r in "${vdev_path[@]}"; do if [ -b "$(readlink -f "$r")" ]; then
Signature · 7 lines
2.21.0 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
PATCH v3 2/5 tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N
In reply to Yi ZhangTo enable it, just do bellow step before we run it:
$ echo "options nvme_core multipath=N" >/etc/modprobe.d/nvme.conf
--- tests/nvmeof-mp/rc | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
tests/nvmeof-mp/rc +5 -3
@@ -16,9 +16,11 @@ group_requires() { # Since the nvmeof-mp tests are based on the dm-mpath driver, these # tests are incompatible with the NVME_MULTIPATH kernel configuration - # option. - if _have_kernel_option NVME_MULTIPATH; then - SKIP_REASON="CONFIG_NVME_MULTIPATH has been set in .config" + # option with multipathing enabled in the nvme_core kernel module. + if _have_kernel_option NVME_MULTIPATH && \ + _have_module_param_value nvme_core multipath Y; then + SKIP_REASON="CONFIG_NVME_MULTIPATH has been set in .config \ +and multipathing has been enabled in the nvme_core kernel module" return fi
Signature · 7 lines
2.21.0 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
PATCH v3 3/5 nvmeof-mp/012, srp/012: fix the scheduler list
In reply to Yi ZhangThere is no cfq scheduler and new added kyber scheduler in lastest kernel, introduce get_scheduler_list and fix nvmeof-mp/012 srp/012
To reproduce it:
$ ./check nvmeof-mp/012
nvmeof-mp/012 (dm-mpath on top of multiple I/O schedulers) [passed]
runtime 5.922s ... 8.804s$ cat results/nodev/nvmeof-mp/012.full | grep -n "Changing scheduler" 31:Changing scheduler of dm-3 from mq-deadline kyber [bfq] none into cfq failed 47:Changing scheduler of dm-3 from mq-deadline kyber [bfq] none into cfq failed
--- common/multipath-over-rdma | 11 +++++++++++ tests/nvmeof-mp/012 | 10 ++++++---- tests/srp/012 | 10 ++++++---- 3 files changed, 23 insertions(+), 8 deletions(-)
common/multipath-over-rdma +11 -0
@@ -326,6 +326,17 @@ set_scheduler() { fi } +# Get block dev scheduler list +get_scheduler_list() { + local b=$1 p + p=/sys/block/"$b"/queue/scheduler + if [ -e "$p" ]; then + sed 's/[][]//g' /sys/block/"$b"/queue/scheduler + else + return 1 + fi +} + # Get a /dev/... path that points at dm device number $1. Set its I/O scheduler # to $2 and its timeout to $3. The shell script that includes this file must # define a function get_bdev_path() that translates device number $1 into a
tests/nvmeof-mp/012 +6 -4
@@ -8,7 +8,7 @@ DESCRIPTION="dm-mpath on top of multiple I/O schedulers" QUICK=1 test_io_schedulers() { - local dev m + local dev m dm scheds # Load all I/O scheduler kernel modules for m in "/lib/modules/$(uname -r)/kernel/block/"*.ko; do
@@ -17,15 +17,17 @@ test_io_schedulers() { for mq in y n; do use_blk_mq ${mq} || return $? dev=$(get_bdev 0) || return $? - for sched in noop deadline bfq cfq; do - set_scheduler "$(basename "$(readlink -f "${dev}")")" $sched \ + dm=$(basename "$(readlink -f "${dev}")") || return $? + scheds="$(get_scheduler_list "$dm")" || return $? + for sched in $scheds; do + set_scheduler "$dm" "$sched" \ >>"$FULL" 2>&1 || continue echo "I/O scheduler: $sched; use_blk_mq: $mq" >>"$FULL" run_fio --verify=md5 --rw=randwrite --bs=4K --size=64K \ --ioengine=libaio --iodepth=64 \ --iodepth_batch=32 --group_reporting --sync=1 \ --direct=1 --filename="$dev" \ - --name=${sched} --thread --numjobs=1 \ + --name="${sched}" --thread --numjobs=1 \ --output="${RESULTS_DIR}/nvmeof-mp/012-${sched}-${mq}.txt" \ >>"$FULL" || return $?
tests/srp/012 +6 -4
@@ -8,7 +8,7 @@ DESCRIPTION="dm-mpath on top of multiple I/O schedulers" QUICK=1 test_io_schedulers() { - local dev m + local dev m dm scheds # Load all I/O scheduler kernel modules for m in "/lib/modules/$(uname -r)/kernel/block/"*.ko; do
@@ -18,15 +18,17 @@ test_io_schedulers() { [ $mq = n ] && ! _have_legacy_dm && continue use_blk_mq ${mq} ${mq} || return $? dev=$(get_bdev 0) || return $? - for sched in noop deadline bfq cfq; do - set_scheduler "$(basename "$(readlink -f "${dev}")")" $sched \ + dm=$(basename "$(readlink -f "${dev}")") || return $? + scheds="$(get_scheduler_list "$dm")" || return $? + for sched in $scheds; do + set_scheduler "$dm" "$sched" \ >>"$FULL" 2>&1 || continue echo "I/O scheduler: $sched; use_blk_mq: $mq" >>"$FULL" run_fio --verify=md5 --rw=randwrite --bs=4K --size=64K \ --ioengine=libaio --iodepth=64 \ --iodepth_batch=32 --group_reporting --sync=1 \ --direct=1 --filename="$dev" \ - --name=${sched} --thread --numjobs=1 \ + --name="${sched}" --thread --numjobs=1 \ --output="${RESULTS_DIR}/srp/012-${sched}-${mq}.txt" \ >>"$FULL" || return $?
Signature · 1 line
2.21.0
Re: [PATCH V3 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list
Quoted 2 lines · Yi Zhang
There is no cfq scheduler and new added kyber scheduler in lastest kernel, introduce
get_scheduler_listand fix nvmeof-mp/012 srp/012
PATCH v3 4/5 common/rc: _have_iproute2 fix for "ip -V" change
In reply to Yi ZhangWith below commit, the version will be updated base on the tag fbef6555 replace SNAPSHOT with auto-generated version string
To reproduce it:
$ ./check srp/015 common/rc: line 98: [: ip utility, iproute2-5.9.0: integer expression expected
--- common/rc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
common/rc +1 -1
@@ -95,7 +95,7 @@ _have_iproute2() { SKIP_REASON="ip utility not found" return 1 fi - if [ "$snapshot" -lt "$1" ]; then + if [[ "$snapshot" =~ ^[0-9]+$ && "$snapshot" -lt "$1" ]]; then SKIP_REASON="ip utility too old" return 1 fi
Signature · 1 line
2.21.0
Re: [PATCH V3 blktests 4/5] common/rc: _have_iproute2 fix for "ip -V" change
Quoted 2 lines · Yi Zhang
With below commit, the version will be updated base on the tag
fbef6555replace SNAPSHOT with auto-generated version string
PATCH v3 5/5 common/multipath-over-rdma: allow to set use_siw
In reply to Yi ZhangWith this change, we can change to use siw for nvme-rdma/nvmeof-mp/srp testing from cmdline:
$ use_siw=1 nvme-trtype=rdma ./check nvme/ $ use_siw=1 ./check nvmeof-mp/ $ use_siw=1 ./check srp/
--- common/multipath-over-rdma | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
common/multipath-over-rdma +1 -1
@@ -12,7 +12,7 @@ filesystem_type=ext4 fio_aux_path=/tmp/fio-state-files memtotal=$(sed -n 's/^MemTotal:[[:blank:]]*\([0-9]*\)[[:blank:]]*kB$/\1/p' /proc/meminfo) max_ramdisk_size=$((1<<25)) -use_siw= +use_siw=${use_siw:-""} ramdisk_size=$((memtotal*(1024/16))) # in bytes if [ $ramdisk_size -gt $max_ramdisk_size ]; then ramdisk_size=$max_ramdisk_size
Signature · 1 line
2.21.0
Re: [PATCH V3 blktests 5/5] common/multipath-over-rdma: allow to set use_siw
Quoted 17 lines · Yi Zhang
With this change, we can change to use siw for nvme-rdma/nvmeof-mp/srp testing from cmdline:
$ use_siw=1 nvme-trtype=rdma ./check nvme/ $ use_siw=1 ./check nvmeof-mp/ $ use_siw=1 ./check srp/Signed-off-by Yi Zhang <[email protected]>--- common/multipath-over-rdma | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)common/multipath-over-rdma +0
-0
common/multipath-over-rdma filesystem_type=ext4 · line 18
Quoted 8 lines
common/multipath-over-rdma +1
-1@@ -12,7 +12,7 @@ filesystem_type=ext4 fio_aux_path=/tmp/fio-state-files memtotal=$(sed -n 's/^MemTotal:[[:blank:]]*\([0-9]*\)[[:blank:]]*kB$/\1/p' /proc/meminfo) max_ramdisk_size=$((1<<25)) -use_siw= +use_siw=${use_siw:-""} ramdisk_size=$((memtotal*(1024/16))) # in bytes if [ $ramdisk_size -gt $max_ramdisk_size ]; then ramdisk_size=$max_ramdisk_size
Omar Sandoval
AppliedQuoted 26 lines · Yi Zhang
Hi
This patch series addressed some failures when I run nvmeof-mp/srp test and also add suport to use siw for nvme-rdma/nvmeof-mp/srp testing from cmdline, like this:
$ use_siw=1 nvme-trtype=rdma ./check nvme $ use_siw=1 ./check nvmeof-mp $ use_siw-1 ./check srpV3:
Use sed to output the scheduler from sysfs directly in patch 3V2:
Update theib_srptmodule path in patch 1, avoid to use ls Update theSKIP_REASONin patch 2
Introduceget_scheduler_list, fix nvmeof-mp/012 and srp/012 in patch 3 Typo fix in patch 4Yi Zhang (5):
tests/srp/rc: update theib_srptmodule name tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N nvmeof-mp/012, srp/012: fix the scheduler list common/rc:_have_iproute2fix for "ip -V" change common/multipath-over-rdma: allow to setuse_siw
Quoted 7 lines
common/multipath-over-rdma | 13 ++++++++++++- common/rc | 2 +- tests/nvmeof-mp/012 | 10 ++++++---- tests/nvmeof-mp/rc | 8 +++++--- tests/srp/012 | 10 ++++++---- tests/srp/rc | 4 ++-- 6 files changed, 32 insertions(+), 15 deletions(-)
Applied, thanks!
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme