Add scsi-stress-remove to blktests
v2 cover Add scsi-stress-remove to blktests
Hi,
v2:
There was a minor typo in 0001 pointed out by Omar which I fixed here. 0002 was refactored to use common/scsi_debug. I also rebased onto the top of osandov#josef changing the test from block/022 -> block/027.
From v1:
Ming Lei's scsi-stress-remove test found a bug in blkg destruction [1] where bios being created when the request_queue was being cleaned up threw a NPE in blkg association. The fix is currently being discussed in [2]. To make this test more accessible, I've ported it to blktests with Ming Lei's copyright. I've tested this in my qemu instance and verified we do not see the NPE on a fixed kernel.
Ming, please let me know if you have any objections.
[1] https://lore.kernel.org/linux-block/CACVXFVO_QXipD3cmPvpLyBYSiEcWPN_ThQ=0pO9AwLqN-Lv93w@mail.gmail.com
[2] https://lore.kernel.org/lkml/[email protected]/
This patchset is ontop of osandov#josef 98db2e11c97f.
diffstats below:
Dennis Zhou (2):
blktests: split out cgroup2 controller and file check
blktests: add Ming Lei's scsi-stress-remove
common/cgroup | 18 +++++++---- tests/block/027 | 73 +++++++++++++++++++++++++++++++++++++++++++++ tests/block/027.out | 2 ++ 3 files changed, 88 insertions(+), 5 deletions(-) create mode 100755 tests/block/027 create mode 100644 tests/block/027.out
Thanks,
Dennis
PATCH v2 1/2 blktests: split out cgroup2 controller and file check
This is a prep patch for a new test that will race blkg association and request_queue cleanup. As blkg association is a underlying cgroup io controller feature, we need the ability to check if the controller is available.
---
v2:
- fixed minor typo.
common/cgroup | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
common/cgroup +13 -5
@@ -37,19 +37,27 @@ _have_cgroup2() return 0 } -_have_cgroup2_controller_file() +_have_cgroup2_controller() { - _have_cgroup2 || return 1 - local controller="$1" - local file="$2" - local dir + + _have_cgroup2 || return 1 dir="$(_cgroup2_base_dir)" + if ! grep -q "$controller" "$dir/cgroup.controllers"; then SKIP_REASON="no support for $controller cgroup controller; if it is enabled, you may need to boot with cgroup_no_v1=$controller" return 1 fi +} + +_have_cgroup2_controller_file() +{ + local controller="$1" + local file="$2" + local dir + + _have_cgroup2_controller "$controller" || return 1 mkdir "$dir/blktests" echo "+$controller" > "$dir/cgroup.subtree_control"
Signature · 1 line
2.17.1
PATCH v2 2/2 blktests: add Ming Lei's scsi-stress-remove
In reply to Dennis ZhouThis test exposed a race condiiton when shutting down a request_queue with active IO against it and blkg association for the IOs [1]. The issue ended up being that while the request_queue will just start failing requests, blkg destruction sets the q->root_blkg to %NULL. This caused a NPE. This was fixed in [2].
So to help prevent this from happening again, integrate Ming's test into blktests so that it can more easily be ran. Here I've ported it to fit better into the blktests framework.
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/
---
v2:
- Used helpers in common/scsi_debug per Omar's suggestion.
tests/block/027 | 73 +++++++++++++++++++++++++++++++++++++++++++++ tests/block/027.out | 2 ++ 2 files changed, 75 insertions(+) create mode 100755 tests/block/027 create mode 100644 tests/block/027.out
tests/block/027 new file +73 -0
@@ -0,0 +1,73 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0+ +# Copyright (C) 2018 Ming Lei +# +# Regression test for commit 0273ac349f08 ("blkcg: handle dying request_queue +# when associating a blkg") +# +# This tries to expose the race condition between blkg association and +# request_queue shutdown. When a request_queue is shutdown, the corresponding +# blkgs are destroyed. Any further associations should fail gracefully and not +# cause a kernel panic. + +. tests/block/rc +. common/scsi_debug +. common/cgroup + +DESCRIPTION="test graceful shutdown of scsi_debug devices with running fio jobs" +QUICK=1 + +requires() { + _have_cgroup2_controller io && _have_scsi_debug && _have_fio +} + +scsi_debug_stress_remove() { + if ! _init_scsi_debug "$@"; then + return + fi + + # set higher aio limit + echo 524288 > /proc/sys/fs/aio-max-nr + + local dev fio_jobs scheds + local cnt=1 + for dev in "${SCSI_DEBUG_DEVICES[@]}"; do + # create fio job + fio_jobs=$fio_jobs" --name=job1 --filename=/dev/$dev " + + # set queue scheduler and queue_depth + queue_path=/sys/block/$dev/queue + scheds=($(sed 's/[][]//g' "$queue_path/scheduler")) + sched_idx=$((cnt % ${#scheds[@]})) + echo "${scheds[$sched_idx]}" > "$queue_path/scheduler" + echo $cnt > "$queue_path/../device/queue_depth" + cnt=$((cnt+1)) + done + + local num_jobs=4 runtime=21 + fio --rw=randread --size=128G --direct=1 --ioengine=libaio \ + --iodepth=2048 --numjobs=$num_jobs --bs=4k \ + --group_reporting=1 --group_reporting=1 --runtime=$runtime \ + --loops=10000 "$fio_jobs" > "$FULL" 2>&1 & + + sleep 7 + local device_path + for dev in "${SCSI_DEBUG_DEVICES[@]}"; do + # shutdown devices in progress + device_path=/sys/block/$dev/device + [ -f "$device_path/delete" ] && echo 1 > "$device_path/delete" + done + + wait + + _exit_scsi_debug +} + +test() { + echo "Running ${TEST_NAME}" + + scsi_debug_stress_remove virtual_gb=128 max_luns=21 ndelay=10000 \ + max_queue=110 + + echo "Test complete" +}
tests/block/027.out new file +2 -0
@@ -0,0 +1,2 @@ +Running block/027 +Test complete
Signature · 1 line
2.17.1
Quoted 28 lines · Dennis Zhou
Hi,
v2:
There was a minor typo in 0001 pointed out by Omar which I fixed here. 0002 was refactored to use common/scsi_debug. I also rebased onto the top of osandov#josef changing the test from block/022 -> block/027.From v1:
Ming Lei's scsi-stress-remove test found a bug in blkg destruction [1] where bios being created when therequest_queuewas being cleaned up threw a NPE in blkg association. The fix is currently being discussed in [2]. To make this test more accessible, I've ported it to blktests with Ming Lei's copyright. I've tested this in my qemu instance and verified we do not see the NPE on a fixed kernel.Ming, please let me know if you have any objections.
[1] https://lore.kernel.org/linux-block/CACVXFVO_QXipD3cmPvpLyBYSiEcWPN_ThQ=0pO9AwLqN-Lv93w@mail.gmail.com
[2] https://lore.kernel.org/lkml/[email protected]/This patchset is ontop of osandov#josef
98db2e11c97f.diffstats below:
Dennis Zhou (2):
blktests: split out cgroup2 controller and file check
blktests: add Ming Lei's scsi-stress-remove
Quoted 6 lines
common/cgroup | 18 +++++++---- tests/block/027 | 73 +++++++++++++++++++++++++++++++++++++++++++++ tests/block/027.out | 2 ++ 3 files changed, 88 insertions(+), 5 deletions(-) create mode 100755 tests/block/027 create mode 100644 tests/block/027.out
Thanks, Dennis, applied along with Josef's cgroup infrastructure patch.