liburing: test: replace ublk test with kernel selftests
cover liburing: test: replace ublk test with kernel selftests
Hi Jens,
The 1st patch removes the liburing ublk test source, and the 2nd patch adds the test back with the kernel ublk selftest source.
The original test case is covered, and io_uring kernel fixed buffer and ublk zero copy is covered too.
Now the ublk source code is one generic ublk server implementation, and test code is shell script, this way is flexible & easy to add new tests.
Thanks,
Ming
Ming Lei (2):
liburing: test: remove test/uring_cmd_ublk.c
liburing: test: replace ublk test with kernel selftests
.gitignore | 1 +
test/Makefile | 22 +-
test/runtests.sh | 8 +
test/ublk/file_backed.c | 217 ++++++
test/{uring_cmd_ublk.c => ublk/kublk.c} | 925 ++++++++++--------------
test/ublk/kublk.h | 336 +++++++++
test/ublk/null.c | 38 +
test/ublk/test_common.sh | 221 ++++++
test/ublk/test_stress_02.sh | 48 ++
9 files changed, 1286 insertions(+), 530 deletions(-)
create mode 100644 test/ublk/file_backed.c
rename test/{uring_cmd_ublk.c => ublk/kublk.c} (56%)
create mode 100644 test/ublk/kublk.h
create mode 100644 test/ublk/null.c
create mode 100755 test/ublk/test_common.sh
create mode 100755 test/ublk/test_stress_02.shSignature · 1 line
2.47.0
PATCH 1/2 liburing: test: remove test/uring_cmd_ublk.c
Remove test/uring_cmd_ublk.c and prepare for reusing kernel ublk selftest code.
--- test/Makefile | 1 - test/uring_cmd_ublk.c | 1252 ----------------------------------------- 2 files changed, 1253 deletions(-) delete mode 100644 test/uring_cmd_ublk.c
test/Makefile +0 -1
@@ -233,7 +233,6 @@ test_srcs := \ truncate.c \ tty-write-dpoll.c \ unlink.c \ - uring_cmd_ublk.c \ version.c \ waitid.c \ wait-timeout.c \
test/uring_cmd_ublk.c deleted +0 -1252
1,252 lines not on this page · raw message
Signature · 1 line
2.47.0
PATCH 2/2 liburing: test: replace ublk test with kernel selftests
In reply to Ming LeiReplace ublk uring_cmd test with kernel ublk selftests which is designed in more flexible way:
- covers previous stress test for running heavy IO and killing uring task
- covers v6.15 kernel fixed buffer and ublk zero copy feature
- easy to extend
--- .gitignore | 1 + test/Makefile | 21 +- test/runtests.sh | 8 + test/ublk/file_backed.c | 217 +++++++ test/ublk/kublk.c | 1123 +++++++++++++++++++++++++++++++++++ test/ublk/kublk.h | 336 +++++++++++ test/ublk/null.c | 38 ++ test/ublk/test_common.sh | 221 +++++++ test/ublk/test_stress_02.sh | 48 ++ 9 files changed, 2011 insertions(+), 2 deletions(-) create mode 100644 test/ublk/file_backed.c create mode 100644 test/ublk/kublk.c create mode 100644 test/ublk/kublk.h create mode 100644 test/ublk/null.c create mode 100755 test/ublk/test_common.sh create mode 100755 test/ublk/test_stress_02.sh
.gitignore +1 -0
@@ -32,6 +32,7 @@ /test/*.t /test/*.dmesg /test/output/ +/test/ublk/kublk # Clang's compilation database file and directory. /.cache
test/Makefile +19 -2
@@ -276,6 +276,21 @@ test_targets := $(patsubst %,%.t,$(test_targets)) all_targets += $(test_targets) helpers = helpers.o +ifdef CONFIG_HAVE_UBLK_HEADER +ublk_target := ublk/test_stress_02.sh +ublk_exec := ublk/kublk +ublk_install := $(ublk_target) $(ublk_exec) ublk/test_common.sh +test_targets += $(ublk_target) +ublk_srcs = $(wildcard ublk/*.c) +ublk_objs = $(ublk_srcs:.c=.o) +override CFLAGS += -I./ublk/ + +$(ublk_exec): $(ublk_objs) + $(QUIET_CC)$(CC) $(CPPFLAGS) $(CFLAGS) $^ -o $@ $(LDFLAGS) +%.o: $(ublk)/%.c + $(QUIET_CC)$(CC) $(CPPFLAGS) $(CFLAGS) -I./ublk/ -c -o $@ $< +endif + ifeq ($(CONFIG_USE_SANITIZER),y) asan_test_targets := $(patsubst %.c,%,$(asan_test_srcs)) asan_test_targets := $(patsubst %.cc,%,$(asan_test_targets))
@@ -284,7 +299,7 @@ ifeq ($(CONFIG_USE_SANITIZER),y) all_targets += $(asan_test_targets) endif -all: $(test_targets) $(asan_test_targets) +all: $(test_targets) $(asan_test_targets) $(ublk_exec) helpers.o: helpers.c $(QUIET_CC)$(CC) $(CPPFLAGS) $(CFLAGS) -o $@ -c $<
@@ -312,13 +327,15 @@ install: $(test_targets) runtests.sh runtests-loop.sh $(INSTALL) -D -m 755 $(test_targets) $(datadir)/liburing-test/ $(INSTALL) -D -m 755 runtests.sh $(datadir)/liburing-test/ $(INSTALL) -D -m 755 runtests-loop.sh $(datadir)/liburing-test/ + $(INSTALL) -D -m 755 $(ublk_install) $(datadir)/liburing-test/ uninstall: @rm -rf $(datadir)/liburing-test/ clean: - @rm -f $(all_targets) helpers.o output/* @rm -rf output/ + @rm -f $(all_targets) helpers.o + @rm -f ublk/*.o $(ublk_exec) runtests: all @./runtests.sh $(test_targets) $(asan_test_targets)
test/runtests.sh +8 -0
@@ -69,6 +69,7 @@ run_test() local test_exec=("./$test_name") local test_string="$test_name" local out_name="$test_name" + local test_dir # Specify test string to print if [ -n "$dev" ]; then
@@ -135,6 +136,13 @@ run_test() else echo "$T_DIFF sec" fi + + # ublk has its own test dir + test_dir=$(dirname "$out_name") + if [ "$test_dir" != "." ]; then + mkdir -p output/"$test_dir" + fi + echo $T_DIFF > "output/$out_name" fi }
test/ublk/file_backed.c new file +217 -0
217 lines not on this page · raw message
test/ublk/kublk.c new file +1123 -0
1,123 lines not on this page · raw message
test/ublk/kublk.h new file +336 -0
336 lines not on this page · raw message
test/ublk/null.c new file +38 -0
@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include "kublk.h" + +static int ublk_null_tgt_init (struct ublk_dev *dev) +{ +const struct ublksrv_ctrl_dev_info *info = &dev->dev_info; +unsigned long dev_size =250UL <<30 ; + + dev->tgt.dev_size = dev_size; + dev->tgt.params = (struct ublk_params ) { + .types =UBLK_PARAM_TYPE_BASIC , + .basic = { + .logical_bs_shift =9 , + .physical_bs_shift =12 , + .io_opt_shift =12 , + .io_min_shift =9 , + .max_sectors = info->max_io_buf_bytes >>9 , + .dev_sectors = dev_size >>9 , + }, + }; + +return 0 ; +} + +static int ublk_null_queue_io (struct ublk_queue *q,int tag) +{ +const struct ublksrv_io_desc *iod =ublk_get_iod (q, tag); + +ublk_complete_io (q, tag, iod->nr_sectors <<9 ); +return 0 ; +} + +const struct ublk_tgt_ops null_tgt_ops = { + .name ="null" , + .init_tgt = ublk_null_tgt_init, + .queue_io = ublk_null_queue_io, +};
test/ublk/test_common.sh new file +221 -0
221 lines not on this page · raw message
test/ublk/test_stress_02.sh new file +48 -0
48 lines not on this page · raw message
Signature · 1 line
2.47.0
Jens Axboe
maintainerQuoted 10 lines · Ming Lei
Hi Jens,
The 1st patch removes the liburing ublk test source, and the 2nd patch adds the test back with the kernel ublk selftest source.
The original test case is covered, and
io_uringkernel fixed buffer and ublk zero copy is covered too.Now the ublk source code is one generic ublk server implementation, and test code is shell script, this way is flexible & easy to add new tests.
Fails locally here, I think you'll need a few ifdefs for having a not completely uptodate header:
ublk//kublk.c: In function ?cmd_dev_get_features?:ublk//kublk.c:997:30: error: ?UBLK_F_USER_RECOVERY_FAIL_IO? undeclared (first use in this function); did you mean ?UBLK_F_USER_RECOVERY_REISSUE??
997 | [const_ilog2(UBLK_F_USER_RECOVERY_FAIL_IO)] = "RECOVERY_FAIL_IO",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
With
#ifndef UBLK_F_USER_RECOVERY_FAIL_IO
#define UBLK_F_USER_RECOVERY_FAIL_IO (1ULL << 9)
#endif
added it works as expected for me, but might not be a bad idea to include a few more? Looks like there's a good spot for it in kublk.h where there's already something for UBLK_U_IO_REGISTER_IO_BUF.
Outside of that, when running this in my usual vm testing, I see:
Running test ublk/test_stress_02.sh modprobe: FATAL: Module ublk_drv not found in directory /lib/modules/6.14.0-rc7-00360-ge07e8363c5e8
as I have ublk built-in. The test still runs, but would be nice to get rid of that complaint.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerQuoted 30 lines · Jens Axboe
Quoted 10 lines · Ming Lei
Hi Jens,
The 1st patch removes the liburing ublk test source, and the 2nd patch adds the test back with the kernel ublk selftest source.
The original test case is covered, and
io_uringkernel fixed buffer and ublk zero copy is covered too.Now the ublk source code is one generic ublk server implementation, and test code is shell script, this way is flexible & easy to add new tests.
Fails locally here, I think you'll need a few ifdefs for having a not completely uptodate header:
ublk//kublk.c: In function ?cmd_dev_get_features?:ublk//kublk.c:997:30: error: ?UBLK_F_USER_RECOVERY_FAIL_IO? undeclared (first use in this function); did you mean ?UBLK_F_USER_RECOVERY_REISSUE??
997 | [const_ilog2(UBLK_F_USER_RECOVERY_FAIL_IO)] = "RECOVERY_FAIL_IO",| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~With
#ifndef
UBLK_F_USER_RECOVERY_FAIL_IO
#defineUBLK_F_USER_RECOVERY_FAIL_IO(1ULL << 9)
#endifadded it works as expected for me, but might not be a bad idea to include a few more? Looks like there's a good spot for it in
kublk.hwhere there's already something forUBLK_U_IO_REGISTER_IO_BUF.
Quoted 6 lines
Outside of that, when running this in my usual vm testing, I see:
Running test
ublk/test_stress_02.shmodprobe: FATAL: Moduleublk_drvnot found in directory/lib/modules/6.14.0-rc7-00360-ge07e8363c5e8as I have ublk built-in. The test still runs, but would be nice to get rid of that complaint.
Oh, and looks like it should also skip the test if an argument is passed in. My usual setup has 4-5 devices/paths defined for testing, and tests that don't take a file argument should just skip.
Forgot to mention, that unifying the selftests and liburing test is a really good idea! Will make it easier to sync them up and get coverage both ways.
Signature · 1 line
Jens Axboe
Quoted 38 lines · Jens Axboe
Quoted 30 lines · Jens Axboe
Quoted 10 lines · Ming Lei
Hi Jens,
The 1st patch removes the liburing ublk test source, and the 2nd patch adds the test back with the kernel ublk selftest source.
The original test case is covered, and
io_uringkernel fixed buffer and ublk zero copy is covered too.Now the ublk source code is one generic ublk server implementation, and test code is shell script, this way is flexible & easy to add new tests.
Fails locally here, I think you'll need a few ifdefs for having a not completely uptodate header:
ublk//kublk.c: In function ?cmd_dev_get_features?:ublk//kublk.c:997:30: error: ?UBLK_F_USER_RECOVERY_FAIL_IO? undeclared (first use in this function); did you mean ?UBLK_F_USER_RECOVERY_REISSUE??
997 | [const_ilog2(UBLK_F_USER_RECOVERY_FAIL_IO)] = "RECOVERY_FAIL_IO",| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~With
#ifndef
UBLK_F_USER_RECOVERY_FAIL_IO
#defineUBLK_F_USER_RECOVERY_FAIL_IO(1ULL << 9)
#endifadded it works as expected for me, but might not be a bad idea to include a few more? Looks like there's a good spot for it in
kublk.hwhere there's already something forUBLK_U_IO_REGISTER_IO_BUF.Quoted 6 lines
Outside of that, when running this in my usual vm testing, I see:
Running test
ublk/test_stress_02.shmodprobe: FATAL: Moduleublk_drvnot found in directory/lib/modules/6.14.0-rc7-00360-ge07e8363c5e8as I have ublk built-in. The test still runs, but would be nice to get rid of that complaint.
Quoted 8 lines
Oh, and looks like it should also skip the test if an argument is passed in. My usual setup has 4-5 devices/paths defined for testing, and tests that don't take a file argument should just skip.
Forgot to mention, that unifying the selftests and liburing test is a really good idea! Will make it easier to sync them up and get coverage both ways.
Yeah, the kernel selftest side can be thought as upstream, :-)
I just sent three ublk kernel selftest patches, which covers all the above problems. With the three changes, only three lines of code change on test_common.sh is needed for liburing test:
``` #liburing UBLK_TEST_SHOW_RESULT=0 UBLK_SKIP_CODE=77 [ $# -ne 0 ] && exit "$UBLK_SKIP_CODE" ```
Thanks,
Ming
Quoted 22 lines · Jens Axboe
Quoted 10 lines · Ming Lei
Hi Jens,
The 1st patch removes the liburing ublk test source, and the 2nd patch adds the test back with the kernel ublk selftest source.
The original test case is covered, and
io_uringkernel fixed buffer and ublk zero copy is covered too.Now the ublk source code is one generic ublk server implementation, and test code is shell script, this way is flexible & easy to add new tests.
Fails locally here, I think you'll need a few ifdefs for having a not completely uptodate header:
ublk//kublk.c: In function ?cmd_dev_get_features?:ublk//kublk.c:997:30: error: ?UBLK_F_USER_RECOVERY_FAIL_IO? undeclared (first use in this function); did you mean ?UBLK_F_USER_RECOVERY_REISSUE??
997 | [const_ilog2(UBLK_F_USER_RECOVERY_FAIL_IO)] = "RECOVERY_FAIL_IO",| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~With
Quoted 7 lines
#ifndef
UBLK_F_USER_RECOVERY_FAIL_IO
#defineUBLK_F_USER_RECOVERY_FAIL_IO(1ULL << 9)
#endifadded it works as expected for me, but might not be a bad idea to include a few more? Looks like there's a good spot for it in
kublk.hwhere there's already something forUBLK_U_IO_REGISTER_IO_BUF.
It might be easier to duplicate ublk_cmd.h under tests/ublk, hmm?
And just a reminder, liburing and hence this test will often be run on kernels that don't have particular ublk features or where ublk is not implemented / available at all, and we should skip the tests in such cases, that would be the main difference to a typical selftest. But maybe it's already handled, and I worry for nothing.
Signature · 1 line
Pavel Begunkov