Add a libblkio engine
cover Add a libblkio engine
The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like io_uring and vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.
Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.
See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.
[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdf
Alberto Faria (10):
Add a libblkio engine
Add engine flag FIO_SKIPPABLE_IOMEM_ALLOC engines/libblkio: Allow setting option mem/iomem engines/libblkio: Add support for poll queues engines/libblkio: Add option libblkio_vectored engines/libblkio: Add option libblkio_write_zeroes_on_trim engines/libblkio: Add option libblkio_wait_mode engines/libblkio: Add option libblkio_force_enable_completion_eventfd engines/libblkio: Add options for some driver-specific properties engines/libblkio: Share a single blkio instance among threads in same
process
HOWTO.rst | 86 ++ Makefile | 6 + configure | 25 + engines/libblkio.c | 966 ++++++++++++++++++++++ examples/libblkio-io_uring.fio | 29 + examples/libblkio-virtio-blk-vfio-pci.fio | 28 + fio.1 | 74 ++ ioengines.h | 2 + memory.c | 22 +- optgroup.h | 2 + 10 files changed, 1230 insertions(+), 10 deletions(-) create mode 100644 engines/libblkio.c create mode 100644 examples/libblkio-io_uring.fio create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fio
Signature · 1 line
2.38.1
PATCH 1/10 Add a libblkio engine
The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like io_uring and vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.
Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.
See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.
[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdf
--- HOWTO.rst | 26 ++ Makefile | 6 + configure | 25 ++ engines/libblkio.c | 463 ++++++++++++++++++++++ examples/libblkio-io_uring.fio | 19 + examples/libblkio-virtio-blk-vfio-pci.fio | 18 + fio.1 | 19 + optgroup.h | 2 + 8 files changed, 578 insertions(+) create mode 100644 engines/libblkio.c create mode 100644 examples/libblkio-io_uring.fio create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fio
HOWTO.rst +26 -0
@@ -2192,6 +2192,12 @@ I/O engine the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes engine specific options. (See https://xnvme.io). + **libblkio** + Use the libblkio library + (https://gitlab.com/libblkio/libblkio). The specific + *driver* to use must be set using + :option:`libblkio_driver`. + I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -2842,6 +2848,26 @@ with the caveat that when used on the command line, they must come after the If this option is set. xnvme will use vectored read/write commands. +.. option:: libblkio_driver=str : [libblkio] + + The driver to be used by libblkio (e.g. **virtio-blk-vfio-pci**). + +.. option:: libblkio_pre_connect_props=str : [libblkio] + + A colon-separated list of libblkio properties to be set after creating + but before connecting the ``struct blkio``. Each property must have the + format ``<name>=<value>``. Colons can be escaped as ``\:``. These are + set after the engine sets any other properties, so those can be + overriden. + +.. option:: libblkio_pre_start_props=str : [libblkio] + + A colon-separated list of libblkio properties to be set after connecting + but before starting the ``struct blkio``. Each property must have the + format ``<name>=<value>``. Colons can be escaped as ``\:``. These are + set after the engine sets any other properties, so those can be + overriden. + I/O depth ~~~~~~~~~
Makefile +6 -0
@@ -237,6 +237,12 @@ ifdef CONFIG_LIBXNVME xnvme_CFLAGS = $(LIBXNVME_CFLAGS) ENGINES += xnvme endif +ifdef CONFIG_LIBBLKIO + libblkio_SRCS = engines/libblkio.c + libblkio_LIBS = $(LIBBLKIO_LIBS) + libblkio_CFLAGS = $(LIBBLKIO_CFLAGS) + ENGINES += libblkio +endif ifeq ($(CONFIG_TARGET_OS), Linux) SOURCE += diskutil.c fifo.c blktrace.c cgroup.c trim.c engines/sg.c \ oslib/linux-dev-lookup.c engines/io_uring.c engines/nvme.c
configure +25 -0
@@ -176,6 +176,7 @@ libiscsi="no" libnbd="no" libnfs="" xnvme="" +libblkio="" libzbc="" dfs="" seed_buckets=""
@@ -248,6 +249,8 @@ for opt do ;; --disable-xnvme) xnvme="no" ;; + --disable-libblkio) libblkio="no" + ;; --disable-tcmalloc) disable_tcmalloc="yes" ;; --disable-libnfs) libnfs="no"
@@ -304,6 +307,7 @@ if test "$show_help" = "yes" ; then echo "--enable-libiscsi Enable iscsi support" echo "--enable-libnbd Enable libnbd (NBD engine) support" echo "--disable-xnvme Disable xnvme support even if found" + echo "--disable-libblkio Disable libblkio support even if found" echo "--disable-libzbc Disable libzbc even if found" echo "--disable-tcmalloc Disable tcmalloc support" echo "--dynamic-libengines Lib-based ioengines as dynamic libraries"
@@ -2663,6 +2667,22 @@ if test "$xnvme" != "no" ; then fi print_config "xnvme engine" "$xnvme" +########################################## +# Check if we have libblkio +if test "$libblkio" != "no" ; then + if check_min_lib_version blkio 1.0.0; then + libblkio="yes" + libblkio_cflags=$(pkg-config --cflags blkio) + libblkio_libs=$(pkg-config --libs blkio) + else + if test "$libblkio" = "yes" ; then + feature_not_found "libblkio" "libblkio-dev or libblkio-devel" + fi + libblkio="no" + fi +fi +print_config "libblkio engine" "$libblkio" + ########################################## # check march=armv8-a+crc+crypto if test "$march_armv8_a_crc_crypto" != "yes" ; then
@@ -3276,6 +3296,11 @@ if test "$xnvme" = "yes" ; then echo "LIBXNVME_CFLAGS=$xnvme_cflags" >> $config_host_mak echo "LIBXNVME_LIBS=$xnvme_libs" >> $config_host_mak fi +if test "$libblkio" = "yes" ; then + output_sym "CONFIG_LIBBLKIO" + echo "LIBBLKIO_CFLAGS=$libblkio_cflags" >> $config_host_mak + echo "LIBBLKIO_LIBS=$libblkio_libs" >> $config_host_mak +fi if test "$dynamic_engines" = "yes" ; then output_sym "CONFIG_DYNAMIC_ENGINES" fi
engines/libblkio.c new file +463 -0
463 lines not on this page · raw message
examples/libblkio-io_uring.fio new file +19 -0
@@ -0,0 +1,19 @@ +; Benchmark accessing a regular file or block device using libblkio. +; +; Replace "libblkio_path" below with the path to your file or device, or +; override it by passing the '--libblkio_pre_connect_props=path=...' flag to +; fio. +; +; For information on libblkio, see: https://gitlab.com/libblkio/libblkio + +[global] +ioengine=libblkio +libblkio_driver=io_uring +libblkio_pre_connect_props=path=/dev/nvme0n1 ; REPLACE THIS WITH THE RIGHT PATH +rw=randread +blocksize=4k +direct=1 +time_based=1 +runtime=10s + +[job]
examples/libblkio-virtio-blk-vfio-pci.fio new file +18 -0
@@ -0,0 +1,18 @@ +; Benchmark accessing a PCI virtio-blk device using libblkio. +; +; Replace "libblkio_path" below with the path to your device's sysfs directory, +; or override it by passing the '--libblkio_pre_connect_props=path=...' flag to +; fio. Note that colons in the path must be escaped with a backslash. +; +; For information on libblkio, see: https://gitlab.com/libblkio/libblkio + +[global] +ioengine=libblkio +libblkio_driver=virtio-blk-vfio-pci +libblkio_pre_connect_props=path=/sys/bus/pci/devices/0000\:00\:01.0 ; REPLACE THIS WITH THE RIGHT PATH +rw=randread +blocksize=4k +time_based=1 +runtime=10s + +[job]
fio.1 +19 -0
@@ -1989,6 +1989,10 @@ I/O engine using the xNVMe C API, for NVMe devices. The xnvme engine provides flexibility to access GNU/Linux Kernel NVMe driver via libaio, IOCTLs, io_uring, the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes engine specific options. (See \fIhttps://xnvme.io/\fR). +.TP +.B libblkio +Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The +specific \fBdriver\fR to use must be set using \fBlibblkio_driver\fR. .SS "I/O engine specific parameters" In addition, there are some parameters which are only valid when a specific \fBioengine\fR is in use. These are used identically to normal parameters,
@@ -2599,6 +2603,21 @@ xnvme namespace identifier for userspace NVMe driver such as SPDK. .TP .BI (xnvme)xnvme_iovec If this option is set, xnvme will use vectored read/write commands. +.TP +.BI (libblkio)libblkio_driver \fR=\fPstr +The driver to be used by libblkio (e.g. \fBvirtio-blk-vfio-pci\fR). +.TP +.BI (libblkio)libblkio_pre_connect_props \fR=\fPstr +A colon-separated list of libblkio properties to be set after creating but +before connecting the \fBstruct blkio\fR. Each property must have the format +\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after +the engine sets any other properties, so those can be overriden. +.TP +.BI (libblkio)libblkio_pre_start_props \fR=\fPstr +A colon-separated list of libblkio properties to be set after connecting but +before starting the \fBstruct blkio\fR. Each property must have the format +\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after +the engine sets any other properties, so those can be overriden. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
optgroup.h +2 -0
@@ -73,6 +73,7 @@ enum opt_category_group { __FIO_OPT_G_NFS, __FIO_OPT_G_WINDOWSAIO, __FIO_OPT_G_XNVME, + __FIO_OPT_G_LIBBLKIO,FIO_OPT_G_RATE = (1ULL << __FIO_OPT_G_RATE), FIO_OPT_G_ZONE = (1ULL << __FIO_OPT_G_ZONE),
@@ -120,6 +121,7 @@ enum opt_category_group {FIO_OPT_G_DFS = (1ULL << __FIO_OPT_G_DFS),FIO_OPT_G_WINDOWSAIO = (1ULL << __FIO_OPT_G_WINDOWSAIO),FIO_OPT_G_XNVME = (1ULL << __FIO_OPT_G_XNVME), + FIO_OPT_G_LIBBLKIO = (1ULL << __FIO_OPT_G_LIBBLKIO), };extern const struct opt_group *opt_group_from_mask (uint64_t *mask);
Signature · 1 line
2.38.1
Quoted 51 lines · Alberto Faria
The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like
io_uringand vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.
See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.
[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdfSigned-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 26 ++ Makefile | 6 + configure | 25 ++ engines/libblkio.c | 463 ++++++++++++++++++++++ examples/libblkio-io_uring.fio | 19 + examples/libblkio-virtio-blk-vfio-pci.fio | 18 + fio.1 | 19 + optgroup.h | 2 + 8 files changed, 578 insertions(+) create mode 100644 engines/libblkio.c create mode 100644 examples/libblkio-io_uring.fio create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fioHOWTO.rst +6
-0@@ -2192,6 +2192,12 @@ I/O engine the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes engine specific options. (See https://xnvme.io). + **libblkio** + Use the libblkio library + (https://gitlab.com/libblkio/libblkio). The specific + *driver* to use must be set using + :option:`libblkio_driver`. + I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2853
Quoted 6 lines
HOWTO.rst +3
-0@@ -2842,6 +2848,26 @@ with the caveat that when used on the command line, they must come after the If this option is set. xnvme will use vectored read/write commands. +.. option:: libblkio_driver=str : [libblkio] + + The driver to be used by libblkio (e.g. **virtio-blk-vfio-pci**).
It would be nice to list the possible values and their meaning here.
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2861
Quoted 8 lines
HOWTO.rst +8
-0+ +.. option:: libblkio_pre_connect_props=str : [libblkio] + + A colon-separated list of libblkio properties to be set after creating + but before connecting the ``struct blkio``. Each property must have the + format ``<name>=<value>``. Colons can be escaped as ``\:``. These are + set after the engine sets any other properties, so those can be + overriden.
"struct blkio" has no meaning whatsoever for the fio command line interface. So could this be reworded without using struct names ? E.g. "without connecting the device accessed by libblkio" ?
And we need to list the possible properties or have at least a link to some documentation listing the possible properties. Otherwise, how can the user find that out ?
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2869
Quoted 8 lines
HOWTO.rst +8
-0+ +.. option:: libblkio_pre_start_props=str : [libblkio] + + A colon-separated list of libblkio properties to be set after connecting + but before starting the ``struct blkio``. Each property must have the + format ``<name>=<value>``. Colons can be escaped as ``\:``. These are + set after the engine sets any other properties, so those can be + overriden.
Same comment as above.
Quoted 157 lines
HOWTO.rst +1
-0+ I/O depth ~~~~~~~~~Makefile +6
-0@@ -237,6 +237,12 @@ ifdef CONFIG_LIBXNVME xnvme_CFLAGS = $(LIBXNVME_CFLAGS) ENGINES += xnvme endif +ifdef CONFIG_LIBBLKIO + libblkio_SRCS = engines/libblkio.c + libblkio_LIBS = $(LIBBLKIO_LIBS) + libblkio_CFLAGS = $(LIBBLKIO_CFLAGS) + ENGINES += libblkio +endif ifeq ($(CONFIG_TARGET_OS), Linux) SOURCE += diskutil.c fifo.c blktrace.c cgroup.c trim.c engines/sg.c \ oslib/linux-dev-lookup.c engines/io_uring.c engines/nvme.cconfigure +25
-0@@ -176,6 +176,7 @@ libiscsi="no" libnbd="no" libnfs="" xnvme="" +libblkio="" libzbc="" dfs="" seed_buckets=""@@ -248,6 +249,8 @@ for opt do ;; --disable-xnvme) xnvme="no" ;; + --disable-libblkio) libblkio="no" + ;; --disable-tcmalloc) disable_tcmalloc="yes" ;; --disable-libnfs) libnfs="no"@@ -304,6 +307,7 @@ if test "$show_help" = "yes" ; then echo "--enable-libiscsi Enable iscsi support" echo "--enable-libnbd Enable libnbd (NBD engine) support" echo "--disable-xnvme Disable xnvme support even if found" + echo "--disable-libblkio Disable libblkio support even if found" echo "--disable-libzbc Disable libzbc even if found" echo "--disable-tcmalloc Disable tcmalloc support" echo "--dynamic-libengines Lib-based ioengines as dynamic libraries"@@ -2663,6 +2667,22 @@ if test "$xnvme" != "no" ; then fi print_config "xnvme engine" "$xnvme" +########################################## +# Check if we have libblkio +if test "$libblkio" != "no" ; then + if check_min_lib_version blkio 1.0.0; then + libblkio="yes" + libblkio_cflags=$(pkg-config --cflags blkio) + libblkio_libs=$(pkg-config --libs blkio) + else + if test "$libblkio" = "yes" ; then + feature_not_found "libblkio" "libblkio-dev or libblkio-devel" + fi + libblkio="no" + fi +fi +print_config "libblkio engine" "$libblkio" + ########################################## # check march=armv8-a+crc+crypto if test "$march_armv8_a_crc_crypto" != "yes" ; then@@ -3276,6 +3296,11 @@ if test "$xnvme" = "yes" ; then echo "LIBXNVME_CFLAGS=$xnvme_cflags" >> $config_host_mak echo "LIBXNVME_LIBS=$xnvme_libs" >> $config_host_mak fi +if test "$libblkio" = "yes" ; then + output_sym "CONFIG_LIBBLKIO" + echo "LIBBLKIO_CFLAGS=$libblkio_cflags" >> $config_host_mak + echo "LIBBLKIO_LIBS=$libblkio_libs" >> $config_host_mak +fi if test "$dynamic_engines" = "yes" ; then output_sym "CONFIG_DYNAMIC_ENGINES" fiengines/libblkio.c new file +66
-0@@ -0,0 +1,463 @@ +/* +* libblkio engine +* +* IO engine using libblkio to access various block I/O interfaces: +* https://gitlab.com/libblkio/libblkio +*/ + +#include <assert.h> +#include <errno.h> +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <blkio.h> + +#include "../fio.h" +#include "../optgroup.h" +#include "../options.h" +#include "../parse.h" + +struct fio_blkio_options { +void *pad;/* option fields must not have offset 0 */ + +char *driver; +char *pre_connect_props; +char *pre_start_props; +}; + +static struct fio_option options[] = { + { + .name ="libblkio_driver" , + .lname ="libblkio driver name" , + .type =FIO_OPT_STR_STORE , + .off1 = offsetof(struct fio_blkio_options , driver), + .help ="Name of the driver to be used by libblkio" , + .category =FIO_OPT_C_ENGINE , + .group =FIO_OPT_G_LIBBLKIO , + }, + { + .name ="libblkio_pre_connect_props" , + .lname ="Properties to be set before blkio_connect()" , + .type =FIO_OPT_STR_STORE , + .off1 = offsetof(struct fio_blkio_options , pre_connect_props), + .help ="" , + .category =FIO_OPT_C_ENGINE , + .group =FIO_OPT_G_LIBBLKIO , + }, + { + .name ="libblkio_pre_start_props" , + .lname ="Properties to be set before blkio_start()" , + .type =FIO_OPT_STR_STORE , + .off1 = offsetof(struct fio_blkio_options , pre_start_props), + .help ="" , + .category =FIO_OPT_C_ENGINE , + .group =FIO_OPT_G_LIBBLKIO , + }, + + { + .name =NULL , + }, +}; + +static int fio_blkio_set_props_from_str (struct blkio *b,const char *opt_name, +const char *str) {
engines/libblkio.c line 76
Quoted 10 lines
engines/libblkio.c new file +10
-0+int ret =0 ; +char *new_str, *name, *value; + +if (!str) +return 0 ; + +/* copy string */ + + new_str =strdup (str); +
No need for the blank lines around the above line.
engines/libblkio.c line 83
Quoted 7 lines
engines/libblkio.c new file +7
-0+if (!new_str) { +log_err ("fio: strdup() failed\n" ); +return 1 ; + } + +/* iterate over property name-value pairs */ +
No need for the blank line here either.
engines/libblkio.c line 88
Quoted 5 lines
engines/libblkio.c new file +5
-0+while ((name =get_next_str (&new_str))) { +/* split into property name and value */ + + value =strchr (name,'=' ); +
Same comment.
Quoted 4 lines
engines/libblkio.c new file +4
-0+if (!value) { +log_err ("fio: missing '=' in option %s\n" , opt_name); + ret =1 ; +break ;
engines/libblkio.c line 102
Quoted 10 lines
engines/libblkio.c new file +10
-0+ } + + *value ='\0' ; + ++value; + +/* strip whitespace from property name and value */ + +strip_blank_front (&name); +strip_blank_end (name); +
same.
engines/libblkio.c line 114
Quoted 12 lines
engines/libblkio.c new file +12
-0+if (name[0 ] =='\0' ) { +log_err ("fio: empty property name in option %s\n" , + opt_name); + ret =1 ; +break ; + } + +strip_blank_front (&value); +strip_blank_end (value); + +/* set property */ +
again here too.
Quoted 23 lines
engines/libblkio.c new file +23
-0+if (blkio_set_str (b, name, value) !=0 ) { +log_err ("fio: error setting property '%s' to '%s': %s\n" , + name, value,blkio_get_error_msg ()); + ret =1 ; +break ; + } + } + +free (new_str); +return ret; +} + +/* +* Log the failure of a libblkio function. +* +* `(void)func` is to ensure `func` exists and prevent typos +*/ +#define fio_blkio_log_err (func) \ + ({ \ + (void)func; \ + log_err("fio: %s() failed: %s\n", #func, \ + blkio_get_error_msg()); \ + })
engines/libblkio.c line 147
Quoted 10 lines
engines/libblkio.c new file +10
-0+ +static int fio_blkio_create_and_connect (struct thread_data *td, +struct blkio **out_blkio) +{ +const struct fio_blkio_options *options = td->eo; +struct blkio *b; +int ret; + +/* create blkio */ +
Useless blank line.
engines/libblkio.c line 160
Quoted 13 lines
engines/libblkio.c new file +13
-0+if (!options->driver) { +log_err ("fio: engine libblkio requires option libblkio_driver to be set\n" ); +return 1 ; + } + +if (blkio_create (options->driver, &b) !=0 ) { +fio_blkio_log_err (blkio_create); +return 1 ; + } + +/* set pre-connect properties */ + +/* don't fail if driver doesn't have a "direct" property */
Use multi-line comment ?
Quoted 7 lines
engines/libblkio.c new file +7
-0+ ret =blkio_set_bool (b,"direct" , td->o.odirect); +if (ret !=0 && ret != -ENOENT ) { +fio_blkio_log_err (blkio_set_bool); + goto err_blkio_destroy; + } + +if (blkio_set_bool (b,"read-only" , read_only) !=0 ) {
engines/libblkio.c line 177
Quoted 10 lines
engines/libblkio.c new file +10
-0+fio_blkio_log_err (blkio_set_bool); + goto err_blkio_destroy; + } + +if (fio_blkio_set_props_from_str (b,"libblkio_pre_connect_props" , + options->pre_connect_props) !=0 ) + goto err_blkio_destroy; + +/* connect blkio */ +
No need for this blnak line either I think... Stopping here about blank lines. Matter of taste I guess, but I think you have too many of them, making the code somewhat unpleasant to read.
Quoted 18 lines
engines/libblkio.c new file +18
-0+if (blkio_connect (b) !=0 ) { +fio_blkio_log_err (blkio_connect); + goto err_blkio_destroy; + } + +/* set pre-start properties */ + +if (fio_blkio_set_props_from_str (b,"libblkio_pre_start_props" , + options->pre_start_props) !=0 ) + goto err_blkio_destroy; + + *out_blkio = b; +return 0 ; + +err_blkio_destroy: +blkio_destroy (&b); +return 1 ; +}
engines/libblkio.c line 205
Quoted 10 lines
engines/libblkio.c new file +10
-0+ +static int fio_blkio_setup (struct thread_data *td) +{ +/* +* We have to determine device/file size here, so we create and connect +* a blkio instance. But this callback is called from the main thread in +* the original fio process, not from the processes in which jobs will +* actually run. We thus subsequently destroy the blkio and create it +* again in the init() callback. +*/
Unusual place for a comment. Move this above the function declaration to explain what the function does.
Quoted 28 lines
engines/libblkio.c new file +28
-0+ +struct blkio *b; +int ret =0 ; +uint64_t capacity; + +assert (td->files_index ==1 ); + +/* get target size */ + +if (fio_blkio_create_and_connect (td, &b) !=0 ) +return 1 ; + +if (blkio_get_uint64 (b,"capacity" , &capacity) !=0 ) { +fio_blkio_log_err (blkio_get_uint64); + ret =1 ; + goto out_blkio_destroy; + } + +/* set file size for fio */ + + td->files[0 ]->real_file_size = capacity; +fio_file_set_size_known (td->files[0 ]); + +out_blkio_destroy: +blkio_destroy (&b); +return ret; +} +
engines/libblkio.c line 243
Quoted 10 lines
engines/libblkio.c new file +10
-0+/* per-thread state */ +struct fio_blkio_data { +struct blkio *b; +struct blkioq *q; + +bool has_mem_region;/* whether mem_region is valid */ +struct blkio_mem_region mem_region; + +struct blkio_completion *completions; +};
Why not have this declaration at the beginning of the file ?
engines/libblkio.c line 254
Quoted 11 lines
engines/libblkio.c new file +11
-0+ +static int fio_blkio_init (struct thread_data *td) +{ +struct fio_blkio_data *data; + +/* allocate per-thread data struct */ + + data =calloc (1 ,sizeof (*data)); +if (!data) { +log_err ("fio: calloc() failed\n" ); + goto err_free;
This will seg-fault as err_free first does "free(data->completions)". Return here.
Quoted 14 lines
engines/libblkio.c new file +14
-0+ } + + data->completions =calloc (td->o.iodepth,sizeof (data->completions[0 ])); +if (!data->completions) { +log_err ("fio: calloc() failed\n" ); + goto err_free; + } + +/* create, connect, and start blkio */ + +if (fio_blkio_create_and_connect (td, &data->b) !=0 ) + goto err_free; + +if (blkio_set_int (data->b,"num-queues" ,1 ) !=0 ) {
engines/libblkio.c line 278
Quoted 10 lines
engines/libblkio.c new file +10
-0+fio_blkio_log_err (blkio_set_int); + goto err_blkio_destroy; + } + +if (blkio_start (data->b) !=0 ) { +fio_blkio_log_err (blkio_start); + goto err_blkio_destroy; + } + +/* get queue */
Useless comment.
Quoted 291 lines
engines/libblkio.c new file +185
-0+ + data->q =blkio_get_queue (data->b,0 ); + +/* Set data only here so cleanup() does nothing if init() fails. */ + td->io_ops_data = data; + +return 0 ; + +err_blkio_destroy: +blkio_destroy (&data->b); +err_free: +free (data->completions); +free (data); +return 1 ; +} + +static void fio_blkio_cleanup (struct thread_data *td) +{ +struct fio_blkio_data *data = td->io_ops_data; + +if (data) { +blkio_destroy (&data->b); +free (data->completions); +free (data); + } +} + +#define align_up (x, y) ((((x) + (y) - 1) / (y)) * (y)) + +static int fio_blkio_iomem_alloc (struct thread_data *td,size_t size) +{ +struct fio_blkio_data *data = td->io_ops_data; +int ret; +uint64_t mem_region_alignment; +size_t aligned_size; + +/* round up size to satisfy mem-region-alignment */ + +if (blkio_get_uint64 (data->b,"mem-region-alignment" , + &mem_region_alignment) !=0 ) { +fio_blkio_log_err (blkio_get_uint64); +return 1 ; + } + + aligned_size =align_up (size, (size_t )mem_region_alignment); + +/* allocate memory region */ + +if (blkio_alloc_mem_region (data->b, &data->mem_region, + aligned_size) !=0 ) { +fio_blkio_log_err (blkio_alloc_mem_region); + ret =1 ; + goto out; + } + +if (blkio_map_mem_region (data->b, &data->mem_region) !=0 ) { +fio_blkio_log_err (blkio_map_mem_region); + ret =1 ; + goto out_free; + } + + td->orig_buffer = data->mem_region.addr; + data->has_mem_region = true; + + ret =0 ; + goto out; + +out_free: +blkio_free_mem_region (data->b, &data->mem_region); +out: +return ret; +} + +static void fio_blkio_iomem_free (struct thread_data *td) +{ +struct fio_blkio_data *data = td->io_ops_data; + +if (data && data->has_mem_region) { +blkio_unmap_mem_region (data->b, &data->mem_region); +blkio_free_mem_region (data->b, &data->mem_region); + + data->has_mem_region = false; + } +} + +static int fio_blkio_open_file (struct thread_data *td,struct fio_file *f) +{ +return 0 ; +} + +static enum fio_q_status fio_blkio_queue (struct thread_data *td, +struct io_u *io_u) +{ +struct fio_blkio_data *data = td->io_ops_data; + +fio_ro_check (td, io_u); + +switch (io_u->ddir) { +case DDIR_READ : +blkioq_read (data->q, io_u->offset, io_u->xfer_buf, + (size_t )io_u->xfer_buflen, io_u,0 ); +break ; + +case DDIR_WRITE : +blkioq_write (data->q, io_u->offset, io_u->xfer_buf, + (size_t )io_u->xfer_buflen, io_u,0 ); +break ; + +case DDIR_TRIM : +blkioq_discard (data->q, io_u->offset, io_u->xfer_buflen, + io_u,0 ); +break ; + +case DDIR_SYNC : +case DDIR_DATASYNC : +blkioq_flush (data->q, io_u,0 ); +break ; + +default : + io_u->error =ENOTSUP ; +io_u_log_error (td, io_u); +return FIO_Q_COMPLETED ; + } + +return FIO_Q_QUEUED ; +} + +static int fio_blkio_getevents (struct thread_data *td,unsigned int min, +unsigned int max,const struct timespec *t) +{ +struct fio_blkio_data *data = td->io_ops_data; +int n; + + n =blkioq_do_io (data->q, data->completions, (int )min, (int )max,NULL ); +if (n <0 ) { +fio_blkio_log_err (blkioq_do_io); +return -1 ; + } + +return n; +} + +static struct io_u *fio_blkio_event (struct thread_data *td,int event) +{ +struct fio_blkio_data *data = td->io_ops_data; +struct blkio_completion *completion = &data->completions[event]; +struct io_u *io_u = completion->user_data; + + io_u->error = -completion->ret; + +return io_u; +} + +FIO_STATIC struct ioengine_ops ioengine = { + .name ="libblkio" , + .version =FIO_IOOPS_VERSION , + .flags =FIO_DISKLESSIO |FIO_NOEXTEND | +FIO_NO_OFFLOAD , + + .setup = fio_blkio_setup, + .init = fio_blkio_init, + .cleanup = fio_blkio_cleanup, + + .iomem_alloc = fio_blkio_iomem_alloc, + .iomem_free = fio_blkio_iomem_free, + + .open_file = fio_blkio_open_file, + + .queue = fio_blkio_queue, + .getevents = fio_blkio_getevents, + .event = fio_blkio_event, + + .options = options, + .option_struct_size =sizeof (struct fio_blkio_options ), +}; + +static void fio_init fio_blkio_register(void) +{ +register_ioengine (&ioengine); +} + +static void fio_exit fio_blkio_unregister(void) +{ +unregister_ioengine (&ioengine); +}examples/libblkio-io_uring.fio new file +19
-0@@ -0,0 +1,19 @@ +; Benchmark accessing a regular file or block device using libblkio. +; +; Replace "libblkio_path" below with the path to your file or device, or +; override it by passing the '--libblkio_pre_connect_props=path=...' flag to +; fio. +; +; For information on libblkio, see: https://gitlab.com/libblkio/libblkio + +[global] +ioengine=libblkio +libblkio_driver=io_uring +libblkio_pre_connect_props=path=/dev/nvme0n1 ; REPLACE THIS WITH THE RIGHT PATH +rw=randread +blocksize=4k +direct=1 +time_based=1 +runtime=10s + +[job]examples/libblkio-virtio-blk-vfio-pci.fio new file +18
-0@@ -0,0 +1,18 @@ +; Benchmark accessing a PCI virtio-blk device using libblkio. +; +; Replace "libblkio_path" below with the path to your device's sysfs directory, +; or override it by passing the '--libblkio_pre_connect_props=path=...' flag to +; fio. Note that colons in the path must be escaped with a backslash. +; +; For information on libblkio, see: https://gitlab.com/libblkio/libblkio + +[global] +ioengine=libblkio +libblkio_driver=virtio-blk-vfio-pci +libblkio_pre_connect_props=path=/sys/bus/pci/devices/0000\:00\:01.0 ; REPLACE THIS WITH THE RIGHT PATH +rw=randread +blocksize=4k +time_based=1 +runtime=10s + +[job]fio.1 +19
-0@@ -1989,6 +1989,10 @@ I/O engine using the xNVMe C API, for NVMe devices. The xnvme engine provides flexibility to access GNU/Linux Kernel NVMe driver via libaio, IOCTLs, io_uring, the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes engine specific options. (See \fIhttps://xnvme.io/\fR). +.TP +.B libblkio +Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The +specific \fBdriver\fR to use must be set using \fBlibblkio_driver\fR. .SS "I/O engine specific parameters" In addition, there are some parameters which are only valid when a specific \fBioengine\fR is in use. These are used identically to normal parameters,@@ -2599,6 +2603,21 @@ xnvme namespace identifier for userspace NVMe driver such as SPDK. .TP .BI (xnvme)xnvme_iovec If this option is set, xnvme will use vectored read/write commands. +.TP +.BI (libblkio)libblkio_driver \fR=\fPstr +The driver to be used by libblkio (e.g. \fBvirtio-blk-vfio-pci\fR). +.TP +.BI (libblkio)libblkio_pre_connect_props \fR=\fPstr +A colon-separated list of libblkio properties to be set after creating but +before connecting the \fBstruct blkio\fR. Each property must have the format +\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after +the engine sets any other properties, so those can be overriden. +.TP +.BI (libblkio)libblkio_pre_start_props \fR=\fPstr +A colon-separated list of libblkio properties to be set after connecting but +before starting the \fBstruct blkio\fR. Each property must have the format +\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after +the engine sets any other properties, so those can be overriden. .SS "I/O depth" .TP .BI iodepth \fR=\fPintoptgroup.h +2
-0@@ -73,6 +73,7 @@ enum opt_category_group { __FIO_OPT_G_NFS, __FIO_OPT_G_WINDOWSAIO, __FIO_OPT_G_XNVME, + __FIO_OPT_G_LIBBLKIO,FIO_OPT_G_RATE = (1ULL << __FIO_OPT_G_RATE), FIO_OPT_G_ZONE = (1ULL << __FIO_OPT_G_ZONE),@@ -120,6 +121,7 @@ enum opt_category_group {FIO_OPT_G_DFS = (1ULL << __FIO_OPT_G_DFS),FIO_OPT_G_WINDOWSAIO = (1ULL << __FIO_OPT_G_WINDOWSAIO),FIO_OPT_G_XNVME = (1ULL << __FIO_OPT_G_XNVME), + FIO_OPT_G_LIBBLKIO = (1ULL << __FIO_OPT_G_LIBBLKIO), };extern const struct opt_group *opt_group_from_mask (uint64_t *mask);
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 74 lines · Damien Le Moal
Quoted 51 lines · Alberto Faria
The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like
io_uringand vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.
See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.
[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdfSigned-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 26 ++ Makefile | 6 + configure | 25 ++ engines/libblkio.c | 463 ++++++++++++++++++++++ examples/libblkio-io_uring.fio | 19 + examples/libblkio-virtio-blk-vfio-pci.fio | 18 + fio.1 | 19 + optgroup.h | 2 + 8 files changed, 578 insertions(+) create mode 100644 engines/libblkio.c create mode 100644 examples/libblkio-io_uring.fio create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fioHOWTO.rst +6
-0@@ -2192,6 +2192,12 @@ I/O engine the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes engine specific options. (See https://xnvme.io). + **libblkio** + Use the libblkio library + (https://gitlab.com/libblkio/libblkio). The specific + *driver* to use must be set using + :option:`libblkio_driver`. + I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2853
Quoted 6 lines
HOWTO.rst +3
-0@@ -2842,6 +2848,26 @@ with the caveat that when used on the command line, they must come after the If this option is set. xnvme will use vectored read/write commands. +.. option:: libblkio_driver=str : [libblkio] + + The driver to be used by libblkio (e.g. **virtio-blk-vfio-pci**).It would be nice to list the possible values and their meaning here.
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2861
Quoted 8 lines
HOWTO.rst +8
-0+ +.. option:: libblkio_pre_connect_props=str : [libblkio] + + A colon-separated list of libblkio properties to be set after creating + but before connecting the ``struct blkio``. Each property must have the + format ``<name>=<value>``. Colons can be escaped as ``\:``. These are + set after the engine sets any other properties, so those can be + overriden."struct blkio" has no meaning whatsoever for the fio command line interface. So could this be reworded without using struct names ? E.g. "without connecting the device accessed by libblkio" ?
Quoted 3 lines
And we need to list the possible properties or have at least a link to some documentation listing the possible properties. Otherwise, how can the user find that out ?
In v2 I ended up linking to the libblkio docs. I avoided adding lists of available drivers and properties since those would become incomplete rather quickly and also depend on the installed version of libblkio.
Thanks,
Alberto
PATCH 2/10 Add engine flag FIO_SKIPPABLE_IOMEM_ALLOC
In reply to Alberto FariaIt makes it valid to set option mem/iomem even when the engine specifies iomem_alloc and iomem_free callbacks, allowing users to optionally use fio's customizable memory allocation logic instead of the engine's.
This is in preparation for giving libblkio engine users the choice between controlling memory allocation or delegating it to the libblkio library.
--- ioengines.h | 2 ++ memory.c | 22 ++++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-)
ioengines.h +2 -0
@@ -87,6 +87,8 @@ enum fio_ioengine_flags {FIO_NO_OFFLOAD =1 <<15 ,/* no async offload */ FIO_ASYNCIO_SETS_ISSUE_TIME =1 <<16 ,/* async ioengine with commit function that sets issue_time */ +FIO_SKIPPABLE_IOMEM_ALLOC + = 1 << 17, /* skip iomem_alloc & iomem_free if job sets mem/iomem */ };/*
memory.c +12 -10
@@ -305,16 +305,18 @@ int allocate_io_mem(struct thread_data *td)dprint (FD_MEM ,"Alloc %llu for buffers\n" , (unsigned long long ) total_mem);/* -* If the IO engine has hooks to allocate/free memory, use those. But -* error out if the user explicitly asked for something else. +* If the IO engine has hooks to allocate/free memory and the user +* doesn't explicitly ask for something else, use those. But fail if the +* user asks for something else with an engine that doesn't allow that. */ -if (td->io_ops->iomem_alloc) { -if (fio_option_is_set (&td->o, mem_type)) { -log_err ("fio: option 'mem/iomem' conflicts with specified IO engine\n" ); - ret =1 ; - }else - ret = td->io_ops->iomem_alloc (td, total_mem); - }else if (td->o.mem_type ==MEM_MALLOC ) +if (td->io_ops->iomem_alloc &&fio_option_is_set (&td->o, mem_type) && + !td_ioengine_flagged (td,FIO_SKIPPABLE_IOMEM_ALLOC )) { +log_err ("fio: option 'mem/iomem' conflicts with specified IO engine\n" ); + ret =1 ; + }else if (td->io_ops->iomem_alloc && + !fio_option_is_set (&td->o, mem_type)) + ret = td->io_ops->iomem_alloc (td, total_mem); +else if (td->o.mem_type ==MEM_MALLOC ) ret =alloc_mem_malloc (td, total_mem);else if (td->o.mem_type ==MEM_SHM || td->o.mem_type ==MEM_SHMHUGE ) ret =alloc_mem_shm (td, total_mem);
@@ -342,7 +344,7 @@ void free_io_mem(struct thread_data *td)if (td->o.odirect || td->o.oatomic) total_mem += page_mask; -if (td->io_ops->iomem_alloc) { +if (td->io_ops->iomem_alloc && !fio_option_is_set (&td->o, mem_type)) {if (td->io_ops->iomem_free) td->io_ops->iomem_free (td); }else if (td->o.mem_type ==MEM_MALLOC )
Signature · 1 line
2.38.1
PATCH 3/10 engines/libblkio: Allow setting option mem/iomem
In reply to Alberto FariaThis allows users to customize data buffer memory using fio's existing options. Users become responsible for ensuring that the allocated memory satisfies all constraints imposed by the libblkio driver under use.
--- HOWTO.rst | 5 ++++- engines/libblkio.c | 46 ++++++++++++++++++++++++++++++++++++++++++++-- fio.1 | 4 +++- 3 files changed, 51 insertions(+), 4 deletions(-)
HOWTO.rst +4 -1
@@ -2196,7 +2196,10 @@ I/O engine Use the libblkio library (https://gitlab.com/libblkio/libblkio). The specific *driver* to use must be set using - :option:`libblkio_driver`. + :option:`libblkio_driver`. If + :option:`mem`/:option:`iomem` is not specified, memory + allocation is delegated to libblkio (and so is + guaranteed to work with the selected *driver*). I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
engines/libblkio.c +44 -2
67 lines not on this page · raw message
fio.1 +3 -1
@@ -1992,7 +1992,9 @@ engine specific options. (See \fIhttps://xnvme.io/\fR). .TP .B libblkio Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The -specific \fBdriver\fR to use must be set using \fBlibblkio_driver\fR. +specific \fBdriver\fR to use must be set using \fBlibblkio_driver\fR. If +\fBmem\fR/\fBiomem\fR is not specified, memory allocation is delegated to +libblkio (and so is guaranteed to work with the selected \fBdriver\fR). .SS "I/O engine specific parameters" In addition, there are some parameters which are only valid when a specific \fBioengine\fR is in use. These are used identically to normal parameters,
Signature · 1 line
2.38.1
PATCH 4/10 engines/libblkio: Add support for poll queues
In reply to Alberto FariaConfigure a poll queue instead of a "regular" queue when option hipri is set.
--- HOWTO.rst | 4 ++++ engines/libblkio.c | 26 ++++++++++++++++++++++++-- fio.1 | 3 +++ 3 files changed, 31 insertions(+), 2 deletions(-)
HOWTO.rst +4 -0
@@ -2330,6 +2330,10 @@ with the caveat that when used on the command line, they must come after the by the application. The benefits are more efficient IO for high IOPS scenarios, and lower latencies for low queue depth IO. + [libblkio] + + Use poll queues. + [pvsync2] Set RWF_HIPRI on I/O, indicating to the kernel that it's of higher priority
engines/libblkio.c +24 -2
@@ -26,6 +26,8 @@ struct fio_blkio_options {char *driver;char *pre_connect_props;char *pre_start_props; + +unsigned int hipri; };static struct fio_option options[] = {
@@ -57,6 +59,16 @@ static struct fio_option options[] = { .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "hipri", + .lname = "Use poll queues", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options, hipri), + .help = "Use poll queues", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, + { .name = NULL, },
@@ -244,6 +256,7 @@ struct fio_blkio_data {static int fio_blkio_init (struct thread_data *td) { +const struct fio_blkio_options *options = td->eo;struct fio_blkio_data *data;/* allocate per-thread data struct */
@@ -265,7 +278,13 @@ static int fio_blkio_init(struct thread_data *td)if (fio_blkio_create_and_connect (td, &data->b) !=0 ) goto err_free; -if (blkio_set_int (data->b,"num-queues" ,1 ) !=0 ) { +if (blkio_set_int (data->b,"num-queues" , options->hipri ?0 :1 ) !=0 ) { +fio_blkio_log_err (blkio_set_int); + goto err_blkio_destroy; + } + +if (blkio_set_int (data->b,"num-poll-queues" , + options->hipri ?1 :0 ) !=0 ) {fio_blkio_log_err (blkio_set_int); goto err_blkio_destroy; }
@@ -277,7 +296,10 @@ static int fio_blkio_init(struct thread_data *td)/* get queue */ - data->q =blkio_get_queue (data->b,0 ); +if (options->hipri) + data->q =blkio_get_poll_queue (data->b,0 ); +else + data->q =blkio_get_queue (data->b,0 );/* Set data only here so cleanup() does nothing if init() fails. */ td->io_ops_data = data;
fio.1 +3 -0
@@ -2620,6 +2620,9 @@ A colon-separated list of libblkio properties to be set after connecting but before starting the \fBstruct blkio\fR. Each property must have the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after the engine sets any other properties, so those can be overriden. +.TP +.BI (libblkio)hipri \fR=\fPbool +Use poll queues. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
Re: [PATCH 04/10] engines/libblkio: Add support for poll queues
Quoted 42 lines · Alberto Faria
Configure a poll queue instead of a "regular" queue when option hipri is set.
Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 4 ++++ engines/libblkio.c | 26 ++++++++++++++++++++++++-- fio.1 | 3 +++ 3 files changed, 31 insertions(+), 2 deletions(-)HOWTO.rst +4
-0@@ -2330,6 +2330,10 @@ with the caveat that when used on the command line, they must come after the by the application. The benefits are more efficient IO for high IOPS scenarios, and lower latencies for low queue depth IO. + [libblkio] + + Use poll queues. + [pvsync2] Set RWF_HIPRI on I/O, indicating to the kernel that it's of higher priorityengines/libblkio.c +2
-0@@ -26,6 +26,8 @@ struct fio_blkio_options {char *driver;char *pre_connect_props;char *pre_start_props; + +unsigned int hipri; };static struct fio_option options[] = {@@ -57,6 +59,16 @@ static struct fio_option options[] = { .group = FIO_OPT_G_LIBBLKIO, },
engines/libblkio.c static struct fio_option options[] = { · line 71
Quoted 10 lines
engines/libblkio.c +10
-0+ { + .name = "hipri", + .lname = "Use poll queues", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options, hipri), + .help = "Use poll queues", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, +
Some of these elements have blank lines before or after the { } and others do not. For consistency I would eliminate the blank lines before and after the { and }.
Quoted 43 lines
engines/libblkio.c +12
-2{ .name = NULL, },@@ -244,6 +256,7 @@ struct fio_blkio_data {static int fio_blkio_init (struct thread_data *td) { +const struct fio_blkio_options *options = td->eo;struct fio_blkio_data *data;/* allocate per-thread data struct */ @@ -265,7 +278,13 @@ static int fio_blkio_init(struct thread_data *td)if (fio_blkio_create_and_connect (td, &data->b) !=0 ) goto err_free; -if (blkio_set_int (data->b,"num-queues" ,1 ) !=0 ) { +if (blkio_set_int (data->b,"num-queues" , options->hipri ?0 :1 ) !=0 ) { +fio_blkio_log_err (blkio_set_int); + goto err_blkio_destroy; + } + +if (blkio_set_int (data->b,"num-poll-queues" , + options->hipri ?1 :0 ) !=0 ) {fio_blkio_log_err (blkio_set_int); goto err_blkio_destroy; }@@ -277,7 +296,10 @@ static int fio_blkio_init(struct thread_data *td)/* get queue */ - data->q =blkio_get_queue (data->b,0 ); +if (options->hipri) + data->q =blkio_get_poll_queue (data->b,0 ); +else + data->q =blkio_get_queue (data->b,0 );/* Set data only here so cleanup() does nothing if init() fails. */ td->io_ops_data = data;fio.1 +0
-0
fio.1 A colon-separated list of libblkio properties to be set after connecting but · line 2624
Quoted 5 lines
fio.1 +2
-0@@ -2620,6 +2620,9 @@ A colon-separated list of libblkio properties to be set after connecting but before starting the \fBstruct blkio\fR. Each property must have the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after the engine sets any other properties, so those can be overriden. +.TP +.BI (libblkio)hipri \fR=\fPbool
The type for hipri is FIO_OPT_STR_SET which is different from a boolean. If it were a boolean fio would accept only hipri=0 or hipri=1. For FIO_OPT_STR_SET fio accepts hipri=<any number>, or just hipri with no value (same meaning as hipri=1), with 0 representing false.
For FIO_OPT_STR_SET options (e.g., thread, time_based) the convention in the man page is to not specify a type.
PATCH 5/10 engines/libblkio: Add option libblkio_vectored
In reply to Alberto FariaWhen enabled, read and write requests are submitted as vectored requests using blkioq_{readv,writev}(), instead of using blkioq_{read,write}().
--- HOWTO.rst | 4 ++++ engines/libblkio.c | 46 +++++++++++++++++++++++++++++++++++++++++----- fio.1 | 3 +++ 3 files changed, 48 insertions(+), 5 deletions(-)
HOWTO.rst +4 -0
@@ -2875,6 +2875,10 @@ with the caveat that when used on the command line, they must come after the set after the engine sets any other properties, so those can be overriden. +.. option:: libblkio_vectored=bool : [libblkio] + + Submit vectored read and write requests. Default is 0. + I/O depth ~~~~~~~~~
engines/libblkio.c +41 -5
98 lines not on this page · raw message
fio.1 +3 -0
@@ -2623,6 +2623,9 @@ the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool Use poll queues. +.TP +.BI (libblkio)libblkio_vectored \fR=\fPbool +Submit vectored read and write requests. Default is 0. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
Re: [PATCH 05/10] engines/libblkio: Add option libblkio_vectored
Quoted 15 lines · Alberto Faria
When enabled, read and write requests are submitted as vectored requests using blkioq_{readv,writev}(), instead of using blkioq_{read,write}().
Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 4 ++++ engines/libblkio.c | 46 +++++++++++++++++++++++++++++++++++++++++----- fio.1 | 3 +++ 3 files changed, 48 insertions(+), 5 deletions(-)HOWTO.rst +0
-0
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2878
Quoted 4 lines
HOWTO.rst +1
-0@@ -2875,6 +2875,10 @@ with the caveat that when used on the command line, they must come after the set after the engine sets any other properties, so those can be overriden. +.. option:: libblkio_vectored=bool : [libblkio]
See my response to the hipri patch regarding the difference between booleans and FIO_OPT_STR_SET. I don't have strong feelings regarding whether this should be a boolean or FIO_OPT_STR_SET but if the type listed in the HOWTO and manpage is boolean then the type in the options struct should be FIO_OPT_BOOL. No type should be listed when it's FIO_OPT_STR_SET.
Quoted 129 lines
HOWTO.rst +3
-0+ + Submit vectored read and write requests. Default is 0. + I/O depth ~~~~~~~~~engines/libblkio.c +41
-5@@ -28,6 +28,7 @@ struct fio_blkio_options {char *pre_start_props;unsigned int hipri; +unsigned int vectored; };static struct fio_option options[] = {@@ -68,6 +69,15 @@ static struct fio_option options[] = { .category = FIO_OPT_C_ENGINE, .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "libblkio_vectored", + .lname = "Use blkioq_{readv,writev}()", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options, vectored), + .help = "Use blkioq_{readv,writev}() instead of blkioq_{read,write}()", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, { .name = NULL,@@ -251,6 +261,7 @@ struct fio_blkio_data {bool has_mem_region;/* whether mem_region is valid */ struct blkio_mem_region mem_region;/* only if allocated by libblkio */ +struct iovec *iovecs;/* for vectored requests */ struct blkio_completion *completions; };@@ -267,8 +278,9 @@ static int fio_blkio_init(struct thread_data *td) goto err_free; } + data->iovecs =calloc (td->o.iodepth,sizeof (data->iovecs[0 ])); data->completions =calloc (td->o.iodepth,sizeof (data->completions[0 ])); -if (!data->completions) { +if (!data->iovecs || !data->completions) {log_err ("fio: calloc() failed\n" ); goto err_free; }@@ -310,6 +322,7 @@ err_blkio_destroy:blkio_destroy (&data->b); err_free:free (data->completions); +free (data->iovecs);free (data);return 1 ; }@@ -362,6 +375,7 @@ static void fio_blkio_cleanup(struct thread_data *td)if (data) {blkio_destroy (&data->b);free (data->completions); +free (data->iovecs);free (data); } }@@ -432,19 +446,41 @@ static int fio_blkio_open_file(struct thread_data *td, struct fio_file *f)static enum fio_q_status fio_blkio_queue (struct thread_data *td,struct io_u *io_u) { +const struct fio_blkio_options *options = td->eo;struct fio_blkio_data *data = td->io_ops_data;fio_ro_check (td, io_u);switch (io_u->ddir) {case DDIR_READ : -blkioq_read (data->q, io_u->offset, io_u->xfer_buf, - (size_t )io_u->xfer_buflen, io_u,0 ); +if (options->vectored) { +struct iovec *iov = &data->iovecs[io_u->index]; + iov->iov_base = io_u->xfer_buf; + iov->iov_len = (size_t )io_u->xfer_buflen; + +blkioq_readv (data->q, io_u->offset, iov,1 , + io_u,0 ); + }else { +blkioq_read (data->q, io_u->offset, + io_u->xfer_buf, + (size_t )io_u->xfer_buflen, io_u,0 ); + }break ;case DDIR_WRITE : -blkioq_write (data->q, io_u->offset, io_u->xfer_buf, - (size_t )io_u->xfer_buflen, io_u,0 ); +if (options->vectored) { +struct iovec *iov = &data->iovecs[io_u->index]; + iov->iov_base = io_u->xfer_buf; + iov->iov_len = (size_t )io_u->xfer_buflen; + +blkioq_writev (data->q, io_u->offset, iov,1 , + io_u,0 ); + }else { +blkioq_write (data->q, io_u->offset, + io_u->xfer_buf, + (size_t )io_u->xfer_buflen, io_u, +0 ); + }break ;case DDIR_TRIM :fio.1 +3
-0@@ -2623,6 +2623,9 @@ the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool Use poll queues. +.TP +.BI (libblkio)libblkio_vectored \fR=\fPbool +Submit vectored read and write requests. Default is 0. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
Re: [PATCH 05/10] engines/libblkio: Add option libblkio_vectored
Quoted 19 lines · Vincent Fu
Quoted 18 lines · Alberto Faria
When enabled, read and write requests are submitted as vectored requests using blkioq_{readv,writev}(), instead of using blkioq_{read,write}().
Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 4 ++++ engines/libblkio.c | 46 +++++++++++++++++++++++++++++++++++++++++----- fio.1 | 3 +++ 3 files changed, 48 insertions(+), 5 deletions(-)HOWTO.rst +0
-0@@ -2875,6 +2875,10 @@ with the caveat that when used on the command line, they must come after the set after the engine sets any other properties, so those can be overriden.
Quoted 8 lines
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2878
Quoted 1 line
HOWTO.rst +1
-0+.. option:: libblkio_vectored=bool : [libblkio]See my response to the hipri patch regarding the difference between booleans and
FIO_OPT_STR_SET. I don't have strong feelings regarding whether this should be a boolean orFIO_OPT_STR_SETbut if the type listed in the HOWTO and manpage is boolean then the type in the options struct should beFIO_OPT_BOOL. No type should be listed when it'sFIO_OPT_STR_SET.
Thanks, I went with keeping FIO_OPT_STR_SET and not listing the type in the docs in v2.
Alberto
PATCH 6/10 engines/libblkio: Add option libblkio_write_zeroes_on_trim
In reply to Alberto FariaWhen set, trim IOs will be submitted as blkioq_write_zeroes() requests instead of blkioq_discard() requests.
--- HOWTO.rst | 5 +++++ engines/libblkio.c | 20 ++++++++++++++++++-- fio.1 | 4 ++++ 3 files changed, 27 insertions(+), 2 deletions(-)
HOWTO.rst +5 -0
@@ -2879,6 +2879,11 @@ with the caveat that when used on the command line, they must come after the Submit vectored read and write requests. Default is 0. +.. option:: libblkio_write_zeroes_on_trim=bool : [libblkio] + + Submit trims as "write zeroes" requests instead of discard requests. + Default is 0. + I/O depth ~~~~~~~~~
engines/libblkio.c +18 -2
@@ -29,6 +29,7 @@ struct fio_blkio_options {unsigned int hipri;unsigned int vectored; +unsigned int write_zeroes_on_trim; };static struct fio_option options[] = {
@@ -78,6 +79,16 @@ static struct fio_option options[] = { .category = FIO_OPT_C_ENGINE, .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "libblkio_write_zeroes_on_trim", + .lname = "Use blkioq_write_zeroes() for TRIM", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options , + write_zeroes_on_trim), + .help = "Use blkioq_write_zeroes() for TRIM instead of blkioq_discard()", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, { .name = NULL,
@@ -484,8 +495,13 @@ static enum fio_q_status fio_blkio_queue(struct thread_data *td,break ;case DDIR_TRIM : -blkioq_discard (data->q, io_u->offset, io_u->xfer_buflen, - io_u,0 ); +if (options->write_zeroes_on_trim) { +blkioq_write_zeroes (data->q, io_u->offset, + io_u->xfer_buflen, io_u,0 ); + }else { +blkioq_discard (data->q, io_u->offset, + io_u->xfer_buflen, io_u,0 ); + }break ;case DDIR_SYNC :
fio.1 +4 -0
@@ -2626,6 +2626,10 @@ Use poll queues. .TP .BI (libblkio)libblkio_vectored \fR=\fPbool Submit vectored read and write requests. Default is 0. +.TP +.BI (libblkio)libblkio_write_zeroes_on_trim \fR=\fPbool +Submit trims as "write zeroes" requests instead of discard requests. Default is +0. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
Re: [PATCH 06/10] engines/libblkio: Add option libblkio_write_zeroes_on_trim
Quoted 15 lines · Alberto Faria
When set, trim IOs will be submitted as
blkioq_write_zeroes()requests instead ofblkioq_discard()requests.Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 5 +++++ engines/libblkio.c | 20 ++++++++++++++++++-- fio.1 | 4 ++++ 3 files changed, 27 insertions(+), 2 deletions(-)HOWTO.rst +0
-0
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2882
Quoted 4 lines
HOWTO.rst +1
-0@@ -2879,6 +2879,11 @@ with the caveat that when used on the command line, they must come after the Submit vectored read and write requests. Default is 0. +.. option:: libblkio_write_zeroes_on_trim=bool : [libblkio]
Same comment here as for libblkio_vectored regarding option types.
Quoted 67 lines
HOWTO.rst +4
-0+ + Submit trims as "write zeroes" requests instead of discard requests. + Default is 0. + I/O depth ~~~~~~~~~engines/libblkio.c +18
-2@@ -29,6 +29,7 @@ struct fio_blkio_options {unsigned int hipri;unsigned int vectored; +unsigned int write_zeroes_on_trim; };static struct fio_option options[] = {@@ -78,6 +79,16 @@ static struct fio_option options[] = { .category = FIO_OPT_C_ENGINE, .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "libblkio_write_zeroes_on_trim", + .lname = "Use blkioq_write_zeroes() for TRIM", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options , + write_zeroes_on_trim), + .help = "Use blkioq_write_zeroes() for TRIM instead of blkioq_discard()", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, { .name = NULL,@@ -484,8 +495,13 @@ static enum fio_q_status fio_blkio_queue(struct thread_data *td,break ;case DDIR_TRIM : -blkioq_discard (data->q, io_u->offset, io_u->xfer_buflen, - io_u,0 ); +if (options->write_zeroes_on_trim) { +blkioq_write_zeroes (data->q, io_u->offset, + io_u->xfer_buflen, io_u,0 ); + }else { +blkioq_discard (data->q, io_u->offset, + io_u->xfer_buflen, io_u,0 ); + }break ;case DDIR_SYNC :fio.1 +4
-0@@ -2626,6 +2626,10 @@ Use poll queues. .TP .BI (libblkio)libblkio_vectored \fR=\fPbool Submit vectored read and write requests. Default is 0. +.TP +.BI (libblkio)libblkio_write_zeroes_on_trim \fR=\fPbool +Submit trims as "write zeroes" requests instead of discard requests. Default is +0. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
PATCH 7/10 engines/libblkio: Add option libblkio_wait_mode
In reply to Alberto FariaIt allows configuring how the engine waits for request completions, instead of always using a blocking blkioq_do_io() call.
--- HOWTO.rst | 14 ++++- engines/libblkio.c | 132 ++++++++++++++++++++++++++++++++++++++++++--- fio.1 | 16 +++++- 3 files changed, 154 insertions(+), 8 deletions(-)
HOWTO.rst +13 -1
@@ -2332,7 +2332,8 @@ with the caveat that when used on the command line, they must come after the [libblkio] - Use poll queues. + Use poll queues. This is incompatible with + :option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`. [pvsync2]
@@ -2884,6 +2885,17 @@ with the caveat that when used on the command line, they must come after the Submit trims as "write zeroes" requests instead of discard requests. Default is 0. +.. option:: libblkio_wait_mode=str : [libblkio] + + How to wait for completions: + + **block** (default) + Use a blocking call to ``blkioq_do_io()``. + **eventfd** + Use a blocking call to ``read()`` on the completion eventfd. + **loop** + Use a busy loop with a non-blocking call to ``blkioq_do_io()``. + I/O depth ~~~~~~~~~
engines/libblkio.c +126 -6
190 lines not on this page · raw message
fio.1 +15 -1
@@ -2622,7 +2622,7 @@ before starting the \fBstruct blkio\fR. Each property must have the format the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool -Use poll queues. +Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR. .TP .BI (libblkio)libblkio_vectored \fR=\fPbool Submit vectored read and write requests. Default is 0.
@@ -2630,6 +2630,20 @@ Submit vectored read and write requests. Default is 0. .BI (libblkio)libblkio_write_zeroes_on_trim \fR=\fPbool Submit trims as "write zeroes" requests instead of discard requests. Default is 0. +.TP +.BI (libblkio)libblkio_wait_mode \fR=\fPstr +How to wait for completions: +.RS +.RS +.TP +.B block \fR(default) +Use a blocking call to \fBblkioq_do_io()\fR. +.TP +.B eventfd +Use a blocking call to \fBread()\fR on the completion eventfd. +.TP +.B loop +Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
Re: [PATCH 07/10] engines/libblkio: Add option libblkio_wait_mode
Quoted 174 lines · Alberto Faria
It allows configuring how the engine waits for request completions, instead of always using a blocking
blkioq_do_io()call.Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 14 ++++- engines/libblkio.c | 132 ++++++++++++++++++++++++++++++++++++++++++--- fio.1 | 16 +++++- 3 files changed, 154 insertions(+), 8 deletions(-)HOWTO.rst +13
-1@@ -2332,7 +2332,8 @@ with the caveat that when used on the command line, they must come after the [libblkio] - Use poll queues. + Use poll queues. This is incompatible with + :option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`. [pvsync2]@@ -2884,6 +2885,17 @@ with the caveat that when used on the command line, they must come after the Submit trims as "write zeroes" requests instead of discard requests. Default is 0. +.. option:: libblkio_wait_mode=str : [libblkio] + + How to wait for completions: + + **block** (default) + Use a blocking call to ``blkioq_do_io()``. + **eventfd** + Use a blocking call to ``read()`` on the completion eventfd. + **loop** + Use a busy loop with a non-blocking call to ``blkioq_do_io()``. + I/O depth ~~~~~~~~~engines/libblkio.c +68
-0@@ -20,6 +20,12 @@#include "../options.h" #include "../parse.h" +enum fio_blkio_wait_mode { +FIO_BLKIO_WAIT_MODE_BLOCK , +FIO_BLKIO_WAIT_MODE_EVENTFD , +FIO_BLKIO_WAIT_MODE_LOOP , +}; +struct fio_blkio_options {void *pad;/* option fields must not have offset 0 */ @@ -30,6 +36,7 @@ struct fio_blkio_options {unsigned int hipri;unsigned int vectored;unsigned int write_zeroes_on_trim; +enum fio_blkio_wait_mode wait_mode; };static struct fio_option options[] = {@@ -89,6 +96,30 @@ static struct fio_option options[] = { .category = FIO_OPT_C_ENGINE, .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "libblkio_wait_mode", + .lname = "How to wait for completions", + .type = FIO_OPT_STR, + .off1 = offsetof(struct fio_blkio_options, wait_mode), + .help = "How to wait for completions", + .def = "block", + .posval = { + { .ival = "block", + .oval = FIO_BLKIO_WAIT_MODE_BLOCK, + .help = "Blocking blkioq_do_io()", + }, + { .ival = "eventfd", + .oval = FIO_BLKIO_WAIT_MODE_EVENTFD, + .help = "Blocking read() on the completion eventfd", + }, + { .ival = "loop", + .oval = FIO_BLKIO_WAIT_MODE_LOOP, + .help = "Busy loop with non-blocking blkioq_do_io()", + }, + }, + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, { .name = NULL,@@ -237,12 +268,21 @@ static int fio_blkio_setup(struct thread_data *td)* again in the init() callback. */ +const struct fio_blkio_options *options = td->eo;struct blkio *b;int ret =0 ;uint64_t capacity;assert (td->files_index ==1 ); +/* validate options */ + +if (options->hipri && + options->wait_mode ==FIO_BLKIO_WAIT_MODE_EVENTFD ) { +log_err ("fio: option hipri is incompatible with option libblkio_wait_mode=eventfd\n" ); +return 1 ; + } +/* get target size */ if (fio_blkio_create_and_connect (td, &b) !=0 )@@ -268,6 +308,7 @@ out_blkio_destroy:struct fio_blkio_data {struct blkio *b;struct blkioq *q; +int completion_fd;/* -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */ bool has_mem_region;/* whether mem_region is valid */ struct blkio_mem_region mem_region;/* only if allocated by libblkio */ @@ -280,6 +321,7 @@ static int fio_blkio_init(struct thread_data *td) {const struct fio_blkio_options *options = td->eo;struct fio_blkio_data *data; +int flags;/* allocate per-thread data struct */ @@ -324,6 +366,31 @@ static int fio_blkio_init(struct thread_data *td)else data->q =blkio_get_queue (data->b,0 ); +/* enable completion fd and make it blocking */ + +if (options->wait_mode ==FIO_BLKIO_WAIT_MODE_EVENTFD ) { +blkioq_set_completion_fd_enabled (data->q, true); + + data->completion_fd =blkioq_get_completion_fd (data->q); + + flags =fcntl (data->completion_fd,F_GETFL ); +if (flags <0 ) { +log_err ("fio: fcntl(F_GETFL) failed: %s\n" , +strerror (errno)); + goto err_blkio_destroy; + } + + flags &= ~O_NONBLOCK ; + +if (fcntl (data->completion_fd,F_SETFL , flags) !=0 ) { +log_err ("fio: fcntl(F_SETFL) failed: %s\n" , +strerror (errno)); + goto err_blkio_destroy; + } + }else { + data->completion_fd =-1 ; + } +/* Set data only here so cleanup() does nothing if init() fails. */ td->io_ops_data = data;@@ -521,16 +588,69 @@ static enum fio_q_status fio_blkio_queue(struct thread_data *td,static int fio_blkio_getevents (struct thread_data *td,unsigned int min,unsigned int max,const struct timespec *t) { +const struct fio_blkio_options *options = td->eo;struct fio_blkio_data *data = td->io_ops_data;
engines/libblkio.c static enum fio_q_status fio_blkio_queue(struct thread_data *td, · line 598
Quoted 10 lines
engines/libblkio.c +5
-4-int n; +int ret, n; +uint64_t event; - n =blkioq_do_io (data->q, data->completions, (int )min, (int )max,NULL ); -if (n <0 ) { -fio_blkio_log_err (blkioq_do_io); +switch (options->wait_mode) { +case FIO_BLKIO_WAIT_MODE_BLOCK : +
Because we already have indentation here the blank lines before and after each "case" don't improve readability much. To be consistent with the rest of the code base and conserve vertical screen real estate I would drop the blank lines.
Quoted 94 lines
engines/libblkio.c +53
-2+ n =blkioq_do_io (data->q, data->completions, (int )min, (int )max, +NULL ); +if (n <0 ) { +fio_blkio_log_err (blkioq_do_io); +return -1 ; + } + +return n; + +case FIO_BLKIO_WAIT_MODE_EVENTFD : + + n =blkioq_do_io (data->q, data->completions,0 , (int )max,NULL ); +if (n <0 ) { +fio_blkio_log_err (blkioq_do_io); +return -1 ; + } + +while (n < (int )min) { + ret =read (data->completion_fd, &event,sizeof (event)); +if (ret !=sizeof (event)) { +log_err ("fio: read() on the completion fd returned %d\n" , + ret); +return -1 ; + } + + ret =blkioq_do_io (data->q, data->completions + n,0 , + (int )max - n,NULL ); +if (ret <0 ) { +fio_blkio_log_err (blkioq_do_io); +return -1 ; + } + + n += ret; + } + +return n; + +case FIO_BLKIO_WAIT_MODE_LOOP : + +for (n =0 ; n < (int )min; ) { + ret =blkioq_do_io (data->q, data->completions + n,0 , + (int )max - n,NULL ); +if (ret <0 ) { +fio_blkio_log_err (blkioq_do_io); +return -1 ; + } + + n += ret; + } + +return n; + +default :return -1 ; } - -return n; }static struct io_u *fio_blkio_event (struct thread_data *td,int event)fio.1 +15
-1@@ -2622,7 +2622,7 @@ before starting the \fBstruct blkio\fR. Each property must have the format the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool -Use poll queues. +Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR. .TP .BI (libblkio)libblkio_vectored \fR=\fPbool Submit vectored read and write requests. Default is 0.@@ -2630,6 +2630,20 @@ Submit vectored read and write requests. Default is 0. .BI (libblkio)libblkio_write_zeroes_on_trim \fR=\fPbool Submit trims as "write zeroes" requests instead of discard requests. Default is 0. +.TP +.BI (libblkio)libblkio_wait_mode \fR=\fPstr +How to wait for completions: +.RS +.RS +.TP +.B block \fR(default) +Use a blocking call to \fBblkioq_do_io()\fR. +.TP +.B eventfd +Use a blocking call to \fBread()\fR on the completion eventfd. +.TP +.B loop +Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
PATCH 8/10 engines/libblkio: Add option libblkio_force_enable_completion_eventfd
In reply to Alberto FariaWhen set, the queue's completion fd is enabled even when it isn't used, i.e., even if option libblkio_wait_mode is _not_ set to "eventfd".
Depending on the libblkio driver, this can have an impact on performance. This option allows evaluating that overhead.
--- HOWTO.rst | 9 ++++++++- engines/libblkio.c | 21 +++++++++++++++++++-- fio.1 | 10 +++++++++- 3 files changed, 36 insertions(+), 4 deletions(-)
HOWTO.rst +8 -1
@@ -2333,7 +2333,8 @@ with the caveat that when used on the command line, they must come after the [libblkio] Use poll queues. This is incompatible with - :option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`. + :option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>` and + :option:`libblkio_force_enable_completion_eventfd`. [pvsync2]
@@ -2896,6 +2897,12 @@ with the caveat that when used on the command line, they must come after the **loop** Use a busy loop with a non-blocking call to ``blkioq_do_io()``. +.. option:: libblkio_force_enable_completion_eventfd=bool : [libblkio] + + Enable the ``struct blkioq``'s completion eventfd even when unused. This + may impact performance. The default is to enable it only if + :option:`libblkio_wait_mode` is **eventfd**. + I/O depth ~~~~~~~~~
engines/libblkio.c +19 -2
@@ -37,6 +37,7 @@ struct fio_blkio_options {unsigned int vectored;unsigned int write_zeroes_on_trim;enum fio_blkio_wait_mode wait_mode; +unsigned int force_enable_completion_eventfd; };static struct fio_option options[] = {
@@ -120,6 +121,16 @@ static struct fio_option options[] = { .category = FIO_OPT_C_ENGINE, .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "libblkio_force_enable_completion_eventfd", + .lname = "Force enable the completion eventfd, even if unused", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options , + force_enable_completion_eventfd), + .help = "This can impact performance", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, { .name = NULL,
@@ -283,6 +294,11 @@ static int fio_blkio_setup(struct thread_data *td)return 1 ; } +if (options->hipri && options->force_enable_completion_eventfd) { +log_err ("fio: option hipri is incompatible with option libblkio_force_enable_completion_eventfd\n" ); +return 1 ; + } +/* get target size */ if (fio_blkio_create_and_connect (td, &b) !=0 )
@@ -308,7 +324,7 @@ out_blkio_destroy:struct fio_blkio_data {struct blkio *b;struct blkioq *q; -int completion_fd;/* -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */ +int completion_fd;/* may be -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */ bool has_mem_region;/* whether mem_region is valid */ struct blkio_mem_region mem_region;/* only if allocated by libblkio */
@@ -368,7 +384,8 @@ static int fio_blkio_init(struct thread_data *td)/* enable completion fd and make it blocking */ -if (options->wait_mode ==FIO_BLKIO_WAIT_MODE_EVENTFD ) { +if (options->wait_mode ==FIO_BLKIO_WAIT_MODE_EVENTFD || + options->force_enable_completion_eventfd) {blkioq_set_completion_fd_enabled (data->q, true); data->completion_fd =blkioq_get_completion_fd (data->q);
fio.1 +9 -1
@@ -2622,7 +2622,8 @@ before starting the \fBstruct blkio\fR. Each property must have the format the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool -Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR. +Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR and +\fBlibblkio_force_enable_completion_eventfd\fR. .TP .BI (libblkio)libblkio_vectored \fR=\fPbool Submit vectored read and write requests. Default is 0.
@@ -2644,6 +2645,13 @@ Use a blocking call to \fBread()\fR on the completion eventfd. .TP .B loop Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR. +.RE +.RE +.TP +.BI (libblkio)libblkio_force_enable_completion_eventfd \fR=\fPbool +Enable the \fBstruct blkioq\fR's completion eventfd even when unused. This may +impact performance. The default is to enable it only if \fBlibblkio_wait_mode\fR +is \fBeventfd\fR. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
Re: [PATCH 08/10] engines/libblkio: Add option libblkio_force_enable_completion_eventfd
Quoted 28 lines · Alberto Faria
When set, the queue's completion fd is enabled even when it isn't used, i.e., even if option
libblkio_wait_modeis _not_ set to "eventfd".Depending on the libblkio driver, this can have an impact on performance. This option allows evaluating that overhead.
Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 9 ++++++++- engines/libblkio.c | 21 +++++++++++++++++++-- fio.1 | 10 +++++++++- 3 files changed, 36 insertions(+), 4 deletions(-)HOWTO.rst +2
-1@@ -2333,7 +2333,8 @@ with the caveat that when used on the command line, they must come after the [libblkio] Use poll queues. This is incompatible with - :option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`. + :option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>` and + :option:`libblkio_force_enable_completion_eventfd`. [pvsync2]
HOWTO.rst with the caveat that when used on the command line, they must come after the · line 2900
Quoted 4 lines
HOWTO.rst +1
-0@@ -2896,6 +2897,12 @@ with the caveat that when used on the command line, they must come after the **loop** Use a busy loop with a non-blocking call to ``blkioq_do_io()``. +.. option:: libblkio_force_enable_completion_eventfd=bool : [libblkio]
Same comment as for previous patches regarding option types.
Quoted 96 lines
HOWTO.rst +5
-0+ + Enable the ``struct blkioq``'s completion eventfd even when unused. This + may impact performance. The default is to enable it only if + :option:`libblkio_wait_mode` is **eventfd**. + I/O depth ~~~~~~~~~engines/libblkio.c +19
-2@@ -37,6 +37,7 @@ struct fio_blkio_options {unsigned int vectored;unsigned int write_zeroes_on_trim;enum fio_blkio_wait_mode wait_mode; +unsigned int force_enable_completion_eventfd; };static struct fio_option options[] = {@@ -120,6 +121,16 @@ static struct fio_option options[] = { .category = FIO_OPT_C_ENGINE, .group = FIO_OPT_G_LIBBLKIO, }, + { + .name = "libblkio_force_enable_completion_eventfd", + .lname = "Force enable the completion eventfd, even if unused", + .type = FIO_OPT_STR_SET, + .off1 = offsetof(struct fio_blkio_options , + force_enable_completion_eventfd), + .help = "This can impact performance", + .category = FIO_OPT_C_ENGINE, + .group = FIO_OPT_G_LIBBLKIO, + }, { .name = NULL,@@ -283,6 +294,11 @@ static int fio_blkio_setup(struct thread_data *td)return 1 ; } +if (options->hipri && options->force_enable_completion_eventfd) { +log_err ("fio: option hipri is incompatible with option libblkio_force_enable_completion_eventfd\n" ); +return 1 ; + } +/* get target size */ if (fio_blkio_create_and_connect (td, &b) !=0 )@@ -308,7 +324,7 @@ out_blkio_destroy:struct fio_blkio_data {struct blkio *b;struct blkioq *q; -int completion_fd;/* -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */ +int completion_fd;/* may be -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */ bool has_mem_region;/* whether mem_region is valid */ struct blkio_mem_region mem_region;/* only if allocated by libblkio */ @@ -368,7 +384,8 @@ static int fio_blkio_init(struct thread_data *td)/* enable completion fd and make it blocking */ -if (options->wait_mode ==FIO_BLKIO_WAIT_MODE_EVENTFD ) { +if (options->wait_mode ==FIO_BLKIO_WAIT_MODE_EVENTFD || + options->force_enable_completion_eventfd) {blkioq_set_completion_fd_enabled (data->q, true); data->completion_fd =blkioq_get_completion_fd (data->q);fio.1 +9
-1@@ -2622,7 +2622,8 @@ before starting the \fBstruct blkio\fR. Each property must have the format the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool -Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR. +Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR and +\fBlibblkio_force_enable_completion_eventfd\fR. .TP .BI (libblkio)libblkio_vectored \fR=\fPbool Submit vectored read and write requests. Default is 0.@@ -2644,6 +2645,13 @@ Use a blocking call to \fBread()\fR on the completion eventfd. .TP .B loop Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR. +.RE +.RE +.TP +.BI (libblkio)libblkio_force_enable_completion_eventfd \fR=\fPbool +Enable the \fBstruct blkioq\fR's completion eventfd even when unused. This may +impact performance. The default is to enable it only if \fBlibblkio_wait_mode\fR +is \fBeventfd\fR. .SS "I/O depth" .TP .BI iodepth \fR=\fPint
PATCH 9/10 engines/libblkio: Add options for some driver-specific properties
In reply to Alberto FariaThe properties are either common to several drivers or particularly relevant for benchmarking, so this should help write cleaner workload files.
--- HOWTO.rst | 39 ++++++++++---- engines/libblkio.c | 64 ++++++++++++++++++++++- examples/libblkio-io_uring.fio | 5 +- examples/libblkio-virtio-blk-vfio-pci.fio | 5 +- fio.1 | 32 +++++++++--- 5 files changed, 119 insertions(+), 26 deletions(-)
HOWTO.rst +29 -10
@@ -2861,21 +2861,40 @@ with the caveat that when used on the command line, they must come after the The driver to be used by libblkio (e.g. **virtio-blk-vfio-pci**). +.. option:: libblkio_path=str : [libblkio] + + Sets the value of the driver-specific "path" property prior to calling + ``blkio_connect()``, which identifies the target device or file on which + to perform I/O. Its exact semantics are driver-dependent and not all + drivers may support it. + .. option:: libblkio_pre_connect_props=str : [libblkio] - A colon-separated list of libblkio properties to be set after creating - but before connecting the ``struct blkio``. Each property must have the - format ``<name>=<value>``. Colons can be escaped as ``\:``. These are - set after the engine sets any other properties, so those can be - overriden. + A colon-separated list of additional libblkio properties to be set after + creating but before connecting the ``struct blkio``. Each property must + have the format ``<name>=<value>``. Colons can be escaped as ``\:``. + These are set after the engine sets any other properties, so those can + be overriden. + +.. option:: libblkio_num_entries=int : [libblkio] + + Sets the value of the driver-specific "num-entries" property prior to + calling ``blkio_start()``. Its exact semantics are driver-dependent and + not all drivers may support it. + +.. option:: libblkio_queue_size=int : [libblkio] + + Sets the value of the driver-specific "queue-size" property prior to + calling ``blkio_start()``. Its exact semantics are driver-dependent and + not all drivers may support it. .. option:: libblkio_pre_start_props=str : [libblkio] - A colon-separated list of libblkio properties to be set after connecting - but before starting the ``struct blkio``. Each property must have the - format ``<name>=<value>``. Colons can be escaped as ``\:``. These are - set after the engine sets any other properties, so those can be - overriden. + A colon-separated list of additional libblkio properties to be set after + connecting but before starting the ``struct blkio``. Each property must + have the format ``<name>=<value>``. Colons can be escaped as ``\:``. + These are set after the engine sets any other properties, so those can + be overriden. .. option:: libblkio_vectored=bool : [libblkio]
engines/libblkio.c +62 -2
99 lines not on this page · raw message
examples/libblkio-io_uring.fio +2 -3
@@ -1,15 +1,14 @@ ; Benchmark accessing a regular file or block device using libblkio. ; ; Replace "libblkio_path" below with the path to your file or device, or -; override it by passing the '--libblkio_pre_connect_props=path=...' flag to -; fio. +; override it by passing the '--libblkio_path=...' flag to fio. ; ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio [global] ioengine=libblkio libblkio_driver=io_uring -libblkio_pre_connect_props=path=/dev/nvme0n1 ; REPLACE THIS WITH THE RIGHT PATH +libblkio_path=/dev/nvme0n1 ; REPLACE THIS WITH THE RIGHT PATH rw=randread blocksize=4k direct=1
examples/libblkio-virtio-blk-vfio-pci.fio +2 -3
@@ -1,15 +1,14 @@ ; Benchmark accessing a PCI virtio-blk device using libblkio. ; ; Replace "libblkio_path" below with the path to your device's sysfs directory, -; or override it by passing the '--libblkio_pre_connect_props=path=...' flag to -; fio. Note that colons in the path must be escaped with a backslash. +; or override it by passing the '--libblkio_path=...' flag to fio. ; ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio [global] ioengine=libblkio libblkio_driver=virtio-blk-vfio-pci -libblkio_pre_connect_props=path=/sys/bus/pci/devices/0000\:00\:01.0 ; REPLACE THIS WITH THE RIGHT PATH +libblkio_path=/sys/bus/pci/devices/0000:00:01.0 ; REPLACE THIS WITH THE RIGHT PATH rw=randread blocksize=4k time_based=1
fio.1 +24 -8
@@ -2609,17 +2609,33 @@ If this option is set, xnvme will use vectored read/write commands. .BI (libblkio)libblkio_driver \fR=\fPstr The driver to be used by libblkio (e.g. \fBvirtio-blk-vfio-pci\fR). .TP +.BI (libblkio)libblkio_path \fR=\fPstr +Sets the value of the driver-specific "path" property prior to calling +\fBblkio_connect()\fR, which identifies the target device or file on which to +perform I/O. Its exact semantics are driver-dependent and not all drivers may +support it. +.TP .BI (libblkio)libblkio_pre_connect_props \fR=\fPstr -A colon-separated list of libblkio properties to be set after creating but -before connecting the \fBstruct blkio\fR. Each property must have the format -\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after -the engine sets any other properties, so those can be overriden. +A colon-separated list of additional libblkio properties to be set after +creating but before connecting the \fBstruct blkio\fR. Each property must have +the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are +set after the engine sets any other properties, so those can be overriden. +.TP +.BI (libblkio)libblkio_num_entries \fR=\fPint +Sets the value of the driver-specific "num-entries" property prior to calling +\fBblkio_start()\fR. Its exact semantics are driver-dependent and not all +drivers may support it. +.TP +.BI (libblkio)libblkio_queue_size \fR=\fPint +Sets the value of the driver-specific "queue-size" property prior to calling +\fBblkio_start()\fR. Its exact semantics are driver-dependent and not all +drivers may support it. .TP .BI (libblkio)libblkio_pre_start_props \fR=\fPstr -A colon-separated list of libblkio properties to be set after connecting but -before starting the \fBstruct blkio\fR. Each property must have the format -\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after -the engine sets any other properties, so those can be overriden. +A colon-separated list of additional libblkio properties to be set after +connecting but before starting the \fBstruct blkio\fR. Each property must have +the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are +set after the engine sets any other properties, so those can be overriden. .TP .BI (libblkio)hipri \fR=\fPbool Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR and
Signature · 1 line
2.38.1
PATCH 10/10 engines/libblkio: Share a single blkio instance among threads in same process
In reply to Alberto Fariafio groups all subjobs that set option 'thread' into a single process. Have them all share a single struct blkio instance, with one struct blkioq per thread/subjob. This allows benchmarking multi-queue setups.
Note that struct blkio instances cannot be shared across different processes.
--- HOWTO.rst | 8 +- engines/libblkio.c | 250 +++++++++++++++++++--- examples/libblkio-io_uring.fio | 13 +- examples/libblkio-virtio-blk-vfio-pci.fio | 13 +- fio.1 | 7 +- 5 files changed, 257 insertions(+), 34 deletions(-)
HOWTO.rst +7 -1
@@ -2199,7 +2199,13 @@ I/O engine :option:`libblkio_driver`. If :option:`mem`/:option:`iomem` is not specified, memory allocation is delegated to libblkio (and so is - guaranteed to work with the selected *driver*). + guaranteed to work with the selected *driver*). One + ``struct blkio`` instance is used per process, so all + jobs setting option :option:`thread` will share a single + ``struct blkio`` (with one queue per thread) and must + specify compatible options. Note that some drivers don't + allow several processes to access the same device or + file simultaneously, but allow it for threads. I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
engines/libblkio.c +220 -30
360 lines not on this page · raw message
examples/libblkio-io_uring.fio +12 -1
@@ -3,6 +3,10 @@ ; Replace "libblkio_path" below with the path to your file or device, or ; override it by passing the '--libblkio_path=...' flag to fio. ; +; In the example below, the two subjobs of "job-B" *and* the single subjob of +; "job-C" will share a single `struct blkio` instance, and "job-A" will use a +; separate `struct blkio` instance. +; ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio [global]
@@ -15,4 +19,11 @@ direct=1 time_based=1 runtime=10s -[job] +[job-A] + +[job-B] +numjobs=2 ; run two copies of this job simultaneously +thread=1 ; have each copy run as a separate thread in the *same* process + +[job-C] +thread=1 ; have the job run as a thread in the *same* process as "job-B"
examples/libblkio-virtio-blk-vfio-pci.fio +12 -1
@@ -3,6 +3,10 @@ ; Replace "libblkio_path" below with the path to your device's sysfs directory, ; or override it by passing the '--libblkio_path=...' flag to fio. ; +; In the example below, the two subjobs of "job-B" *and* the single subjob of +; "job-C" will share a single `struct blkio` instance, and "job-A" will use a +; separate `struct blkio` instance. +; ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio [global]
@@ -14,4 +18,11 @@ blocksize=4k time_based=1 runtime=10s -[job] +[job-A] + +[job-B] +numjobs=2 ; run two copies of this job simultaneously +thread=1 ; have each copy run as a separate thread in the *same* process + +[job-C] +thread=1 ; have the job run as a thread in the *same* process as "job-B"
fio.1 +6 -1
@@ -1994,7 +1994,12 @@ engine specific options. (See \fIhttps://xnvme.io/\fR). Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The specific \fBdriver\fR to use must be set using \fBlibblkio_driver\fR. If \fBmem\fR/\fBiomem\fR is not specified, memory allocation is delegated to -libblkio (and so is guaranteed to work with the selected \fBdriver\fR). +libblkio (and so is guaranteed to work with the selected \fBdriver\fR). One +\fBstruct blkio\fR instance is used per process, so all jobs setting option +\fBthread\fR will share a single \fBstruct blkio\fR (with one queue per thread) +and must specify compatible options. Note that some drivers don't allow several +processes to access the same device or file simultaneously, but allow it for +threads. .SS "I/O engine specific parameters" In addition, there are some parameters which are only valid when a specific \fBioengine\fR is in use. These are used identically to normal parameters,
Signature · 1 line
2.38.1
Re: [PATCH 10/10] engines/libblkio: Share a single blkio instance among threads in same process
Quoted 56 lines · Alberto Faria
fio groups all subjobs that set option 'thread' into a single process. Have them all share a single
struct blkioinstance, with onestruct blkioqper thread/subjob. This allows benchmarking multi-queue setups.Note that
struct blkioinstances cannot be shared across different processes.Signed-off-by Alberto Faria <[email protected]>--- HOWTO.rst | 8 +- engines/libblkio.c | 250 +++++++++++++++++++--- examples/libblkio-io_uring.fio | 13 +- examples/libblkio-virtio-blk-vfio-pci.fio | 13 +- fio.1 | 7 +- 5 files changed, 257 insertions(+), 34 deletions(-)HOWTO.rst +7
-1@@ -2199,7 +2199,13 @@ I/O engine :option:`libblkio_driver`. If :option:`mem`/:option:`iomem` is not specified, memory allocation is delegated to libblkio (and so is - guaranteed to work with the selected *driver*). + guaranteed to work with the selected *driver*). One + ``struct blkio`` instance is used per process, so all + jobs setting option :option:`thread` will share a single + ``struct blkio`` (with one queue per thread) and must + specify compatible options. Note that some drivers don't + allow several processes to access the same device or + file simultaneously, but allow it for threads. I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~engines/libblkio.c +13
-0@@ -249,6 +249,105 @@ static int fio_blkio_set_props_from_str(struct blkio *b, const char *opt_name, blkio_get_error_msg()); \ }) +static bool possibly_null_strs_equal (const char *a,const char *b) +{ +return (!a && !b) || (a && b &&strcmp (a, b) ==0 ); +} + +/* +* Returns the total number of subjobs using option 'thread' in the entire +* workload that have the given value for the 'hipri' option. +*/ +static int total_threaded_subjobs (bool hipri) +{ +struct thread_data *td; +unsigned int i;
engines/libblkio.c static int fio_blkio_set_props_from_str(struct blkio *b, const char *opt_name, · line 274
Quoted 10 lines
engines/libblkio.c +10
-0+int count =0 ; + +for_each_td (td, i) { +const struct fio_blkio_options *options = td->eo; +if (td->o.use_thread && (bool )options->hipri == hipri) + ++count; + } + +return count; +}
The loop should skip jobs not using the libblkio ioengine because, for example, some ioengines don't have any ioengine options.