Skip to contentSkip to conversation

Add a libblkio engine

PATCH Applied Alberto Faria 2 participants 12 messages Dec 1, 2022 – Dec 2, 2022 Kbuild

lore
Share & tools
lore
#0 lore raw

v2 cover Add a libblkio engine

The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like io_uring and vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.

Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.

See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.

[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdf

v2:
- In the HOWTO and manpage, point users to the libblkio docs to know what drivers and properties are available.
- Avoid mentioning libblkio API symbols in the HOWTO and manpage.
- Don't document FIO_OPT_STR_SET options as having type bool in the HOWTO and manpage.
- Prevent null deref in fio_blkio_init() error path.
- Fix total_threaded_subjobs() to only count jobs using the libblkio engine.
- Force td->o.disable_slat = 1, as request enqueueing is fast and it is not possible to know exactly when a request is submitted.
- Have all type declarations near the top of the file.
- Be more economical with blank lines.

Alberto Faria (10):
Add a libblkio engine
Add engine flag FIO_SKIPPABLE_IOMEM_ALLOC engines/libblkio: Allow setting option mem/iomem engines/libblkio: Add support for poll queues engines/libblkio: Add option libblkio_vectored engines/libblkio: Add option libblkio_write_zeroes_on_trim engines/libblkio: Add option libblkio_wait_mode engines/libblkio: Add option libblkio_force_enable_completion_eventfd engines/libblkio: Add options for some driver-specific properties engines/libblkio: Share a single blkio instance among threads in same
process

 HOWTO.rst                                 |  95 +++
 Makefile                                  |   6 +
 configure                                 |  25 +
 engines/libblkio.c                        | 914 ++++++++++++++++++++++
 examples/libblkio-io_uring.fio            |  29 +
 examples/libblkio-virtio-blk-vfio-pci.fio |  29 +
 fio.1                                     |  78 ++
 ioengines.h                               |   2 +
 memory.c                                  |  22 +-
 optgroup.h                                |   2 +
 10 files changed, 1192 insertions(+), 10 deletions(-)
 create mode 100644 engines/libblkio.c
 create mode 100644 examples/libblkio-io_uring.fio
 create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fio
Signature · 1 line
2.38.1
#1 lore raw

PATCH v2 1/10 Add a libblkio engine

The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like io_uring and vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.

Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.

See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.

[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdf

---
 HOWTO.rst                                 |  33 ++
 Makefile                                  |   6 +
 configure                                 |  25 ++
 engines/libblkio.c                        | 433 ++++++++++++++++++++++
 examples/libblkio-io_uring.fio            |  18 +
 examples/libblkio-virtio-blk-vfio-pci.fio |  19 +
 fio.1                                     |  25 ++
 optgroup.h                                |   2 +
 8 files changed, 561 insertions(+)
 create mode 100644 engines/libblkio.c
 create mode 100644 examples/libblkio-io_uring.fio
 create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fio
HOWTO.rst +33 -0
index 0aaf033a..08140165 100644
@@ -2195,6 +2195,12 @@ I/O engine
 			the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes
 			engine specific options. (See https://xnvme.io).
 
+		**libblkio**
+			Use the libblkio library
+			(https://gitlab.com/libblkio/libblkio). The specific
+			*driver* to use must be set using
+			:option:`libblkio_driver`.
+
 I/O engine specific parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -2847,6 +2853,33 @@ with the caveat that when used on the command line, they must come after the
 
 	If this option is set. xnvme will use vectored read/write commands.
 
+.. option:: libblkio_driver=str : [libblkio]
+
+	The libblkio *driver* to use. Different drivers access devices through
+	different underlying interfaces. Available drivers depend on the
+	libblkio version in use and are listed at
+	https://libblkio.gitlab.io/libblkio/blkio.html#drivers
+
+.. option:: libblkio_pre_connect_props=str : [libblkio]
+
+	A colon-separated list of libblkio properties to be set after creating
+	but before connecting the libblkio instance. Each property must have the
+	format ``<name>=<value>``. Colons can be escaped as ``\:``. These are
+	set after the engine sets any other properties, so those can be
+	overriden. Available properties depend on the libblkio version in use
+	and are listed at
+	https://libblkio.gitlab.io/libblkio/blkio.html#properties
+
+.. option:: libblkio_pre_start_props=str : [libblkio]
+
+	A colon-separated list of libblkio properties to be set after connecting
+	but before starting the libblkio instance. Each property must have the
+	format ``<name>=<value>``. Colons can be escaped as ``\:``. These are
+	set after the engine sets any other properties, so those can be
+	overriden. Available properties depend on the libblkio version in use
+	and are listed at
+	https://libblkio.gitlab.io/libblkio/blkio.html#properties
+
 I/O depth
 ~~~~~~~~~
 
Makefile +6 -0
index 7bd572d7..9fd8f59b 100644
@@ -237,6 +237,12 @@ ifdef CONFIG_LIBXNVME
   xnvme_CFLAGS = $(LIBXNVME_CFLAGS)
   ENGINES += xnvme
 endif
+ifdef CONFIG_LIBBLKIO
+  libblkio_SRCS = engines/libblkio.c
+  libblkio_LIBS = $(LIBBLKIO_LIBS)
+  libblkio_CFLAGS = $(LIBBLKIO_CFLAGS)
+  ENGINES += libblkio
+endif
 ifeq ($(CONFIG_TARGET_OS), Linux)
   SOURCE += diskutil.c fifo.c blktrace.c cgroup.c trim.c engines/sg.c \
 		oslib/linux-dev-lookup.c engines/io_uring.c engines/nvme.c
configure +25 -0
index 1b12d268..6d8e3a87 100755
@@ -176,6 +176,7 @@ libiscsi="no"
 libnbd="no"
 libnfs=""
 xnvme=""
+libblkio=""
 libzbc=""
 dfs=""
 seed_buckets=""
@@ -248,6 +249,8 @@ for opt do
   ;;
   --disable-xnvme) xnvme="no"
   ;;
+  --disable-libblkio) libblkio="no"
+  ;;
   --disable-tcmalloc) disable_tcmalloc="yes"
   ;;
   --disable-libnfs) libnfs="no"
@@ -304,6 +307,7 @@ if test "$show_help" = "yes" ; then
   echo "--enable-libiscsi       Enable iscsi support"
   echo "--enable-libnbd         Enable libnbd (NBD engine) support"
   echo "--disable-xnvme         Disable xnvme support even if found"
+  echo "--disable-libblkio      Disable libblkio support even if found"
   echo "--disable-libzbc        Disable libzbc even if found"
   echo "--disable-tcmalloc      Disable tcmalloc support"
   echo "--dynamic-libengines    Lib-based ioengines as dynamic libraries"
@@ -2663,6 +2667,22 @@ if test "$xnvme" != "no" ; then
 fi
 print_config "xnvme engine" "$xnvme"
 
+##########################################
+# Check if we have libblkio
+if test "$libblkio" != "no" ; then
+  if check_min_lib_version blkio 1.0.0; then
+    libblkio="yes"
+    libblkio_cflags=$(pkg-config --cflags blkio)
+    libblkio_libs=$(pkg-config --libs blkio)
+  else
+    if test "$libblkio" = "yes" ; then
+      feature_not_found "libblkio" "libblkio-dev or libblkio-devel"
+    fi
+    libblkio="no"
+  fi
+fi
+print_config "libblkio engine" "$libblkio"
+
 ##########################################
 # check march=armv8-a+crc+crypto
 if test "$march_armv8_a_crc_crypto" != "yes" ; then
@@ -3276,6 +3296,11 @@ if test "$xnvme" = "yes" ; then
   echo "LIBXNVME_CFLAGS=$xnvme_cflags" >> $config_host_mak
   echo "LIBXNVME_LIBS=$xnvme_libs" >> $config_host_mak
 fi
+if test "$libblkio" = "yes" ; then
+  output_sym "CONFIG_LIBBLKIO"
+  echo "LIBBLKIO_CFLAGS=$libblkio_cflags" >> $config_host_mak
+  echo "LIBBLKIO_LIBS=$libblkio_libs" >> $config_host_mak
+fi
 if test "$dynamic_engines" = "yes" ; then
   output_sym "CONFIG_DYNAMIC_ENGINES"
 fi
engines/libblkio.c new file +433 -0

433 lines not on this page · raw message

examples/libblkio-io_uring.fio new file +18 -0
new file mode 100644
index 00000000..655a0b50
@@ -0,0 +1,18 @@
+; Benchmark accessing a regular file or block device using libblkio.
+;
+; Replace "/dev/nvme0n1" below with the path to your file or device, or override
+; it by passing the '--libblkio_pre_connect_props=path=...' flag to fio.
+;
+; For information on libblkio, see: https://gitlab.com/libblkio/libblkio
+
+[global]
+ioengine=libblkio
+libblkio_driver=io_uring
+libblkio_pre_connect_props=path=/dev/nvme0n1  ; REPLACE THIS WITH THE RIGHT PATH
+rw=randread
+blocksize=4k
+direct=1
+time_based=1
+runtime=10s
+
+[job]
examples/libblkio-virtio-blk-vfio-pci.fio new file +19 -0
new file mode 100644
index 00000000..425df4a6
@@ -0,0 +1,19 @@
+; Benchmark accessing a PCI virtio-blk device using libblkio.
+;
+; Replace "/sys/bus/pci/devices/0000\:00\:01.0" below with the path to your
+; device's sysfs directory, or override it by passing the
+; '--libblkio_pre_connect_props=path=...' flag to fio. Note that colons in the
+; path must be escaped with a backslash.
+;
+; For information on libblkio, see: https://gitlab.com/libblkio/libblkio
+
+[global]
+ioengine=libblkio
+libblkio_driver=virtio-blk-vfio-pci
+libblkio_pre_connect_props=path=/sys/bus/pci/devices/0000\:00\:01.0  ; REPLACE THIS WITH THE RIGHT PATH
+rw=randread
+blocksize=4k
+time_based=1
+runtime=10s
+
+[job]
fio.1 +25 -0
index 62af0bd2..3615f258 100644
@@ -1992,6 +1992,10 @@ I/O engine using the xNVMe C API, for NVMe devices. The xnvme engine provides
 flexibility to access GNU/Linux Kernel NVMe driver via libaio, IOCTLs, io_uring,
 the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes
 engine specific options. (See \fIhttps://xnvme.io/\fR).
+.TP
+.B libblkio
+Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The
+specific driver to use must be set using \fBlibblkio_driver\fR.
 .SS "I/O engine specific parameters"
 In addition, there are some parameters which are only valid when a specific
 \fBioengine\fR is in use. These are used identically to normal parameters,
@@ -2604,6 +2608,27 @@ xnvme namespace identifier for userspace NVMe driver such as SPDK.
 .TP
 .BI (xnvme)xnvme_iovec
 If this option is set, xnvme will use vectored read/write commands.
+.TP
+.BI (libblkio)libblkio_driver \fR=\fPstr
+The libblkio driver to use. Different drivers access devices through different
+underlying interfaces. Available drivers depend on the libblkio version in use
+and are listed at \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
+.BI (libblkio)libblkio_pre_connect_props \fR=\fPstr
+A colon-separated list of libblkio properties to be set after creating but
+before connecting the libblkio instance. Each property must have the format
+\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after
+the engine sets any other properties, so those can be overriden. Available
+properties depend on the libblkio version in use and are listed at
+\fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
+.TP
+.BI (libblkio)libblkio_pre_start_props \fR=\fPstr
+A colon-separated list of libblkio properties to be set after connecting but
+before starting the libblkio instance. Each property must have the format
+\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after
+the engine sets any other properties, so those can be overriden. Available
+properties depend on the libblkio version in use and are listed at
+\fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
 .SS "I/O depth"
 .TP
 .BI iodepth \fR=\fPint
optgroup.h +2 -0
index dc73c8f3..024b902f 100644
@@ -73,6 +73,7 @@ enum opt_category_group {
 	__FIO_OPT_G_NFS,
 	__FIO_OPT_G_WINDOWSAIO,
 	__FIO_OPT_G_XNVME,
+	__FIO_OPT_G_LIBBLKIO,
 
 	FIO_OPT_G_RATE		= (1ULL << __FIO_OPT_G_RATE),
 	FIO_OPT_G_ZONE		= (1ULL << __FIO_OPT_G_ZONE),
@@ -120,6 +121,7 @@ enum opt_category_group {
 	FIO_OPT_G_DFS		= (1ULL << __FIO_OPT_G_DFS),
 	FIO_OPT_G_WINDOWSAIO	= (1ULL << __FIO_OPT_G_WINDOWSAIO),
 	FIO_OPT_G_XNVME         = (1ULL << __FIO_OPT_G_XNVME),
+	FIO_OPT_G_LIBBLKIO	= (1ULL << __FIO_OPT_G_LIBBLKIO),
 };
 
 extern const struct opt_group *opt_group_from_mask(uint64_t *mask);
Signature · 1 line
2.38.1
#2 lore raw

PATCH v2 2/10 Add engine flag FIO_SKIPPABLE_IOMEM_ALLOC

In reply to Alberto Faria

It makes it valid to set option mem/iomem even when the engine specifies iomem_alloc and iomem_free callbacks, allowing users to optionally use fio's customizable memory allocation logic instead of the engine's.

This is in preparation for giving libblkio engine users the choice between controlling memory allocation or delegating it to the libblkio library.

---
 ioengines.h |  2 ++
 memory.c    | 22 ++++++++++++----------
 2 files changed, 14 insertions(+), 10 deletions(-)
ioengines.h +2 -0
index 11d2115c..d43540d0 100644
@@ -87,6 +87,8 @@ enum fio_ioengine_flags {
 	FIO_NO_OFFLOAD	= 1 << 15,	/* no async offload */
 	FIO_ASYNCIO_SETS_ISSUE_TIME
 			= 1 << 16,	/* async ioengine with commit function that sets issue_time */
+	FIO_SKIPPABLE_IOMEM_ALLOC
+			= 1 << 17,	/* skip iomem_alloc & iomem_free if job sets mem/iomem */
 };
 
 /*
memory.c +12 -10
index 6cf73333..577d3dd5 100644
@@ -305,16 +305,18 @@ int allocate_io_mem(struct thread_data *td)
 	dprint(FD_MEM, "Alloc %llu for buffers\n", (unsigned long long) total_mem);
 
 	/*
-	 * If the IO engine has hooks to allocate/free memory, use those. But
-	 * error out if the user explicitly asked for something else.
+	 * If the IO engine has hooks to allocate/free memory and the user
+	 * doesn't explicitly ask for something else, use those. But fail if the
+	 * user asks for something else with an engine that doesn't allow that.
 	 */
-	if (td->io_ops->iomem_alloc) {
-		if (fio_option_is_set(&td->o, mem_type)) {
-			log_err("fio: option 'mem/iomem' conflicts with specified IO engine\n");
-			ret = 1;
-		} else
-			ret = td->io_ops->iomem_alloc(td, total_mem);
-	} else if (td->o.mem_type == MEM_MALLOC)
+	if (td->io_ops->iomem_alloc && fio_option_is_set(&td->o, mem_type) &&
+	    !td_ioengine_flagged(td, FIO_SKIPPABLE_IOMEM_ALLOC)) {
+		log_err("fio: option 'mem/iomem' conflicts with specified IO engine\n");
+		ret = 1;
+	} else if (td->io_ops->iomem_alloc &&
+		   !fio_option_is_set(&td->o, mem_type))
+		ret = td->io_ops->iomem_alloc(td, total_mem);
+	else if (td->o.mem_type == MEM_MALLOC)
 		ret = alloc_mem_malloc(td, total_mem);
 	else if (td->o.mem_type == MEM_SHM || td->o.mem_type == MEM_SHMHUGE)
 		ret = alloc_mem_shm(td, total_mem);
@@ -342,7 +344,7 @@ void free_io_mem(struct thread_data *td)
 	if (td->o.odirect || td->o.oatomic)
 		total_mem += page_mask;
 
-	if (td->io_ops->iomem_alloc) {
+	if (td->io_ops->iomem_alloc && !fio_option_is_set(&td->o, mem_type)) {
 		if (td->io_ops->iomem_free)
 			td->io_ops->iomem_free(td);
 	} else if (td->o.mem_type == MEM_MALLOC)
Signature · 1 line
2.38.1
#3 lore raw

PATCH v2 3/10 engines/libblkio: Allow setting option mem/iomem

In reply to Alberto Faria

This allows users to customize data buffer memory using fio's existing options. Users become responsible for ensuring that the allocated memory satisfies all constraints imposed by the libblkio driver under use.

---
 HOWTO.rst          |  5 ++++-
 engines/libblkio.c | 46 ++++++++++++++++++++++++++++++++++++++++++++--
 fio.1              |  4 +++-
 3 files changed, 51 insertions(+), 4 deletions(-)
HOWTO.rst +4 -1
index 08140165..69ca1e79 100644
@@ -2199,7 +2199,10 @@ I/O engine
 			Use the libblkio library
 			(https://gitlab.com/libblkio/libblkio). The specific
 			*driver* to use must be set using
-			:option:`libblkio_driver`.
+			:option:`libblkio_driver`. If
+			:option:`mem`/:option:`iomem` is not specified, memory
+			allocation is delegated to libblkio (and so is
+			guaranteed to work with the selected *driver*).
 
 I/O engine specific parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
engines/libblkio.c +44 -2

67 lines not on this page · raw message

fio.1 +3 -1
index 3615f258..18428774 100644
@@ -1995,7 +1995,9 @@ engine specific options. (See \fIhttps://xnvme.io/\fR).
 .TP
 .B libblkio
 Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The
-specific driver to use must be set using \fBlibblkio_driver\fR.
+specific driver to use must be set using \fBlibblkio_driver\fR. If
+\fBmem\fR/\fBiomem\fR is not specified, memory allocation is delegated to
+libblkio (and so is guaranteed to work with the selected driver).
 .SS "I/O engine specific parameters"
 In addition, there are some parameters which are only valid when a specific
 \fBioengine\fR is in use. These are used identically to normal parameters,
Signature · 1 line
2.38.1
#4 lore raw

PATCH v2 4/10 engines/libblkio: Add support for poll queues

In reply to Alberto Faria

Configure a poll queue instead of a "regular" queue when option hipri is set.

---
 HOWTO.rst          |  4 ++++
 engines/libblkio.c | 25 +++++++++++++++++++++++--
 fio.1              |  3 +++
 3 files changed, 30 insertions(+), 2 deletions(-)
HOWTO.rst +4 -0
index 69ca1e79..2b3fe9c2 100644
@@ -2335,6 +2335,10 @@ with the caveat that when used on the command line, they must come after the
         by the application. The benefits are more efficient IO for high IOPS
         scenarios, and lower latencies for low queue depth IO.
 
+   [libblkio]
+
+	Use poll queues.
+
    [pvsync2]
 
 	Set RWF_HIPRI on I/O, indicating to the kernel that it's of higher priority
engines/libblkio.c +23 -2
index 11f45e2a..6decfd39 100644
@@ -37,6 +37,8 @@ struct fio_blkio_options {
 	char *driver;
 	char *pre_connect_props;
 	char *pre_start_props;
+
+	unsigned int hipri;
 };
 
 static struct fio_option options[] = {
@@ -67,6 +69,15 @@ static struct fio_option options[] = {
 		.category = FIO_OPT_C_ENGINE,
 		.group	= FIO_OPT_G_LIBBLKIO,
 	},
+	{
+		.name	= "hipri",
+		.lname	= "Use poll queues",
+		.type	= FIO_OPT_STR_SET,
+		.off1	= offsetof(struct fio_blkio_options, hipri),
+		.help	= "Use poll queues",
+		.category = FIO_OPT_C_ENGINE,
+		.group	= FIO_OPT_G_LIBBLKIO,
+	},
 	{
 		.name = NULL,
 	},
@@ -223,6 +234,7 @@ out_blkio_destroy:
 
 static int fio_blkio_init(struct thread_data *td)
 {
+	const struct fio_blkio_options *options = td->eo;
 	struct fio_blkio_data *data;
 
 	/*
@@ -246,7 +258,13 @@ static int fio_blkio_init(struct thread_data *td)
 	if (fio_blkio_create_and_connect(td, &data->b) != 0)
 		goto err_free;
 
-	if (blkio_set_int(data->b, "num-queues", 1) != 0) {
+	if (blkio_set_int(data->b, "num-queues", options->hipri ? 0 : 1) != 0) {
+		fio_blkio_log_err(blkio_set_int);
+		goto err_blkio_destroy;
+	}
+
+	if (blkio_set_int(data->b, "num-poll-queues",
+			  options->hipri ? 1 : 0) != 0) {
 		fio_blkio_log_err(blkio_set_int);
 		goto err_blkio_destroy;
 	}
@@ -256,7 +274,10 @@ static int fio_blkio_init(struct thread_data *td)
 		goto err_blkio_destroy;
 	}
 
-	data->q = blkio_get_queue(data->b, 0);
+	if (options->hipri)
+		data->q = blkio_get_poll_queue(data->b, 0);
+	else
+		data->q = blkio_get_queue(data->b, 0);
 
 	/* Set data last so cleanup() does nothing if init() fails. */
 	td->io_ops_data = data;
fio.1 +3 -0
index 18428774..7f97e940 100644
@@ -2631,6 +2631,9 @@ before starting the libblkio instance. Each property must have the format
 the engine sets any other properties, so those can be overriden. Available
 properties depend on the libblkio version in use and are listed at
 \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
+.TP
+.BI (libblkio)hipri
+Use poll queues.
 .SS "I/O depth"
 .TP
 .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
#5 lore raw

PATCH v2 5/10 engines/libblkio: Add option libblkio_vectored

In reply to Alberto Faria

When enabled, read and write requests are submitted as vectored requests using blkioq_{readv,writev}(), instead of using blkioq_{read,write}().

---
 HOWTO.rst          |  4 ++++
 engines/libblkio.c | 46 +++++++++++++++++++++++++++++++++++++++++-----
 fio.1              |  3 +++
 3 files changed, 48 insertions(+), 5 deletions(-)
HOWTO.rst +4 -0
index 2b3fe9c2..f77d76cc 100644
@@ -2887,6 +2887,10 @@ with the caveat that when used on the command line, they must come after the
 	and are listed at
 	https://libblkio.gitlab.io/libblkio/blkio.html#properties
 
+.. option:: libblkio_vectored : [libblkio]
+
+	Submit vectored read and write requests.
+
 I/O depth
 ~~~~~~~~~
 
engines/libblkio.c +41 -5

97 lines not on this page · raw message

fio.1 +3 -0
index 7f97e940..9b8995b5 100644
@@ -2634,6 +2634,9 @@ properties depend on the libblkio version in use and are listed at
 .TP
 .BI (libblkio)hipri
 Use poll queues.
+.TP
+.BI (libblkio)libblkio_vectored
+Submit vectored read and write requests.
 .SS "I/O depth"
 .TP
 .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
#6 lore raw

PATCH v2 6/10 engines/libblkio: Add option libblkio_write_zeroes_on_trim

In reply to Alberto Faria

When set, trim IOs will be submitted as blkioq_write_zeroes() requests instead of blkioq_discard() requests.

---
 HOWTO.rst          |  4 ++++
 engines/libblkio.c | 20 ++++++++++++++++++--
 fio.1              |  3 +++
 3 files changed, 25 insertions(+), 2 deletions(-)
HOWTO.rst +4 -0
index f77d76cc..273b7a68 100644
@@ -2891,6 +2891,10 @@ with the caveat that when used on the command line, they must come after the
 
 	Submit vectored read and write requests.
 
+.. option:: libblkio_write_zeroes_on_trim : [libblkio]
+
+	Submit trims as "write zeroes" requests instead of discard requests.
+
 I/O depth
 ~~~~~~~~~
 
engines/libblkio.c +18 -2
index 219bb0dc..66a9bd31 100644
@@ -41,6 +41,7 @@ struct fio_blkio_options {
 
 	unsigned int hipri;
 	unsigned int vectored;
+	unsigned int write_zeroes_on_trim;
 };
 
 static struct fio_option options[] = {
@@ -89,6 +90,16 @@ static struct fio_option options[] = {
 		.category = FIO_OPT_C_ENGINE,
 		.group	= FIO_OPT_G_LIBBLKIO,
 	},
+	{
+		.name	= "libblkio_write_zeroes_on_trim",
+		.lname	= "Use blkioq_write_zeroes() for TRIM",
+		.type	= FIO_OPT_STR_SET,
+		.off1	= offsetof(struct fio_blkio_options,
+				   write_zeroes_on_trim),
+		.help	= "Use blkioq_write_zeroes() for TRIM instead of blkioq_discard()",
+		.category = FIO_OPT_C_ENGINE,
+		.group	= FIO_OPT_G_LIBBLKIO,
+	},
 	{
 		.name = NULL,
 	},
@@ -455,8 +466,13 @@ static enum fio_q_status fio_blkio_queue(struct thread_data *td,
 			}
 			break;
 		case DDIR_TRIM:
-			blkioq_discard(data->q, io_u->offset, io_u->xfer_buflen,
-				       io_u, 0);
+			if (options->write_zeroes_on_trim) {
+				blkioq_write_zeroes(data->q, io_u->offset,
+						    io_u->xfer_buflen, io_u, 0);
+			} else {
+				blkioq_discard(data->q, io_u->offset,
+					       io_u->xfer_buflen, io_u, 0);
+			}
 		        break;
 		case DDIR_SYNC:
 		case DDIR_DATASYNC:
fio.1 +3 -0
index 9b8995b5..ad061caa 100644
@@ -2637,6 +2637,9 @@ Use poll queues.
 .TP
 .BI (libblkio)libblkio_vectored
 Submit vectored read and write requests.
+.TP
+.BI (libblkio)libblkio_write_zeroes_on_trim
+Submit trims as "write zeroes" requests instead of discard requests.
 .SS "I/O depth"
 .TP
 .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
#7 lore raw

PATCH v2 7/10 engines/libblkio: Add option libblkio_wait_mode

In reply to Alberto Faria

It allows configuring how the engine waits for request completions, instead of always using a blocking blkioq_do_io() call.

---
 HOWTO.rst          |  14 +++++-
 engines/libblkio.c | 117 ++++++++++++++++++++++++++++++++++++++++++---
 fio.1              |  16 ++++++-
 3 files changed, 139 insertions(+), 8 deletions(-)
HOWTO.rst +13 -1
index 273b7a68..e9602f59 100644
@@ -2337,7 +2337,8 @@ with the caveat that when used on the command line, they must come after the
 
    [libblkio]
 
-	Use poll queues.
+	Use poll queues. This is incompatible with
+	:option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`.
 
    [pvsync2]
 
@@ -2895,6 +2896,17 @@ with the caveat that when used on the command line, they must come after the
 
 	Submit trims as "write zeroes" requests instead of discard requests.
 
+.. option:: libblkio_wait_mode=str : [libblkio]
+
+	How to wait for completions:
+
+	**block** (default)
+		Use a blocking call to ``blkioq_do_io()``.
+	**eventfd**
+		Use a blocking call to ``read()`` on the completion eventfd.
+	**loop**
+		Use a busy loop with a non-blocking call to ``blkioq_do_io()``.
+
 I/O depth
 ~~~~~~~~~
 
engines/libblkio.c +111 -6

175 lines not on this page · raw message

fio.1 +15 -1
index ad061caa..1a810214 100644
@@ -2633,13 +2633,27 @@ properties depend on the libblkio version in use and are listed at
 \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
 .TP
 .BI (libblkio)hipri
-Use poll queues.
+Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR.
 .TP
 .BI (libblkio)libblkio_vectored
 Submit vectored read and write requests.
 .TP
 .BI (libblkio)libblkio_write_zeroes_on_trim
 Submit trims as "write zeroes" requests instead of discard requests.
+.TP
+.BI (libblkio)libblkio_wait_mode \fR=\fPstr
+How to wait for completions:
+.RS
+.RS
+.TP
+.B block \fR(default)
+Use a blocking call to \fBblkioq_do_io()\fR.
+.TP
+.B eventfd
+Use a blocking call to \fBread()\fR on the completion eventfd.
+.TP
+.B loop
+Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR.
 .SS "I/O depth"
 .TP
 .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
#8 lore raw

PATCH v2 8/10 engines/libblkio: Add option libblkio_force_enable_completion_eventfd

In reply to Alberto Faria

When set, the queue's completion fd is enabled even when it isn't used, i.e., even if option libblkio_wait_mode is _not_ set to "eventfd".

Depending on the libblkio driver, this can have an impact on performance. This option allows evaluating that overhead.

---
 HOWTO.rst          |  9 ++++++++-
 engines/libblkio.c | 21 +++++++++++++++++++--
 fio.1              | 10 +++++++++-
 3 files changed, 36 insertions(+), 4 deletions(-)
HOWTO.rst +8 -1
index e9602f59..1cea16c4 100644
@@ -2338,7 +2338,8 @@ with the caveat that when used on the command line, they must come after the
    [libblkio]
 
 	Use poll queues. This is incompatible with
-	:option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`.
+	:option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>` and
+	:option:`libblkio_force_enable_completion_eventfd`.
 
    [pvsync2]
 
@@ -2907,6 +2908,12 @@ with the caveat that when used on the command line, they must come after the
 	**loop**
 		Use a busy loop with a non-blocking call to ``blkioq_do_io()``.
 
+.. option:: libblkio_force_enable_completion_eventfd : [libblkio]
+
+	Enable the queue's completion eventfd even when unused. This may impact
+	performance. The default is to enable it only if
+	:option:`libblkio_wait_mode=eventfd <libblkio_wait_mode>`.
+
 I/O depth
 ~~~~~~~~~
 
engines/libblkio.c +19 -2
index 8a4828c3..7f5dcf36 100644
@@ -24,7 +24,7 @@
 struct fio_blkio_data {
 	struct blkio *b;
 	struct blkioq *q;
-	int completion_fd; /* -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */
+	int completion_fd; /* may be -1 if not FIO_BLKIO_WAIT_MODE_EVENTFD */
 
 	bool has_mem_region; /* whether mem_region is valid */
 	struct blkio_mem_region mem_region; /* only if allocated by libblkio */
@@ -50,6 +50,7 @@ struct fio_blkio_options {
 	unsigned int vectored;
 	unsigned int write_zeroes_on_trim;
 	enum fio_blkio_wait_mode wait_mode;
+	unsigned int force_enable_completion_eventfd;
 };
 
 static struct fio_option options[] = {
@@ -132,6 +133,16 @@ static struct fio_option options[] = {
 		.category = FIO_OPT_C_ENGINE,
 		.group	= FIO_OPT_G_LIBBLKIO,
 	},
+	{
+		.name	= "libblkio_force_enable_completion_eventfd",
+		.lname	= "Force enable the completion eventfd, even if unused",
+		.type	= FIO_OPT_STR_SET,
+		.off1	= offsetof(struct fio_blkio_options,
+				   force_enable_completion_eventfd),
+		.help	= "This can impact performance",
+		.category = FIO_OPT_C_ENGINE,
+		.group	= FIO_OPT_G_LIBBLKIO,
+	},
 	{
 		.name = NULL,
 	},
@@ -276,6 +287,11 @@ static int fio_blkio_setup(struct thread_data *td)
 		return 1;
 	}
 
+	if (options->hipri && options->force_enable_completion_eventfd) {
+		log_err("fio: option hipri is incompatible with option libblkio_force_enable_completion_eventfd\n");
+		return 1;
+	}
+
 	if (fio_blkio_create_and_connect(td, &b) != 0)
 		return 1;
 
@@ -342,7 +358,8 @@ static int fio_blkio_init(struct thread_data *td)
 	else
 		data->q = blkio_get_queue(data->b, 0);
 
-	if (options->wait_mode == FIO_BLKIO_WAIT_MODE_EVENTFD) {
+	if (options->wait_mode == FIO_BLKIO_WAIT_MODE_EVENTFD ||
+		options->force_enable_completion_eventfd) {
 		/* enable completion fd and make it blocking */
 		blkioq_set_completion_fd_enabled(data->q, true);
 		data->completion_fd = blkioq_get_completion_fd(data->q);
fio.1 +9 -1
index 1a810214..da548303 100644
@@ -2633,7 +2633,8 @@ properties depend on the libblkio version in use and are listed at
 \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
 .TP
 .BI (libblkio)hipri
-Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR.
+Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR and
+\fBlibblkio_force_enable_completion_eventfd\fR.
 .TP
 .BI (libblkio)libblkio_vectored
 Submit vectored read and write requests.
@@ -2654,6 +2655,13 @@ Use a blocking call to \fBread()\fR on the completion eventfd.
 .TP
 .B loop
 Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR.
+.RE
+.RE
+.TP
+.BI (libblkio)libblkio_force_enable_completion_eventfd
+Enable the queue's completion eventfd even when unused. This may impact
+performance. The default is to enable it only if
+\fBlibblkio_wait_mode=eventfd\fR.
 .SS "I/O depth"
 .TP
 .BI iodepth \fR=\fPint
Signature · 1 line
2.38.1
#9 lore raw

PATCH v2 9/10 engines/libblkio: Add options for some driver-specific properties

In reply to Alberto Faria

The properties are either common to several drivers or particularly relevant for benchmarking, so this should help write cleaner workload files.

---
 HOWTO.rst                                 | 42 +++++++++++----
 engines/libblkio.c                        | 62 ++++++++++++++++++++++-
 examples/libblkio-io_uring.fio            |  4 +-
 examples/libblkio-virtio-blk-vfio-pci.fio |  9 ++--
 fio.1                                     | 36 +++++++++----
 5 files changed, 124 insertions(+), 29 deletions(-)
HOWTO.rst +32 -10
index 1cea16c4..10e6bc79 100644
@@ -2869,23 +2869,45 @@ with the caveat that when used on the command line, they must come after the
 	libblkio version in use and are listed at
 	https://libblkio.gitlab.io/libblkio/blkio.html#drivers
 
+.. option:: libblkio_path=str : [libblkio]
+
+	Sets the value of the driver-specific "path" property before connecting
+	the libblkio instance, which identifies the target device or file on
+	which to perform I/O. Its exact semantics are driver-dependent and not
+	all drivers may support it; see
+	https://libblkio.gitlab.io/libblkio/blkio.html#drivers
+
 .. option:: libblkio_pre_connect_props=str : [libblkio]
 
-	A colon-separated list of libblkio properties to be set after creating
-	but before connecting the libblkio instance. Each property must have the
-	format ``<name>=<value>``. Colons can be escaped as ``\:``. These are
-	set after the engine sets any other properties, so those can be
-	overriden. Available properties depend on the libblkio version in use
+	A colon-separated list of additional libblkio properties to be set after
+	creating but before connecting the libblkio instance. Each property must
+	have the format ``<name>=<value>``. Colons can be escaped as ``\:``.
+	These are set after the engine sets any other properties, so those can
+	be overriden. Available properties depend on the libblkio version in use
 	and are listed at
 	https://libblkio.gitlab.io/libblkio/blkio.html#properties
 
+.. option:: libblkio_num_entries=int : [libblkio]
+
+	Sets the value of the driver-specific "num-entries" property before
+	starting the libblkio instance. Its exact semantics are driver-dependent
+	and not all drivers may support it; see
+	https://libblkio.gitlab.io/libblkio/blkio.html#drivers
+
+.. option:: libblkio_queue_size=int : [libblkio]
+
+	Sets the value of the driver-specific "queue-size" property before
+	starting the libblkio instance. Its exact semantics are driver-dependent
+	and not all drivers may support it; see
+	https://libblkio.gitlab.io/libblkio/blkio.html#drivers
+
 .. option:: libblkio_pre_start_props=str : [libblkio]
 
-	A colon-separated list of libblkio properties to be set after connecting
-	but before starting the libblkio instance. Each property must have the
-	format ``<name>=<value>``. Colons can be escaped as ``\:``. These are
-	set after the engine sets any other properties, so those can be
-	overriden. Available properties depend on the libblkio version in use
+	A colon-separated list of additional libblkio properties to be set after
+	connecting but before starting the libblkio instance. Each property must
+	have the format ``<name>=<value>``. Colons can be escaped as ``\:``.
+	These are set after the engine sets any other properties, so those can
+	be overriden. Available properties depend on the libblkio version in use
 	and are listed at
 	https://libblkio.gitlab.io/libblkio/blkio.html#properties
 
engines/libblkio.c +60 -2

97 lines not on this page · raw message

examples/libblkio-io_uring.fio +2 -2
index 655a0b50..3485b97e 100644
@@ -1,14 +1,14 @@
 ; Benchmark accessing a regular file or block device using libblkio.
 ;
 ; Replace "/dev/nvme0n1" below with the path to your file or device, or override
-; it by passing the '--libblkio_pre_connect_props=path=...' flag to fio.
+; it by passing the '--libblkio_path=...' flag to fio.
 ;
 ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio
 
 [global]
 ioengine=libblkio
 libblkio_driver=io_uring
-libblkio_pre_connect_props=path=/dev/nvme0n1  ; REPLACE THIS WITH THE RIGHT PATH
+libblkio_path=/dev/nvme0n1  ; REPLACE THIS WITH THE RIGHT PATH
 rw=randread
 blocksize=4k
 direct=1
examples/libblkio-virtio-blk-vfio-pci.fio +4 -5
index 425df4a6..6bed664b 100644
@@ -1,16 +1,15 @@
 ; Benchmark accessing a PCI virtio-blk device using libblkio.
 ;
-; Replace "/sys/bus/pci/devices/0000\:00\:01.0" below with the path to your
-; device's sysfs directory, or override it by passing the
-; '--libblkio_pre_connect_props=path=...' flag to fio. Note that colons in the
-; path must be escaped with a backslash.
+; Replace "/sys/bus/pci/devices/0000:00:01.0" below with the path to your
+; device's sysfs directory, or override it by passing the '--libblkio_path=...'
+; flag to fio.
 ;
 ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio
 
 [global]
 ioengine=libblkio
 libblkio_driver=virtio-blk-vfio-pci
-libblkio_pre_connect_props=path=/sys/bus/pci/devices/0000\:00\:01.0  ; REPLACE THIS WITH THE RIGHT PATH
+libblkio_path=/sys/bus/pci/devices/0000:00:01.0  ; REPLACE THIS WITH THE RIGHT PATH
 rw=randread
 blocksize=4k
 time_based=1
fio.1 +26 -10
index da548303..6f7a608d 100644
@@ -2616,20 +2616,36 @@ The libblkio driver to use. Different drivers access devices through different
 underlying interfaces. Available drivers depend on the libblkio version in use
 and are listed at \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
 .TP
+.BI (libblkio)libblkio_path \fR=\fPstr
+Sets the value of the driver-specific "path" property before connecting the
+libblkio instance, which identifies the target device or file on which to
+perform I/O. Its exact semantics are driver-dependent and not all drivers may
+support it; see \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
 .BI (libblkio)libblkio_pre_connect_props \fR=\fPstr
-A colon-separated list of libblkio properties to be set after creating but
-before connecting the libblkio instance. Each property must have the format
-\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after
-the engine sets any other properties, so those can be overriden. Available
-properties depend on the libblkio version in use and are listed at
+A colon-separated list of additional libblkio properties to be set after
+creating but before connecting the libblkio instance. Each property must have
+the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are
+set after the engine sets any other properties, so those can be overriden.
+Available properties depend on the libblkio version in use and are listed at
 \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
 .TP
+.BI (libblkio)libblkio_num_entries \fR=\fPint
+Sets the value of the driver-specific "num-entries" property before starting the
+libblkio instance. Its exact semantics are driver-dependent and not all drivers
+may support it; see \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
+.BI (libblkio)libblkio_queue_size \fR=\fPint
+Sets the value of the driver-specific "queue-size" property before starting the
+libblkio instance. Its exact semantics are driver-dependent and not all drivers
+may support it; see \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
 .BI (libblkio)libblkio_pre_start_props \fR=\fPstr
-A colon-separated list of libblkio properties to be set after connecting but
-before starting the libblkio instance. Each property must have the format
-\fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are set after
-the engine sets any other properties, so those can be overriden. Available
-properties depend on the libblkio version in use and are listed at
+A colon-separated list of additional libblkio properties to be set after
+connecting but before starting the libblkio instance. Each property must have
+the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are
+set after the engine sets any other properties, so those can be overriden.
+Available properties depend on the libblkio version in use and are listed at
 \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
 .TP
 .BI (libblkio)hipri
Signature · 1 line
2.38.1
#10 lore raw

PATCH v2 10/10 engines/libblkio: Share a single blkio instance among threads in same process

In reply to Alberto Faria

fio groups all subjobs that set option 'thread' into a single process. Have them all share a single struct blkio instance, with one struct blkioq per thread/subjob. This allows benchmarking multi-queue setups.

Note that struct blkio instances cannot be shared across different processes.

---
 HOWTO.rst                                 |   8 +-
 engines/libblkio.c                        | 242 +++++++++++++++++++---
 examples/libblkio-io_uring.fio            |  13 +-
 examples/libblkio-virtio-blk-vfio-pci.fio |  13 +-
 fio.1                                     |   6 +-
 5 files changed, 250 insertions(+), 32 deletions(-)
HOWTO.rst +7 -1
index 10e6bc79..5a5263c3 100644
@@ -2202,7 +2202,13 @@ I/O engine
 			:option:`libblkio_driver`. If
 			:option:`mem`/:option:`iomem` is not specified, memory
 			allocation is delegated to libblkio (and so is
-			guaranteed to work with the selected *driver*).
+			guaranteed to work with the selected *driver*). One
+			libblkio instance is used per process, so all jobs
+			setting option :option:`thread` will share a single
+			instance (with one queue per thread) and must specify
+			compatible options. Note that some drivers don't allow
+			several instances to access the same device or file
+			simultaneously, but allow it for threads.
 
 I/O engine specific parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
engines/libblkio.c +214 -28

343 lines not on this page · raw message

examples/libblkio-io_uring.fio +12 -1
index 3485b97e..40f625cf 100644
@@ -3,6 +3,10 @@
 ; Replace "/dev/nvme0n1" below with the path to your file or device, or override
 ; it by passing the '--libblkio_path=...' flag to fio.
 ;
+; In the example below, the two subjobs of "job-B" *and* the single subjob of
+; "job-C" will share a single libblkio instance, and "job-A" will use a separate
+; libblkio instance.
+;
 ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio
 
 [global]
@@ -15,4 +19,11 @@ direct=1
 time_based=1
 runtime=10s
 
-[job]
+[job-A]
+
+[job-B]
+numjobs=2  ; run two copies of this job simultaneously
+thread=1   ; have each copy run as a separate thread in the *same* process
+
+[job-C]
+thread=1  ; have the job run as a thread in the *same* process as "job-B"
examples/libblkio-virtio-blk-vfio-pci.fio +12 -1
index 6bed664b..024224a6 100644
@@ -4,6 +4,10 @@
 ; device's sysfs directory, or override it by passing the '--libblkio_path=...'
 ; flag to fio.
 ;
+; In the example below, the two subjobs of "job-B" *and* the single subjob of
+; "job-C" will share a single libblkio instance, and "job-A" will use a separate
+; libblkio instance.
+;
 ; For information on libblkio, see: https://gitlab.com/libblkio/libblkio
 
 [global]
@@ -15,4 +19,11 @@ blocksize=4k
 time_based=1
 runtime=10s
 
-[job]
+[job-A]
+
+[job-B]
+numjobs=2  ; run two copies of this job simultaneously
+thread=1   ; have each copy run as a separate thread in the *same* process
+
+[job-C]
+thread=1  ; have the job run as a thread in the *same* process as "job-B"
fio.1 +5 -1
index 6f7a608d..7a153731 100644
@@ -1997,7 +1997,11 @@ engine specific options. (See \fIhttps://xnvme.io/\fR).
 Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The
 specific driver to use must be set using \fBlibblkio_driver\fR. If
 \fBmem\fR/\fBiomem\fR is not specified, memory allocation is delegated to
-libblkio (and so is guaranteed to work with the selected driver).
+libblkio (and so is guaranteed to work with the selected driver). One libblkio
+instance is used per process, so all jobs setting option \fBthread\fR will share
+a single instance (with one queue per thread) and must specify compatible
+options. Note that some drivers don't allow several instances to access the same
+device or file simultaneously, but allow it for threads.
 .SS "I/O engine specific parameters"
 In addition, there are some parameters which are only valid when a specific
 \fBioengine\fR is in use. These are used identically to normal parameters,
Signature · 1 line
2.38.1
Applied
#11 lore raw
In reply to Alberto Faria
Quoted 56 lines · Alberto Faria

The libblkio library provides a unified API for efficiently accessing block devices using modern high-performance block I/O interfaces like io_uring and vhost-user-blk. Using libblkio reduces the amount of code needed for interfacing with storage devices and allows developers to focus on their applcations.

Add a libblkio engine that uses libblkio to perform I/O. This is useful to benchmark the library itself, and also adds support for storage interfaces and devices otherwise not supported by fio, such as virtio-blk PCI, vhost-user, and vhost-vDPA devices.

See the libblkio documentation [2] or KVM Forum 2022 [3] presentation for more information on the library itself.

[1] https://gitlab.com/libblkio/libblkio
[2] https://libblkio.gitlab.io/libblkio/index.html
[3] https://static.sched.com/hosted_files/kvmforum2022/8c/libblkio-kvm-forum-2022.pdf

v2:
- In the HOWTO and manpage, point users to the libblkio docs to know what drivers and properties are available.
- Avoid mentioning libblkio API symbols in the HOWTO and manpage.
- Don't document FIO_OPT_STR_SET options as having type bool in the HOWTO and manpage.
- Prevent null deref in fio_blkio_init() error path.
- Fix total_threaded_subjobs() to only count jobs using the libblkio engine.
- Force td->o.disable_slat = 1, as request enqueueing is fast and it is not possible to know exactly when a request is submitted.
- Have all type declarations near the top of the file.
- Be more economical with blank lines.

Alberto Faria (10):
Add a libblkio engine
Add engine flag FIO_SKIPPABLE_IOMEM_ALLOC engines/libblkio: Allow setting option mem/iomem engines/libblkio: Add support for poll queues engines/libblkio: Add option libblkio_vectored engines/libblkio: Add option libblkio_write_zeroes_on_trim engines/libblkio: Add option libblkio_wait_mode engines/libblkio: Add option libblkio_force_enable_completion_eventfd engines/libblkio: Add options for some driver-specific properties engines/libblkio: Share a single blkio instance among threads in same
process

  HOWTO.rst                                 |  95 +++
  Makefile                                  |   6 +
  configure                                 |  25 +
  engines/libblkio.c                        | 914 ++++++++++++++++++++++
  examples/libblkio-io_uring.fio            |  29 +
  examples/libblkio-virtio-blk-vfio-pci.fio |  29 +
  fio.1                                     |  78 ++
  ioengines.h                               |   2 +
  memory.c                                  |  22 +-
  optgroup.h                                |   2 +
  10 files changed, 1192 insertions(+), 10 deletions(-)
Quoted 4 lines

create mode 100644 engines/libblkio.c
create mode 100644 examples/libblkio-io_uring.fio create mode 100644 examples/libblkio-virtio-blk-vfio-pci.fio

Applied. Thanks.