Skip to contentSkip to conversation

io_uring: add Linux io_uring AIO engine

PATCH Applied Stefan Hajnoczi 6 participants 27 messages Oct 25, 2019 – Dec 3, 2019 Block layer

v2 of 5 Applied Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover io_uring: add Linux io_uring AIO engine

v11:
* Drop fd registration because it breaks QEMU's file locking and will need to be resolved in a separate patch series
* Drop line-wrapping changes that accidentally broke several qemu-iotests

v10:
* Dropped kernel submission queue polling, it requires root and has additional limitations. It should be benchmarked and considered for inclusion later, maybe even together with kernel side changes.
* Add io_uring_register_files() return value to trace_luring_fd_register()
* Fix indentation in luring_fd_unregister()
* Set s->fd_reg.fd_array to NULL after g_free() to avoid dangling pointers
* Simplify fd registration code
* Add luring_fd_unregister() and call it from file-posix.c to prevent fd leaks
* Add trace_luring_fd_unregister() trace event
* Add missing space to qemu-img command-line documentation
* Update MAINTAINERS file [Julia]
* Rename MAX_EVENTS to MAX_ENTRIES [Julia]
* Define ioq_submit() before callers so the prototype isn't necessary [Julia]
* Declare variables at the beginning of the block in luring_init() [Julia]

This patch series is based on Aarushi Mehta's v9 patch series written for Google Summer of Code 2019:

https://lists.gnu.org/archive/html/qemu-devel/2019-08/msg00179.html

It adds a new AIO engine that uses the new Linux io_uring API. This is the successor to Linux AIO with a number of improvements:
1. Both O_DIRECT and buffered I/O work
2. fdatasync(2) is supported (no need for a separate thread pool!)
3. True async behavior so the syscall doesn't block (Linux AIO got there to some degree...)
4. Advanced performance optimizations are available (file registration, memory buffer registration, completion polling, submission polling).

Since Aarushi has been busy, I have taken up this patch series. Booting a guest works with -drive aio=io_uring and -drive aio=io_uring,cache=none with a raw file on XFS.

I currently recommend using -drive aio=io_uring only with host block devices (like NVMe devices). As of Linux v5.4-rc1 I still hit kernel bugs when using image files on ext4 or XFS.

Aarushi Mehta (15):
configure: permit use of io_uring qapi/block-core: add option for io_uring block/block: add BDRV flag for io_uring block/io_uring: implements interfaces for io_uring
stubs: add stubs for io_uring interface util/async: add aio interfaces for io_uring
blockdev: adds bdrv_parse_aio to use io_uring block/file-posix.c: extend to use io_uring
block: add trace events for io_uring block/io_uring: adds userspace completion polling
qemu-io: adds option to use aio engine
qemu-img: adds option to use aio engine for benchmarking
qemu-nbd: adds option for aio engines tests/qemu-iotests: enable testing with aio options tests/qemu-iotests: use AIOMODE with various tests

 MAINTAINERS                   |   9 +
 qapi/block-core.json          |   4 +-
 configure                     |  27 +++
 block/Makefile.objs           |   3 +
 stubs/Makefile.objs           |   1 +
 include/block/aio.h           |  16 +-
 include/block/block.h         |   2 +
 include/block/raw-aio.h       |  12 +
 block.c                       |  22 ++
 block/file-posix.c            |  99 ++++++--
 block/io_uring.c              | 433 ++++++++++++++++++++++++++++++++++
 blockdev.c                    |  12 +-
 qemu-img.c                    |  11 +-
 qemu-io.c                     |  25 +-
 qemu-nbd.c                    |  12 +-
 stubs/io_uring.c              |  32 +++
 util/async.c                  |  36 +++
 block/trace-events            |  12 +
 qemu-img-cmds.hx              |   4 +-
 qemu-img.texi                 |   5 +-
 qemu-nbd.texi                 |   4 +-
 tests/qemu-iotests/028        |   2 +-
 tests/qemu-iotests/058        |   2 +-
 tests/qemu-iotests/089        |   4 +-
 tests/qemu-iotests/091        |   4 +-
 tests/qemu-iotests/109        |   2 +-
 tests/qemu-iotests/147        |   5 +-
 tests/qemu-iotests/181        |   8 +-
 tests/qemu-iotests/183        |   4 +-
 tests/qemu-iotests/185        |  10 +-
 tests/qemu-iotests/200        |   2 +-
 tests/qemu-iotests/201        |   8 +-
 tests/qemu-iotests/check      |  15 +-
 tests/qemu-iotests/common.rc  |  14 ++
 tests/qemu-iotests/iotests.py |  12 +-
 35 files changed, 797 insertions(+), 76 deletions(-)
 create mode 100644 block/io_uring.c
 create mode 100644 stubs/io_uring.c
Signature · 1 line
2.21.0
#1 lore raw

PATCH v2 1/15 configure: permit use of io_uring

From: Aarushi Mehta <[email protected]>

---
 configure | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
configure +27 -0
index 145fcabbb3..745429c925 100755
@@ -370,6 +370,7 @@ xen=""
 xen_ctrl_version=""
 xen_pci_passthrough=""
 linux_aio=""
+linux_io_uring=""
 cap_ng=""
 attr=""
 libattr=""
@@ -1250,6 +1251,10 @@ for opt do
   ;;
   --enable-linux-aio) linux_aio="yes"
   ;;
+  --disable-linux-io-uring) linux_io_uring="no"
+  ;;
+  --enable-linux-io-uring) linux_io_uring="yes"
+  ;;
   --disable-attr) attr="no"
   ;;
   --enable-attr) attr="yes"
@@ -1760,6 +1765,7 @@ disabled with --disable-FEATURE, default is enabled if available:
   vde             support for vde network
   netmap          support for netmap network
   linux-aio       Linux AIO support
+  linux-io-uring  Linux io_uring support
   cap-ng          libcap-ng support
   attr            attr and xattr support
   vhost-net       vhost-net kernel acceleration support
@@ -3950,6 +3956,21 @@ EOF
     linux_aio=no
   fi
 fi
+##########################################
+# linux-io-uring probe
+
+if test "$linux_io_uring" != "no" ; then
+  if $pkg_config liburing; then
+    linux_io_uring_cflags=$($pkg_config --cflags liburing)
+    linux_io_uring_libs=$($pkg_config --libs liburing)
+    linux_io_uring=yes
+  else
+    if test "$linux_io_uring" = "yes" ; then
+      feature_not_found "linux io_uring" "Install liburing devel"
+    fi
+    linux_io_uring=no
+  fi
+fi
 
 ##########################################
 # TPM emulation is only on POSIX
@@ -6356,6 +6377,7 @@ echo "PIE               $pie"
 echo "vde support       $vde"
 echo "netmap support    $netmap"
 echo "Linux AIO support $linux_aio"
+echo "Linux io_uring support $linux_io_uring"
 echo "ATTR/XATTR support $attr"
 echo "Install blobs     $blobs"
 echo "KVM support       $kvm"
@@ -6844,6 +6866,11 @@ fi
 if test "$linux_aio" = "yes" ; then
   echo "CONFIG_LINUX_AIO=y" >> $config_host_mak
 fi
+if test "$linux_io_uring" = "yes" ; then
+  echo "CONFIG_LINUX_IO_URING=y" >> $config_host_mak
+  echo "LINUX_IO_URING_CFLAGS=$linux_io_uring_cflags" >> $config_host_mak
+  echo "LINUX_IO_URING_LIBS=$linux_io_uring_libs" >> $config_host_mak
+fi
 if test "$attr" = "yes" ; then
   echo "CONFIG_ATTR=y" >> $config_host_mak
 fi
Signature · 1 line
2.21.0
#2 lore raw

PATCH v2 2/15 qapi/block-core: add option for io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

Only enumerates option for devices that support it. Since QAPI schema supports io_uring, which is the actual name of the Linux API, it is preferred over io-uring.

---
 qapi/block-core.json | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
qapi/block-core.json +3 -1
index b274aef713..3196f40178 100644
@@ -2851,11 +2851,13 @@
 #
 # @threads:     Use qemu's thread pool
 # @native:      Use native AIO backend (only Linux and Windows)
+# @io_uring:    Use linux io_uring (since 4.2)
 #
 # Since: 2.9
 ##
 { 'enum': 'BlockdevAioOptions',
-  'data': [ 'threads', 'native' ] }
+  'data': [ 'threads', 'native',
+            { 'name': 'io_uring', 'if': 'defined(CONFIG_LINUX_IO_URING)' } ] }
 
 ##
 # @BlockdevCacheOptions:
Signature · 1 line
2.21.0
Acked-by on v2 2/15
#3 lore raw

Re: [PATCH v2 02/15] qapi/block-core: add option for io_uring

Quoted 3 lines · Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

Only enumerates option for devices that support it.

I'm not sure I get this sentence.

Quoted 3 lines

Since QAPI schema
supports io_uring, which is the actual name of the Linux API, it is preferred over io-uring.

I guess this one means something like "Since io_uring is the actual name of the Linux API, we use it as enum value even though the QAPI schema conventions would prefer io-uring."

Quoted 15 lines
---
 qapi/block-core.json | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
qapi/block-core.json +1 -0
index b274aef713..3196f40178 100644
@@ -2851,11 +2851,13 @@
 #
 # @threads:     Use qemu's thread pool
 # @native:      Use native AIO backend (only Linux and Windows)
+# @io_uring:    Use linux io_uring (since 4.2)

qapi/block-core.json line 2863

Quoted 10 lines
qapi/block-core.json +2 -1
 #
 # Since: 2.9
 ##
 { 'enum': 'BlockdevAioOptions',
-  'data': [ 'threads', 'native' ] }
+  'data': [ 'threads', 'native',
+            { 'name': 'io_uring', 'if': 'defined(CONFIG_LINUX_IO_URING)' } ] }
 
 ##
 # @BlockdevCacheOptions:

I encourage you to polish the commit message some.

#4 lore raw

PATCH v2 3/15 block/block: add BDRV flag for io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 include/block/block.h | 1 +
 1 file changed, 1 insertion(+)
include/block/block.h +1 -0
index 89606bd9f8..bdb48dcd1b 100644
@@ -126,6 +126,7 @@ typedef struct HDGeometry {
                                       ignoring the format layer */
 #define BDRV_O_NO_IO       0x10000 /* don't initialize for I/O */
 #define BDRV_O_AUTO_RDONLY 0x20000 /* degrade to read-only if opening read-write fails */
+#define BDRV_O_IO_URING    0x40000 /* use io_uring instead of the thread pool */
 
 #define BDRV_O_CACHE_MASK  (BDRV_O_NOCACHE | BDRV_O_NO_FLUSH)
 
Signature · 1 line
2.21.0
#5 lore raw

Re: [PATCH v2 03/15] block/block: add BDRV flag for io_uring

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 14 lines

From: Aarushi Mehta <[email protected]>

---
 include/block/block.h | 1 +
 1 file changed, 1 insertion(+)
include/block/block.h +0 -0
index 89606bd9f8..bdb48dcd1b 100644

include/block/block.h typedef struct HDGeometry { · line 129

Quoted 4 lines
include/block/block.h +1 -0
@@ -126,6 +126,7 @@ typedef struct HDGeometry {
                                       ignoring the format layer */
 #define BDRV_O_NO_IO       0x10000 /* don't initialize for I/O */
 #define BDRV_O_AUTO_RDONLY 0x20000 /* degrade to read-only if opening read-write fails */
+#define BDRV_O_IO_URING    0x40000 /* use io_uring instead of the thread pool */

Why do we need a new (global!) flag rather than a driver-specific option for file-posix? This looks wrong to me, the flag has no sensible meaning for any other driver.

(Yes, BDRV_O_NATIVE_AIO is wrong, too, but compatibility means we can't remove it easily.)

Kevin

#6 lore raw

Re: [PATCH v2 03/15] block/block: add BDRV flag for io_uring

Quoted 24 lines · Kevin Wolf

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 14 lines

From: Aarushi Mehta <[email protected]>

---
 include/block/block.h | 1 +
 1 file changed, 1 insertion(+)
include/block/block.h +0 -0
index 89606bd9f8..bdb48dcd1b 100644

include/block/block.h typedef struct HDGeometry { · line 129

Quoted 4 lines
include/block/block.h +1 -0
@@ -126,6 +126,7 @@ typedef struct HDGeometry {
                                       ignoring the format layer */
 #define BDRV_O_NO_IO       0x10000 /* don't initialize for I/O */
 #define BDRV_O_AUTO_RDONLY 0x20000 /* degrade to read-only if opening read-write fails */
+#define BDRV_O_IO_URING    0x40000 /* use io_uring instead of the thread pool */

Why do we need a new (global!) flag rather than a driver-specific option for file-posix? This looks wrong to me, the flag has no sensible meaning for any other driver.

Quoted 2 lines

(Yes, BDRV_O_NATIVE_AIO is wrong, too, but compatibility means we can't remove it easily.)

To add to that, Kevin and me had a short talk on IRC, and it seemed like the reason would be that it’s easier to use this way than an option would be.

To me, the problem seems to be that it’s only easier to use because of the option inheritance we have in the block layer (so you can set this flag on a qcow2 node and its file node will have it, too). But naturally this inheritance is a bit of magic (because qemu has to guess where you probably want the flag to end up), so I’d too rather avoid adding to it.

OTOH, I wonder whether ease of use is really that important here. Would people who want to use io_uring really care about a command line that’s going to be a bit more complicated than just
-drive file=foo.img,format=$imgfmt,aio=io_uring,if=$interface? (Namely file.aio in this case, or maybe even a full-on block graph definition.)

Max

#7 lore raw

Re: [PATCH v2 03/15] block/block: add BDRV flag for io_uring

Quoted 39 lines · Max Reitz
Quoted 24 lines · Kevin Wolf

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 14 lines

From: Aarushi Mehta <[email protected]>

---
 include/block/block.h | 1 +
 1 file changed, 1 insertion(+)
include/block/block.h +0 -0
index 89606bd9f8..bdb48dcd1b 100644

include/block/block.h typedef struct HDGeometry { · line 129

Quoted 4 lines
include/block/block.h +1 -0
@@ -126,6 +126,7 @@ typedef struct HDGeometry {
                                       ignoring the format layer */
 #define BDRV_O_NO_IO       0x10000 /* don't initialize for I/O */
 #define BDRV_O_AUTO_RDONLY 0x20000 /* degrade to read-only if opening read-write fails */
+#define BDRV_O_IO_URING    0x40000 /* use io_uring instead of the thread pool */

Why do we need a new (global!) flag rather than a driver-specific option for file-posix? This looks wrong to me, the flag has no sensible meaning for any other driver.

Quoted 2 lines

(Yes, BDRV_O_NATIVE_AIO is wrong, too, but compatibility means we can't remove it easily.)

To add to that, Kevin and me had a short talk on IRC, and it seemed like the reason would be that it’s easier to use this way than an option would be.

To me, the problem seems to be that it’s only easier to use because of the option inheritance we have in the block layer (so you can set this flag on a qcow2 node and its file node will have it, too). But naturally this inheritance is a bit of magic (because qemu has to guess where you probably want the flag to end up), so I’d too rather avoid adding to it.

Quoted 5 lines

OTOH, I wonder whether ease of use is really that important here. Would people who want to use io_uring really care about a command line that’s going to be a bit more complicated than just
-drive file=foo.img,format=$imgfmt,aio=io_uring,if=$interface? (Namely file.aio in this case, or maybe even a full-on block graph definition.)

Thanks for the idea. I agree it's cleaner to avoid a new global flag.

I'll take a look at fixing this and let you know if I hit any problems.

Stefan

#8 lore raw

PATCH v2 4/15 block/io_uring: implements interfaces for io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

Aborts when sqe fails to be set as sqes cannot be returned to the ring. Adds slow path for short reads for older kernels

---
v10:
* Update MAINTAINERS file [Julia]
* Rename MAX_EVENTS to MAX_ENTRIES [Julia]
* Define ioq_submit() before callers so the prototype isn't necessary [Julia]
* Declare variables at the beginning of the block in luring_init() [Julia]

---
 MAINTAINERS             |   8 +
 block/Makefile.objs     |   3 +
 include/block/aio.h     |  16 +-
 include/block/raw-aio.h |  12 ++
 block/io_uring.c        | 403 ++++++++++++++++++++++++++++++++++++++++
 5 files changed, 441 insertions(+), 1 deletion(-)
 create mode 100644 block/io_uring.c
MAINTAINERS +8 -0
index ed41d7d1b6..31b090033c 100644
@@ -2544,6 +2544,14 @@ F: block/file-posix.c
 F: block/file-win32.c
 F: block/win32-aio.c
 
+Linux io_uring
+M: Aarushi Mehta <[email protected]>
+M: Julia Suvorova <[email protected]>
+M: Stefan Hajnoczi <[email protected]>
+L: [email protected]
+S: Maintained
+F: block/io_uring.c
+
 qcow2
 M: Kevin Wolf <[email protected]>
 M: Max Reitz <[email protected]>
block/Makefile.objs +3 -0
index e394fe0b6c..035abb9c5c 100644
@@ -18,6 +18,7 @@ block-obj-y += block-backend.o snapshot.o qapi.o
 block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
+block-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
 block-obj-y += null.o mirror.o commit.o io.o create.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
@@ -65,5 +66,7 @@ block-obj-$(if $(CONFIG_LZFSE),m,n) += dmg-lzfse.o
 dmg-lzfse.o-libs   := $(LZFSE_LIBS)
 qcow.o-libs        := -lz
 linux-aio.o-libs   := -laio
+io_uring.o-cflags  := $(LINUX_IO_URING_CFLAGS)
+io_uring.o-libs    := $(LINUX_IO_URING_LIBS)
 parallels.o-cflags := $(LIBXML2_CFLAGS)
 parallels.o-libs   := $(LIBXML2_LIBS)
include/block/aio.h +15 -1
index 6b0d52f732..7ba9bd7874 100644
@@ -49,6 +49,7 @@ typedef void IOHandler(void *opaque);
 struct Coroutine;
 struct ThreadPool;
 struct LinuxAioState;
+struct LuringState;
 
 struct AioContext {
     GSource source;
@@ -117,11 +118,19 @@ struct AioContext {
     struct ThreadPool *thread_pool;
 
 #ifdef CONFIG_LINUX_AIO
-    /* State for native Linux AIO.  Uses aio_context_acquire/release for
+    /*
+     * State for native Linux AIO.  Uses aio_context_acquire/release for
      * locking.
      */
     struct LinuxAioState *linux_aio;
 #endif
+#ifdef CONFIG_LINUX_IO_URING
+    /*
+     * State for Linux io_uring.  Uses aio_context_acquire/release for
+     * locking.
+     */
+    struct LuringState *linux_io_uring;
+#endif
 
     /* TimerLists for calling timers - one per clock type.  Has its own
      * locking.
@@ -386,6 +395,11 @@ struct LinuxAioState *aio_setup_linux_aio(AioContext *ctx, Error **errp);
 /* Return the LinuxAioState bound to this AioContext */
 struct LinuxAioState *aio_get_linux_aio(AioContext *ctx);
 
+/* Setup the LuringState bound to this AioContext */
+struct LuringState *aio_setup_linux_io_uring(AioContext *ctx, Error **errp);
+
+/* Return the LuringState bound to this AioContext */
+struct LuringState *aio_get_linux_io_uring(AioContext *ctx);
 /**
  * aio_timer_new_with_attrs:
  * @ctx: the aio context
include/block/raw-aio.h +12 -0
index 4629f24d08..251b10d273 100644
@@ -57,6 +57,18 @@ void laio_attach_aio_context(LinuxAioState *s, AioContext *new_context);
 void laio_io_plug(BlockDriverState *bs, LinuxAioState *s);
 void laio_io_unplug(BlockDriverState *bs, LinuxAioState *s);
 #endif
+/* io_uring.c - Linux io_uring implementation */
+#ifdef CONFIG_LINUX_IO_URING
+typedef struct LuringState LuringState;
+LuringState *luring_init(Error **errp);
+void luring_cleanup(LuringState *s);
+int coroutine_fn luring_co_submit(BlockDriverState *bs, LuringState *s, int fd,
+                                uint64_t offset, QEMUIOVector *qiov, int type);
+void luring_detach_aio_context(LuringState *s, AioContext *old_context);
+void luring_attach_aio_context(LuringState *s, AioContext *new_context);
+void luring_io_plug(BlockDriverState *bs, LuringState *s);
+void luring_io_unplug(BlockDriverState *bs, LuringState *s);
+#endif
 
 #ifdef _WIN32
 typedef struct QEMUWin32AIOState QEMUWin32AIOState;
block/io_uring.c new file +403 -0

403 lines not on this page · raw message

Signature · 1 line
2.21.0
#9 lore raw

PATCH v2 5/15 stubs: add stubs for io_uring interface

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 MAINTAINERS         |  1 +
 stubs/Makefile.objs |  1 +
 stubs/io_uring.c    | 32 ++++++++++++++++++++++++++++++++
 3 files changed, 34 insertions(+)
 create mode 100644 stubs/io_uring.c
MAINTAINERS +1 -0
index 31b090033c..c5cf2466b6 100644
@@ -2551,6 +2551,7 @@ M: Stefan Hajnoczi <[email protected]>
 L: [email protected]
 S: Maintained
 F: block/io_uring.c
+F: stubs/io_uring.c
 
 qcow2
 M: Kevin Wolf <[email protected]>
stubs/Makefile.objs +1 -0
index 4a50e95ec3..56c177c507 100644
@@ -13,6 +13,7 @@ stub-obj-y += iothread.o
 stub-obj-y += iothread-lock.o
 stub-obj-y += is-daemonized.o
 stub-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
+stub-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
 stub-obj-y += machine-init-done.o
 stub-obj-y += migr-blocker.o
 stub-obj-y += change-state-handler.o
stubs/io_uring.c new file +32 -0
new file mode 100644
index 0000000000..622d1e4648
@@ -0,0 +1,32 @@
+/*
+ * Linux io_uring support.
+ *
+ * Copyright (C) 2009 IBM, Corp.
+ * Copyright (C) 2009 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "block/aio.h"
+#include "block/raw-aio.h"
+
+void luring_detach_aio_context(LuringState *s, AioContext *old_context)
+{
+    abort();
+}
+
+void luring_attach_aio_context(LuringState *s, AioContext *new_context)
+{
+    abort();
+}
+
+LuringState *luring_init(Error **errp)
+{
+    abort();
+}
+
+void luring_cleanup(LuringState *s)
+{
+    abort();
+}
Signature · 1 line
2.21.0
#10 lore raw

Re: [PATCH v2 05/15] stubs: add stubs for io_uring interface

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 4 lines

From: Aarushi Mehta <[email protected]>

This commit message needs to answer at least where these stubs are actually used. Aren't all callers protected with #ifdef CONFIG_LINUX_IO_URING? (And if they aren't, why is abort() okay?)

Kevin

#11 lore raw

Re: [PATCH v2 05/15] stubs: add stubs for io_uring interface

Quoted 9 lines · Kevin Wolf

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 4 lines

From: Aarushi Mehta <[email protected]>

This commit message needs to answer at least where these stubs are actually used. Aren't all callers protected with #ifdef CONFIG_LINUX_IO_URING? (And if they aren't, why is abort() okay?)

Okay, will look into this.

Stefan

#12 lore raw

Re: [PATCH v2 05/15] stubs: add stubs for io_uring interface

In reply to Kevin Wolf
Quoted 9 lines · Kevin Wolf

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 4 lines

From: Aarushi Mehta <[email protected]>

This commit message needs to answer at least where these stubs are actually used. Aren't all callers protected with #ifdef CONFIG_LINUX_IO_URING? (And if they aren't, why is abort() okay?)

Okay, I'll clarify in the commit description.

I didn't find a program that actually requires these stubs today, but in theory they are required when:
1. A program links against util-obj-y but not block-obj-y (e.g. vhost-user-gpu)
AND
2. It uses util/async.o, which depends on luring_*() APIs

You can test this by adding a call to qemu_bh_new() into vhost-user-gpu's main.c:

ld: libqemuutil.a(async.o): in function `aio_ctx_finalize':
qemu/util/async.c:281: undefined reference to `luring_detach_aio_context'
ld: /home/stefanha/qemu/util/async.c:282: undefined reference to `luring_cleanup'
ld: libqemuutil.a(async.o): in function `aio_setup_linux_io_uring':
qemu/util/async.c:358: undefined reference to `luring_init'
ld: /home/stefanha/qemu/util/async.c:363: undefined reference to `luring_attach_aio_context'

The program may have no intention of using io_uring, just the QEMU main loop and BH, so there is an argument for avoiding linking block-obj-y*. That's the purpose of the stubs and why abort() is okay.

* It's possible to argue against this and personally I'm not that convinced by stubs for this scenario. But io_uring.o should be consistent with how things work today with linux-aio.o. If you feel strongly against having stubs then the linux-aio.o stubs should also be removed (see commit c2b38b277a788).

Stefan

#13 lore raw

Re: [PATCH v2 05/15] stubs: add stubs for io_uring interface

Am 03.12.2019 um 12:16 hat Stefan Hajnoczi geschrieben:

Quoted 30 lines · Kevin Wolf
Quoted 9 lines

Am 25.10.2019 um 18:04 hat Stefan Hajnoczi geschrieben:

Quoted 4 lines

From: Aarushi Mehta <[email protected]>

This commit message needs to answer at least where these stubs are actually used. Aren't all callers protected with #ifdef CONFIG_LINUX_IO_URING? (And if they aren't, why is abort() okay?)

Okay, I'll clarify in the commit description.

I didn't find a program that actually requires these stubs today, but in theory they are required when:
1. A program links against util-obj-y but not block-obj-y (e.g. vhost-user-gpu)
AND
2. It uses util/async.o, which depends on luring_*() APIs

You can test this by adding a call to qemu_bh_new() into vhost-user-gpu's main.c:

ld: libqemuutil.a(async.o): in function `aio_ctx_finalize':
qemu/util/async.c:281: undefined reference to `luring_detach_aio_context'
ld: /home/stefanha/qemu/util/async.c:282: undefined reference to `luring_cleanup'
ld: libqemuutil.a(async.o): in function `aio_setup_linux_io_uring':
qemu/util/async.c:358: undefined reference to `luring_init'
ld: /home/stefanha/qemu/util/async.c:363: undefined reference to `luring_attach_aio_context'

Quoted 3 lines

The program may have no intention of using io_uring, just the QEMU main loop and BH, so there is an argument for avoiding linking block-obj-y*. That's the purpose of the stubs and why abort() is okay.

Okay, make sense to me then.

Quoted 5 lines

* It's possible to argue against this and personally I'm not that convinced by stubs for this scenario. But io_uring.o should be consistent with how things work today with linux-aio.o. If you feel strongly against having stubs then the linux-aio.o stubs should also be removed (see commit c2b38b277a788).

I don't really like having block-specific things like Linux AIO or io_uring in util/async.c, but given that they have per-AioContext state, it's not clearly wrong either.

Maybe we could consider moving these functions to the file-posix driver, it's the only caller of them. But it's nothing that should stop this series.

Kevin

#14 lore raw

Re: [PATCH v2 05/15] stubs: add stubs for io_uring interface

Quoted 8 lines · Kevin Wolf
Quoted 5 lines

* It's possible to argue against this and personally I'm not that convinced by stubs for this scenario. But io_uring.o should be consistent with how things work today with linux-aio.o. If you feel strongly against having stubs then the linux-aio.o stubs should also be removed (see commit c2b38b277a788).

I don't really like having block-specific things like Linux AIO or io_uring in util/async.c, but given that they have per-AioContext state, it's not clearly wrong either.

A good reason to have AioContext in util/async.c is that nowadays Linux supports poll() via the AioContext ring too, and that would allow doing busy polling of network sockets.

Paolo

#15 lore raw

PATCH v2 6/15 util/async: add aio interfaces for io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 util/async.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
util/async.c +36 -0
index ca83e32c7f..c9880e1812 100644
@@ -276,6 +276,14 @@ aio_ctx_finalize(GSource     *source)
     }
 #endif
 
+#ifdef CONFIG_LINUX_IO_URING
+    if (ctx->linux_io_uring) {
+        luring_detach_aio_context(ctx->linux_io_uring, ctx);
+        luring_cleanup(ctx->linux_io_uring);
+        ctx->linux_io_uring = NULL;
+    }
+#endif
+
     assert(QSLIST_EMPTY(&ctx->scheduled_coroutines));
     qemu_bh_delete(ctx->co_schedule_bh);
 
@@ -340,6 +348,29 @@ LinuxAioState *aio_get_linux_aio(AioContext *ctx)
 }
 #endif
 
+#ifdef CONFIG_LINUX_IO_URING
+LuringState *aio_setup_linux_io_uring(AioContext *ctx, Error **errp)
+{
+    if (ctx->linux_io_uring) {
+        return ctx->linux_io_uring;
+    }
+
+    ctx->linux_io_uring = luring_init(errp);
+    if (!ctx->linux_io_uring) {
+        return NULL;
+    }
+
+    luring_attach_aio_context(ctx->linux_io_uring, ctx);
+    return ctx->linux_io_uring;
+}
+
+LuringState *aio_get_linux_io_uring(AioContext *ctx)
+{
+    assert(ctx->linux_io_uring);
+    return ctx->linux_io_uring;
+}
+#endif
+
 void aio_notify(AioContext *ctx)
 {
     /* Write e.g. bh->scheduled before reading ctx->notify_me.  Pairs
@@ -435,6 +466,11 @@ AioContext *aio_context_new(Error **errp)
 #ifdef CONFIG_LINUX_AIO
     ctx->linux_aio = NULL;
 #endif
+
+#ifdef CONFIG_LINUX_IO_URING
+    ctx->linux_io_uring = NULL;
+#endif
+
     ctx->thread_pool = NULL;
     qemu_rec_mutex_init(&ctx->lock);
     timerlistgroup_init(&ctx->tlg, aio_timerlist_notify, ctx);
Signature · 1 line
2.21.0
#16 lore raw

PATCH v2 7/15 blockdev: adds bdrv_parse_aio to use io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 include/block/block.h |  1 +
 block.c               | 22 ++++++++++++++++++++++
 blockdev.c            | 12 ++++--------
 3 files changed, 27 insertions(+), 8 deletions(-)
include/block/block.h +1 -0
index bdb48dcd1b..bf36fa9863 100644
@@ -300,6 +300,7 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
 void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                        Error **errp);
 
+int bdrv_parse_aio(const char *mode, int *flags);
 int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough);
 int bdrv_parse_discard_flags(const char *mode, int *flags);
 BdrvChild *bdrv_open_child(const char *filename,
block.c +22 -0
index dad5a3d8e0..290ab7d2bf 100644
@@ -845,6 +845,28 @@ static BlockdevDetectZeroesOptions bdrv_parse_detect_zeroes(QemuOpts *opts,
     return detect_zeroes;
 }
 
+/**
+ * Set open flags for aio engine
+ *
+ * Return 0 on success, -1 if the engine specified is invalid
+ */
+int bdrv_parse_aio(const char *mode, int *flags)
+{
+    if (!strcmp(mode, "threads")) {
+        /* do nothing, default */
+    } else if (!strcmp(mode, "native")) {
+        *flags |= BDRV_O_NATIVE_AIO;
+#ifdef CONFIG_LINUX_IO_URING
+    } else if (!strcmp(mode, "io_uring")) {
+        *flags |= BDRV_O_IO_URING;
+#endif
+    } else {
+        return -1;
+    }
+
+    return 0;
+}
+
 /**
  * Set open flags for a given discard mode
  *
blockdev.c +4 -8
index 03c7cd7651..6ceef376a6 100644
@@ -385,13 +385,9 @@ static void extract_common_blockdev_options(QemuOpts *opts, int *bdrv_flags,
         }
 
         if ((aio = qemu_opt_get(opts, "aio")) != NULL) {
-            if (!strcmp(aio, "native")) {
-                *bdrv_flags |= BDRV_O_NATIVE_AIO;
-            } else if (!strcmp(aio, "threads")) {
-                /* this is the default */
-            } else {
-               error_setg(errp, "invalid aio option");
-               return;
+            if (bdrv_parse_aio(aio, bdrv_flags) < 0) {
+                error_setg(errp, "invalid aio option");
+                return;
             }
         }
     }
@@ -4642,7 +4638,7 @@ QemuOptsList qemu_common_drive_opts = {
         },{
             .name = "aio",
             .type = QEMU_OPT_STRING,
-            .help = "host AIO implementation (threads, native)",
+            .help = "host AIO implementation (threads, native, io_uring)",
         },{
             .name = BDRV_OPT_CACHE_WB,
             .type = QEMU_OPT_BOOL,
Signature · 1 line
2.21.0
#17 lore raw

PATCH v2 8/15 block/file-posix.c: extend to use io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 block/file-posix.c | 99 ++++++++++++++++++++++++++++++++++++----------
 1 file changed, 79 insertions(+), 20 deletions(-)
block/file-posix.c +79 -20
index 695fcf740d..14c80b8cb3 100644
@@ -156,6 +156,7 @@ typedef struct BDRVRawState {
     bool has_write_zeroes:1;
     bool discard_zeroes:1;
     bool use_linux_aio:1;
+    bool use_linux_io_uring:1;
     bool page_cache_inconsistent:1;
     bool has_fallocate;
     bool needs_alignment;
@@ -444,7 +445,7 @@ static QemuOptsList raw_runtime_opts = {
         {
             .name = "aio",
             .type = QEMU_OPT_STRING,
-            .help = "host AIO implementation (threads, native)",
+            .help = "host AIO implementation (threads, native, io_uring)",
         },
         {
             .name = "locking",
@@ -503,9 +504,15 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
         goto fail;
     }
 
-    aio_default = (bdrv_flags & BDRV_O_NATIVE_AIO)
-                  ? BLOCKDEV_AIO_OPTIONS_NATIVE
-                  : BLOCKDEV_AIO_OPTIONS_THREADS;
+    if (bdrv_flags & BDRV_O_NATIVE_AIO) {
+        aio_default = BLOCKDEV_AIO_OPTIONS_NATIVE;
+#ifdef CONFIG_LINUX_IO_URING
+    } else if (bdrv_flags & BDRV_O_IO_URING) {
+        aio_default = BLOCKDEV_AIO_OPTIONS_IO_URING;
+#endif
+    } else {
+        aio_default = BLOCKDEV_AIO_OPTIONS_THREADS;
+    }
     aio = qapi_enum_parse(&BlockdevAioOptions_lookup,
                           qemu_opt_get(opts, "aio"),
                           aio_default, &local_err);
@@ -514,7 +521,11 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
         ret = -EINVAL;
         goto fail;
     }
+
     s->use_linux_aio = (aio == BLOCKDEV_AIO_OPTIONS_NATIVE);
+#ifdef CONFIG_LINUX_IO_URING
+    s->use_linux_io_uring = (aio == BLOCKDEV_AIO_OPTIONS_IO_URING);
+#endif
 
     locking = qapi_enum_parse(&OnOffAuto_lookup,
                               qemu_opt_get(opts, "locking"),
@@ -578,7 +589,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
     s->shared_perm = BLK_PERM_ALL;
 
 #ifdef CONFIG_LINUX_AIO
-     /* Currently Linux does AIO only for files opened with O_DIRECT */
+    /* Currently Linux does AIO only for files opened with O_DIRECT */
     if (s->use_linux_aio) {
         if (!(s->open_flags & O_DIRECT)) {
             error_setg(errp, "aio=native was specified, but it requires "
@@ -600,6 +611,22 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
     }
 #endif /* !defined(CONFIG_LINUX_AIO) */
 
+#ifdef CONFIG_LINUX_IO_URING
+    if (s->use_linux_io_uring) {
+        if (!aio_setup_linux_io_uring(bdrv_get_aio_context(bs), errp)) {
+            error_prepend(errp, "Unable to use io_uring: ");
+            goto fail;
+        }
+    }
+#else
+    if (s->use_linux_io_uring) {
+        error_setg(errp, "aio=io_uring was specified, but is not supported "
+                         "in this build.");
+        ret = -EINVAL;
+        goto fail;
+    }
+#endif /* !defined(CONFIG_LINUX_IO_URING) */
+
     s->has_discard = true;
     s->has_write_zeroes = true;
     if ((bs->open_flags & BDRV_O_NOCACHE) != 0) {
@@ -1876,21 +1903,25 @@ static int coroutine_fn raw_co_prw(BlockDriverState *bs, uint64_t offset,
         return -EIO;
 
     /*
-     * Check if the underlying device requires requests to be aligned,
-     * and if the request we are trying to submit is aligned or not.
-     * If this is the case tell the low-level driver that it needs
-     * to copy the buffer.
+     * When using O_DIRECT, the request must be aligned to be able to use
+     * either libaio or io_uring interface. If not fail back to regular thread
+     * pool read/write code which emulates this for us if we
+     * set QEMU_AIO_MISALIGNED.
      */
-    if (s->needs_alignment) {
-        if (!bdrv_qiov_is_aligned(bs, qiov)) {
-            type |= QEMU_AIO_MISALIGNED;
+    if (s->needs_alignment && !bdrv_qiov_is_aligned(bs, qiov)) {
+        type |= QEMU_AIO_MISALIGNED;
+#ifdef CONFIG_LINUX_IO_URING
+    } else if (s->use_linux_io_uring) {
+        LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
+        assert(qiov->size == bytes);
+        return luring_co_submit(bs, aio, s->fd, offset, qiov, type);
+#endif
 #ifdef CONFIG_LINUX_AIO
-        } else if (s->use_linux_aio) {
-            LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
-            assert(qiov->size == bytes);
-            return laio_co_submit(bs, aio, s->fd, offset, qiov, type);
+    } else if (s->use_linux_aio) {
+        LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
+        assert(qiov->size == bytes);
+        return laio_co_submit(bs, aio, s->fd, offset, qiov, type);
 #endif
-        }
     }
 
     acb = (RawPosixAIOData) {
@@ -1926,24 +1957,36 @@ static int coroutine_fn raw_co_pwritev(BlockDriverState *bs, uint64_t offset,
 
 static void raw_aio_plug(BlockDriverState *bs)
 {
+    BDRVRawState __attribute__((unused)) *s = bs->opaque;
 #ifdef CONFIG_LINUX_AIO
-    BDRVRawState *s = bs->opaque;
     if (s->use_linux_aio) {
         LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
         laio_io_plug(bs, aio);
     }
 #endif
+#ifdef CONFIG_LINUX_IO_URING
+    if (s->use_linux_io_uring) {
+        LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
+        luring_io_plug(bs, aio);
+    }
+#endif
 }
 
 static void raw_aio_unplug(BlockDriverState *bs)
 {
+    BDRVRawState __attribute__((unused)) *s = bs->opaque;
 #ifdef CONFIG_LINUX_AIO
-    BDRVRawState *s = bs->opaque;
     if (s->use_linux_aio) {
         LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
         laio_io_unplug(bs, aio);
     }
 #endif
+#ifdef CONFIG_LINUX_IO_URING
+    if (s->use_linux_io_uring) {
+        LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
+        luring_io_unplug(bs, aio);
+    }
+#endif
 }
 
 static int raw_co_flush_to_disk(BlockDriverState *bs)
@@ -1963,14 +2006,20 @@ static int raw_co_flush_to_disk(BlockDriverState *bs)
         .aio_type       = QEMU_AIO_FLUSH,
     };
 
+#ifdef CONFIG_LINUX_IO_URING
+    if (s->use_linux_io_uring) {
+        LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
+        return luring_co_submit(bs, aio, s->fd, 0, NULL, QEMU_AIO_FLUSH);
+    }
+#endif
     return raw_thread_pool_submit(bs, handle_aiocb_flush, &acb);
 }
 
 static void raw_aio_attach_aio_context(BlockDriverState *bs,
                                        AioContext *new_context)
 {
+    BDRVRawState __attribute__((unused)) *s = bs->opaque;
 #ifdef CONFIG_LINUX_AIO
-    BDRVRawState *s = bs->opaque;
     if (s->use_linux_aio) {
         Error *local_err;
         if (!aio_setup_linux_aio(new_context, &local_err)) {
@@ -1980,6 +2029,16 @@ static void raw_aio_attach_aio_context(BlockDriverState *bs,
         }
     }
 #endif
+#ifdef CONFIG_LINUX_IO_URING
+    if (s->use_linux_io_uring) {
+        Error *local_err;
+        if (!aio_setup_linux_io_uring(new_context, &local_err)) {
+            error_reportf_err(local_err, "Unable to use linux io_uring, "
+                                         "falling back to thread pool: ");
+            s->use_linux_io_uring = false;
+        }
+    }
+#endif
 }
 
 static void raw_close(BlockDriverState *bs)
Signature · 1 line
2.21.0
#18 lore raw

PATCH v2 9/15 block: add trace events for io_uring

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 block/io_uring.c   | 21 ++++++++++++++++++---
 block/trace-events | 12 ++++++++++++
 2 files changed, 30 insertions(+), 3 deletions(-)
block/io_uring.c +18 -3
index 307c4c5823..a5c0d16220 100644
@@ -17,6 +17,7 @@
 #include "block/raw-aio.h"
 #include "qemu/coroutine.h"
 #include "qapi/error.h"
+#include "trace.h"
 
 /* io_uring ring size */
 #define MAX_ENTRIES 128
@@ -85,6 +86,8 @@ static void luring_resubmit_short_read(LuringState *s, LuringAIOCB *luringcb,
     QEMUIOVector *resubmit_qiov;
     size_t remaining;
 
+    trace_luring_resubmit_short_read(s, luringcb, nread);
+
     /* Update read position */
     luringcb->total_read = nread;
     remaining = luringcb->qiov->size - luringcb->total_read;
@@ -156,6 +159,7 @@ static void luring_process_completions(LuringState *s)
 
         /* Change counters one-by-one because we can be nested. */
         s->io_q.in_flight--;
+        trace_luring_process_completion(s, luringcb, ret);
 
         /* total_read is non-zero only for resubmitted read requests */
         total_bytes = ret + luringcb->total_read;
@@ -224,6 +228,7 @@ static int ioq_submit(LuringState *s)
             QSIMPLEQ_REMOVE_HEAD(&s->io_q.submit_queue, next);
         }
         ret = io_uring_submit(&s->ring);
+        trace_luring_io_uring_submit(s, ret);
         /* Prevent infinite loop if submission is refused */
         if (ret <= 0) {
             if (ret == -EAGAIN) {
@@ -280,12 +285,15 @@ static void ioq_init(LuringQueue *io_q)
 
 void luring_io_plug(BlockDriverState *bs, LuringState *s)
 {
+    trace_luring_io_plug(s);
     s->io_q.plugged++;
 }
 
 void luring_io_unplug(BlockDriverState *bs, LuringState *s)
 {
     assert(s->io_q.plugged);
+    trace_luring_io_unplug(s, s->io_q.blocked, s->io_q.plugged,
+                           s->io_q.in_queue, s->io_q.in_flight);
     if (--s->io_q.plugged == 0 &&
         !s->io_q.blocked && s->io_q.in_queue > 0) {
         ioq_submit(s);
@@ -306,6 +314,7 @@ void luring_io_unplug(BlockDriverState *bs, LuringState *s)
 static int luring_do_submit(int fd, LuringAIOCB *luringcb, LuringState *s,
                             uint64_t offset, int type)
 {
+    int ret;
     struct io_uring_sqe *sqes = &luringcb->sqeq;
 
     switch (type) {
@@ -329,11 +338,14 @@ static int luring_do_submit(int fd, LuringAIOCB *luringcb, LuringState *s,
 
     QSIMPLEQ_INSERT_TAIL(&s->io_q.submit_queue, luringcb, next);
     s->io_q.in_queue++;
-
+    trace_luring_do_submit(s, s->io_q.blocked, s->io_q.plugged,
+                           s->io_q.in_queue, s->io_q.in_flight);
     if (!s->io_q.blocked &&
         (!s->io_q.plugged ||
          s->io_q.in_flight + s->io_q.in_queue >= MAX_ENTRIES)) {
-        return ioq_submit(s);
+        ret = ioq_submit(s);
+        trace_luring_do_submit_done(s, ret);
+        return ret;
     }
     return 0;
 }
@@ -348,8 +360,10 @@ int coroutine_fn luring_co_submit(BlockDriverState *bs, LuringState *s, int fd,
         .qiov       = qiov,
         .is_read    = (type == QEMU_AIO_READ),
     };
-
+    trace_luring_co_submit(bs, s, &luringcb, fd, offset, qiov ? qiov->size : 0,
+                           type);
     ret = luring_do_submit(fd, &luringcb, s, offset, type);
+
     if (ret < 0) {
         return ret;
     }
@@ -400,4 +414,5 @@ void luring_cleanup(LuringState *s)
 {
     io_uring_queue_exit(&s->ring);
     g_free(s);
+    trace_luring_cleanup_state(s);
 }
block/trace-events +12 -0
index b8d70f5242..c8e1647e90 100644
@@ -63,6 +63,18 @@ qmp_block_stream(void *bs) "bs %p"
 file_paio_submit(void *acb, void *opaque, int64_t offset, int count, int type) "acb %p opaque %p offset %"PRId64" count %d type %d"
 file_copy_file_range(void *bs, int src, int64_t src_off, int dst, int64_t dst_off, int64_t bytes, int flags, int64_t ret) "bs %p src_fd %d offset %"PRIu64" dst_fd %d offset %"PRIu64" bytes %"PRIu64" flags %d ret %"PRId64
 
+#io_uring.c
+luring_init_state(void *s, size_t size) "s %p size %zu"
+luring_cleanup_state(void *s) "%p freed"
+luring_io_plug(void *s) "LuringState %p plug"
+luring_io_unplug(void *s, int blocked, int plugged, int queued, int inflight) "LuringState %p blocked %d plugged %d queued %d inflight %d"
+luring_do_submit(void *s, int blocked, int plugged, int queued, int inflight) "LuringState %p blocked %d plugged %d queued %d inflight %d"
+luring_do_submit_done(void *s, int ret) "LuringState %p submitted to kernel %d"
+luring_co_submit(void *bs, void *s, void *luringcb, int fd, uint64_t offset, size_t nbytes, int type) "bs %p s %p luringcb %p fd %d offset %" PRId64 " nbytes %zd type %d"
+luring_process_completion(void *s, void *aiocb, int ret) "LuringState %p luringcb %p ret %d"
+luring_io_uring_submit(void *s, int ret) "LuringState %p ret %d"
+luring_resubmit_short_read(void *s, void *luringcb, int nread) "LuringState %p luringcb %p nread %d"
+
 # qcow2.c
 qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p qiov_offset %zu"
 qcow2_writev_start_req(void *co, int64_t offset, int bytes) "co %p offset 0x%" PRIx64 " bytes %d"
Signature · 1 line
2.21.0
#19 lore raw

PATCH v2 10/15 block/io_uring: adds userspace completion polling

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 block/io_uring.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
block/io_uring.c +16 -1
index a5c0d16220..56892fd1ab 100644
@@ -274,6 +274,21 @@ static void qemu_luring_completion_cb(void *opaque)
     luring_process_completions_and_submit(s);
 }
 
+static bool qemu_luring_poll_cb(void *opaque)
+{
+    LuringState *s = opaque;
+    struct io_uring_cqe *cqes;
+
+    if (io_uring_peek_cqe(&s->ring, &cqes) == 0) {
+        if (cqes) {
+            luring_process_completions_and_submit(s);
+            return true;
+        }
+    }
+
+    return false;
+}
+
 static void ioq_init(LuringQueue *io_q)
 {
     QSIMPLEQ_INIT(&io_q->submit_queue);
@@ -387,7 +402,7 @@ void luring_attach_aio_context(LuringState *s, AioContext *new_context)
     s->aio_context = new_context;
     s->completion_bh = aio_bh_new(new_context, qemu_luring_completion_bh, s);
     aio_set_fd_handler(s->aio_context, s->ring.ring_fd, false,
-                       qemu_luring_completion_cb, NULL, NULL, s);
+                       qemu_luring_completion_cb, NULL, qemu_luring_poll_cb, s);
 }
 
 LuringState *luring_init(Error **errp)
Signature · 1 line
2.21.0
#20 lore raw

PATCH v2 11/15 qemu-io: adds option to use aio engine

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 qemu-io.c | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
qemu-io.c +21 -4
index 91e3276592..3adc5a7d0d 100644
@@ -130,7 +130,8 @@ static void open_help(void)
 " -C, -- use copy-on-read\n"
 " -n, -- disable host cache, short for -t none\n"
 " -U, -- force shared permissions\n"
-" -k, -- use kernel AIO implementation (on Linux only)\n"
+" -k, -- use kernel AIO implementation (Linux only, prefer use of -i)\n"
+" -i, -- use AIO mode (threads, native or io_uring)\n"
 " -t, -- use the given cache mode for the image\n"
 " -d, -- use the given discard mode for the image\n"
 " -o, -- options to be given to the block driver"
@@ -172,7 +173,7 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
     QDict *opts;
     bool force_share = false;
 
-    while ((c = getopt(argc, argv, "snCro:kt:d:U")) != -1) {
+    while ((c = getopt(argc, argv, "snCro:ki:t:d:U")) != -1) {
         switch (c) {
         case 's':
             flags |= BDRV_O_SNAPSHOT;
@@ -204,6 +205,13 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
                 return -EINVAL;
             }
             break;
+        case 'i':
+            if (bdrv_parse_aio(optarg, &flags) < 0) {
+                error_report("Invalid aio option: %s", optarg);
+                qemu_opts_reset(&empty_opts);
+                return -EINVAL;
+            }
+            break;
         case 'o':
             if (imageOpts) {
                 printf("--image-opts and 'open -o' are mutually exclusive\n");
@@ -291,7 +299,9 @@ static void usage(const char *name)
 "  -n, --nocache        disable host cache, short for -t none\n"
 "  -C, --copy-on-read   enable copy-on-read\n"
 "  -m, --misalign       misalign allocations for O_DIRECT\n"
-"  -k, --native-aio     use kernel AIO implementation (on Linux only)\n"
+"  -k, --native-aio     use kernel AIO implementation\n"
+"                       (Linux only, prefer use of -i)\n"
+"  -i, --aio=MODE       use AIO mode (threads, native or io_uring)\n"
 "  -t, --cache=MODE     use the given cache mode for the image\n"
 "  -d, --discard=MODE   use the given discard mode for the image\n"
 "  -T, --trace [[enable=]<pattern>][,events=<file>][,file=<file>]\n"
@@ -496,7 +506,7 @@ static QemuOptsList file_opts = {
 int main(int argc, char **argv)
 {
     int readonly = 0;
-    const char *sopt = "hVc:d:f:rsnCmkt:T:U";
+    const char *sopt = "hVc:d:f:rsnCmki:t:T:U";
     const struct option lopt[] = {
         { "help", no_argument, NULL, 'h' },
         { "version", no_argument, NULL, 'V' },
@@ -508,6 +518,7 @@ int main(int argc, char **argv)
         { "copy-on-read", no_argument, NULL, 'C' },
         { "misalign", no_argument, NULL, 'm' },
         { "native-aio", no_argument, NULL, 'k' },
+        { "aio", required_argument, NULL, 'i' },
         { "discard", required_argument, NULL, 'd' },
         { "cache", required_argument, NULL, 't' },
         { "trace", required_argument, NULL, 'T' },
@@ -575,6 +586,12 @@ int main(int argc, char **argv)
         case 'k':
             flags |= BDRV_O_NATIVE_AIO;
             break;
+        case 'i':
+            if (bdrv_parse_aio(optarg, &flags) < 0) {
+                error_report("Invalid aio option: %s", optarg);
+                exit(1);
+            }
+            break;
         case 't':
             if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) {
                 error_report("Invalid cache option: %s", optarg);
Signature · 1 line
2.21.0
#21 lore raw

PATCH v2 12/15 qemu-img: adds option to use aio engine for benchmarking

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
v10:
* Add missing space to qemu-img command-line documentation

---
 qemu-img.c       | 11 ++++++++++-
 qemu-img-cmds.hx |  4 ++--
 qemu-img.texi    |  5 ++++-
 3 files changed, 16 insertions(+), 4 deletions(-)
qemu-img.c +10 -1
index 8b03ef8171..3e696a4d8e 100644
@@ -4210,7 +4210,8 @@ static int img_bench(int argc, char **argv)
             {"force-share", no_argument, 0, 'U'},
             {0, 0, 0, 0}
         };
-        c = getopt_long(argc, argv, ":hc:d:f:no:qs:S:t:wU", long_options, NULL);
+        c = getopt_long(argc, argv, ":hc:d:f:ni:o:qs:S:t:wU", long_options,
+                        NULL);
         if (c == -1) {
             break;
         }
@@ -4253,6 +4254,14 @@ static int img_bench(int argc, char **argv)
         case 'n':
             flags |= BDRV_O_NATIVE_AIO;
             break;
+        case 'i':
+            ret = bdrv_parse_aio(optarg, &flags);
+            if (ret < 0) {
+                error_report("Invalid aio option: %s", optarg);
+                ret = -1;
+                goto out;
+            }
+            break;
         case 'o':
         {
             offset = cvtnum(optarg);
qemu-img-cmds.hx +2 -2
index 1c93e6d185..77b5a8dda8 100644
@@ -20,9 +20,9 @@ STEXI
 ETEXI
 
 DEF("bench", img_bench,
-    "bench [-c count] [-d depth] [-f fmt] [--flush-interval=flush_interval] [-n] [--no-drain] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] [-U] filename")
+    "bench [-c count] [-d depth] [-f fmt] [--flush-interval=flush_interval] [-n] [--no-drain] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-i aio] [-w] [-U] filename")
 STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-i @var{aio}] [-w] [-U] @var{filename}
 ETEXI
 
 DEF("check", img_check,
qemu-img.texi +4 -1
index b5156d6316..20136fcb94 100644
@@ -206,7 +206,7 @@ Command description:
 Amends the image format specific @var{options} for the image file
 @var{filename}. Not all file formats support this operation.
 
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [-i @var{aio}] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
 
 Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
 specified, a write test is performed, otherwise a read test is performed.
@@ -227,6 +227,9 @@ If @code{-n} is specified, the native AIO backend is used if possible. On
 Linux, this option only works if @code{-t none} or @code{-t directsync} is
 specified as well.
 
+If @code{-i} is specified, aio option can be used to specify different AIO
+backends: @var{threads}, @var{native} or @var{io_uring}.
+
 For write tests, by default a buffer filled with zeros is written. This can be
 overridden with a pattern byte specified by @var{pattern}.
 
Signature · 1 line
2.21.0
#22 lore raw

PATCH v2 13/15 qemu-nbd: adds option for aio engines

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 qemu-nbd.c    | 12 ++++--------
 qemu-nbd.texi |  4 ++--
 2 files changed, 6 insertions(+), 10 deletions(-)
qemu-nbd.c +4 -8
index caacf0ed73..1761b3e70a 100644
@@ -135,7 +135,7 @@ static void usage(const char *name)
 "                            '[ID_OR_NAME]'\n"
 "  -n, --nocache             disable host cache\n"
 "      --cache=MODE          set cache mode (none, writeback, ...)\n"
-"      --aio=MODE            set AIO mode (native or threads)\n"
+"      --aio=MODE            set AIO mode (native, io_uring or threads)\n"
 "      --discard=MODE        set discard mode (ignore, unmap)\n"
 "      --detect-zeroes=MODE  set detect-zeroes mode (off, on, unmap)\n"
 "      --image-opts          treat FILE as a full set of image options\n"
@@ -726,13 +726,9 @@ int main(int argc, char **argv)
                 exit(EXIT_FAILURE);
             }
             seen_aio = true;
-            if (!strcmp(optarg, "native")) {
-                flags |= BDRV_O_NATIVE_AIO;
-            } else if (!strcmp(optarg, "threads")) {
-                /* this is the default */
-            } else {
-               error_report("invalid aio mode `%s'", optarg);
-               exit(EXIT_FAILURE);
+            if (bdrv_parse_aio(optarg, &flags) < 0) {
+                error_report("Invalid aio mode '%s'", optarg);
+                exit(EXIT_FAILURE);
             }
             break;
         case QEMU_NBD_OPT_DISCARD:
qemu-nbd.texi +2 -2
index 7f55657722..3ee3e4bdee 100644
@@ -77,8 +77,8 @@ as an read-only device, @var{snapshot_param} format is
 The cache mode to be used with the file.  See the documentation of
 the emulator's @code{-drive cache=...} option for allowed values.
 @item --aio=@var{aio}
-Set the asynchronous I/O mode between @samp{threads} (the default)
-and @samp{native} (Linux only).
+Set the asynchronous I/O mode between @samp{threads} (the default),
+@samp{native} (Linux only) and @samp{io_uring} (Linux 5.1+).
 @item --discard=@var{discard}
 Control whether @dfn{discard} (also known as @dfn{trim} or @dfn{unmap})
 requests are ignored or passed to the filesystem.  @var{discard} is one of
Signature · 1 line
2.21.0
#23 lore raw

PATCH v2 14/15 tests/qemu-iotests: enable testing with aio options

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
 tests/qemu-iotests/check      | 15 ++++++++++++++-
 tests/qemu-iotests/common.rc  | 14 ++++++++++++++
 tests/qemu-iotests/iotests.py | 12 ++++++++++--
 3 files changed, 38 insertions(+), 3 deletions(-)
tests/qemu-iotests/check +14 -1
index 588c453a94..150fb71cf9 100755
@@ -132,6 +132,7 @@ sortme=false
 expunge=true
 have_test_arg=false
 cachemode=false
+aiomode=false
 
 tmp="${TEST_DIR}"/$$
 rm -f $tmp.list $tmp.tmp $tmp.sed
@@ -141,6 +142,7 @@ export IMGFMT_GENERIC=true
 export IMGPROTO=file
 export IMGOPTS=""
 export CACHEMODE="writeback"
+export AIOMODE="threads"
 export QEMU_IO_OPTIONS=""
 export QEMU_IO_OPTIONS_NO_FMT=""
 export CACHEMODE_IS_DEFAULT=true
@@ -225,6 +227,11 @@ s/ .*//p
         CACHEMODE_IS_DEFAULT=false
         cachemode=false
         continue
+    elif $aiomode
+    then
+        AIOMODE="$r"
+        aiomode=false
+        continue
     fi
 
     xpand=true
@@ -269,6 +276,7 @@ other options
     -n                  show me, do not run tests
     -o options          -o options to pass to qemu-img create/convert
     -c mode             cache mode
+    -i mode             AIO mode
     -makecheck          pretty print output for make check
 
 testlist options
@@ -433,10 +441,13 @@ testlist options
             cachemode=true
             xpand=false
             ;;
+        -i)
+            aiomode=true
+            xpand=false
+            ;;
         -T)        # deprecated timestamp option
             xpand=false
             ;;
-
         -v)
             verbose=true
             xpand=false
@@ -515,6 +526,8 @@ done
 
 # Set qemu-io cache mode with $CACHEMODE we have
 QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --cache $CACHEMODE"
+# Set qemu-io aio mode with $AIOMODE we have
+QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --aio $AIOMODE"
 
 QEMU_IO_OPTIONS_NO_FMT="$QEMU_IO_OPTIONS"
 if [ "$IMGOPTSSYNTAX" != "true" ]; then
tests/qemu-iotests/common.rc +14 -0
index 12b4751848..13f934f79b 100644
@@ -576,6 +576,20 @@ _default_cache_mode()
         return
     fi
 }
+_supported_aio_modes()
+{
+    for mode; do
+        if [ "$mode" = "$AIOMODE" ]; then
+            return
+        fi
+    done
+    _notrun "not suitable for aio mode: $AIOMODE"
+}
+_default_aio_mode()
+{
+    AIOMODE="$1"
+    QEMU_IO="$QEMU_IO --aio $1"
+}
 
 _unsupported_imgopts()
 {
tests/qemu-iotests/iotests.py +10 -2
index 693fde155a..0d4ef6b3e6 100644
@@ -59,6 +59,7 @@ imgproto = os.environ.get('IMGPROTO', 'file')
 test_dir = os.environ.get('TEST_DIR')
 output_dir = os.environ.get('OUTPUT_DIR', '.')
 cachemode = os.environ.get('CACHEMODE')
+aiomode = os.environ.get('AIOMODE')
 qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
@@ -483,6 +484,7 @@ class VM(qtest.QEMUQtestMachine):
             options.append('file=%s' % path)
             options.append('format=%s' % format)
             options.append('cache=%s' % cachemode)
+            options.append('aio=%s' % aiomode)
 
         if opts:
             options.append(opts)
@@ -888,6 +890,10 @@ def verify_cache_mode(supported_cache_modes=[]):
     if supported_cache_modes and (cachemode not in supported_cache_modes):
         notrun('not suitable for this cache mode: %s' % cachemode)
 
+def verify_aio_mode(supported_aio_modes=[]):
+    if supported_aio_modes and (aiomode not in supported_aio_modes):
+        notrun('not suitable for this aio mode: %s' % aiomode)
+
 def supports_quorum():
     return 'quorum' in qemu_img_pipe('--help')
 
@@ -945,8 +951,9 @@ def execute_unittest(output, verbosity, debug):
 
 def execute_test(test_function=None,
                  supported_fmts=[], supported_oses=['linux'],
-                 supported_cache_modes=[], unsupported_fmts=[],
-                 supported_protocols=[], unsupported_protocols=[]):
+                 supported_cache_modes=[], supported_aio_modes={},
+                 unsupported_fmts=[], supported_protocols=[],
+                 unsupported_protocols=[]):
     """Run either unittest or script-style tests."""
 
     # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
@@ -963,6 +970,7 @@ def execute_test(test_function=None,
     verify_protocol(supported_protocols, unsupported_protocols)
     verify_platform(supported_oses)
     verify_cache_mode(supported_cache_modes)
+    verify_aio_mode(supported_aio_modes)
 
     if debug:
         output = sys.stdout
Signature · 1 line
2.21.0
#24 lore raw

PATCH v2 15/15 tests/qemu-iotests: use AIOMODE with various tests

In reply to Stefan Hajnoczi

From: Aarushi Mehta <[email protected]>

---
v11:
* Drop line-wrapping changes that accidentally broke qemu-iotests

---
 tests/qemu-iotests/028 |  2 +-
 tests/qemu-iotests/058 |  2 +-
 tests/qemu-iotests/089 |  4 ++--
 tests/qemu-iotests/091 |  4 ++--
 tests/qemu-iotests/109 |  2 +-
 tests/qemu-iotests/147 |  5 +++--
 tests/qemu-iotests/181 |  8 ++++----
 tests/qemu-iotests/183 |  4 ++--
 tests/qemu-iotests/185 | 10 +++++-----
 tests/qemu-iotests/200 |  2 +-
 tests/qemu-iotests/201 |  8 ++++----
 11 files changed, 26 insertions(+), 25 deletions(-)
tests/qemu-iotests/028 +1 -1
index bba1ee59ae..d4d972c884 100755
@@ -108,7 +108,7 @@ echo block-backup
 echo
 
 qemu_comm_method="monitor"
-_launch_qemu -drive file="${TEST_IMG}",cache=${CACHEMODE},id=disk
+_launch_qemu -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=${AIOMODE},id=disk
 h=$QEMU_HANDLE
 if [ "${VALGRIND_QEMU}" == "y" ]; then
     QEMU_COMM_TIMEOUT=7
tests/qemu-iotests/058 +1 -1
index 8c3212a72f..38d1ed90c0 100755
@@ -64,7 +64,7 @@ nbd_snapshot_img="nbd:unix:$nbd_unix_socket"
 converted_image=$TEST_IMG.converted
 
 # Use -f raw instead of -f $IMGFMT for the NBD connection
-QEMU_IO_NBD="$QEMU_IO -f raw --cache=$CACHEMODE"
+QEMU_IO_NBD="$QEMU_IO -f raw --cache=$CACHEMODE --aio=$AIOMODE"
 
 echo
 echo "== preparing image =="
tests/qemu-iotests/089 +2 -2
index ad029f1f09..059ad75e28 100755
@@ -64,7 +64,7 @@ $QEMU_IO -c 'write -P 42 0 512' -c 'write -P 23 512 512' \
 
 $QEMU_IMG convert -f raw -O $IMGFMT "$TEST_IMG.base" "$TEST_IMG"
 
-$QEMU_IO_PROG --cache $CACHEMODE \
+$QEMU_IO_PROG --cache $CACHEMODE --aio $AIOMODE \
          -c 'read -P 42 0 512' -c 'read -P 23 512 512' \
          -c 'read -P 66 1024 512' "json:{
     \"driver\": \"$IMGFMT\",
@@ -111,7 +111,7 @@ $QEMU_IO -c 'write -P 42 0x38000 512' "$TEST_IMG" | _filter_qemu_io
 
 # The "image.filename" part tests whether "a": { "b": "c" } and "a.b": "c" do
 # the same (which they should).
-$QEMU_IO_PROG --cache $CACHEMODE \
+$QEMU_IO_PROG --cache $CACHEMODE --aio $AIOMODE  \
      -c 'read -P 42 0x38000 512' "json:{
     \"driver\": \"$IMGFMT\",
     \"file\": {
tests/qemu-iotests/091 +2 -2
index f4b44659ae..3a29469324 100755
@@ -60,13 +60,13 @@ echo === Starting QEMU VM1 ===
 echo
 
 qemu_comm_method="monitor"
-_launch_qemu -drive file="${TEST_IMG}",cache=${CACHEMODE},id=disk
+_launch_qemu -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=${AIOMODE},id=disk
 h1=$QEMU_HANDLE
 
 echo
 echo === Starting QEMU VM2 ===
 echo
-_launch_qemu -drive file="${TEST_IMG}",cache=${CACHEMODE},id=disk \
+_launch_qemu -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=${AIOMODE},id=disk \
              -incoming "exec: cat '${MIG_FIFO}'"
 h2=$QEMU_HANDLE
 
tests/qemu-iotests/109 +1 -1
index 9897ceb6cd..3a3dcf94eb 100755
@@ -52,7 +52,7 @@ run_qemu()
     local qmp_format="$3"
     local qmp_event="$4"
 
-    _launch_qemu -drive file="${source_img}",format=raw,cache=${CACHEMODE},id=src
+    _launch_qemu -drive file="${source_img}",format=raw,cache=${CACHEMODE},aio=${AIOMODE},id=src
     _send_qemu_cmd $QEMU_HANDLE "{ 'execute': 'qmp_capabilities' }" "return"
 
     _send_qemu_cmd $QEMU_HANDLE \
tests/qemu-iotests/147 +3 -2
index ab8480b9a4..6bc50ff5d9 100755
@@ -24,7 +24,7 @@ import socket
 import stat
 import time
 import iotests
-from iotests import cachemode, imgfmt, qemu_img, qemu_nbd, qemu_nbd_early_pipe
+from iotests import cachemode, aiomode, imgfmt, qemu_img, qemu_nbd, qemu_nbd_early_pipe
 
 NBD_PORT_START      = 32768
 NBD_PORT_END        = NBD_PORT_START + 1024
@@ -134,7 +134,8 @@ class BuiltinNBD(NBDBlockdevAddBase):
         self.server.add_drive_raw('if=none,id=nbd-export,' +
                                   'file=%s,' % test_img +
                                   'format=%s,' % imgfmt +
-                                  'cache=%s' % cachemode)
+                                  'cache=%s' % cachemode +
+                                  'aio=%s' % aiomode)
         self.server.launch()
 
     def tearDown(self):
tests/qemu-iotests/181 +4 -4
index e317e63422..5b7de72f18 100755
@@ -58,20 +58,20 @@ qemu_comm_method="monitor"
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     _launch_qemu \
-        -drive "${TEST_IMG}",cache=${CACHEMODE},id=disk
+        -drive "${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,id=disk
 else
     _launch_qemu \
-        -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk
+        -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,driver=$IMGFMT,id=disk
 fi
 src=$QEMU_HANDLE
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     _launch_qemu \
-        -drive "${TEST_IMG}",cache=${CACHEMODE},id=disk \
+        -drive "${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,id=disk \
         -incoming "unix:${MIG_SOCKET}"
 else
     _launch_qemu \
-        -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk \
+        -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,driver=$IMGFMT,id=disk \
         -incoming "unix:${MIG_SOCKET}"
 fi
 dest=$QEMU_HANDLE
tests/qemu-iotests/183 +2 -2
index 04fb344d08..1501141f90 100755
@@ -56,12 +56,12 @@ echo
 qemu_comm_method="qmp"
 
 _launch_qemu \
-    -drive file="${TEST_IMG}",cache=$CACHEMODE,driver=$IMGFMT,id=disk
+    -drive file="${TEST_IMG}",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk
 src=$QEMU_HANDLE
 _send_qemu_cmd $src "{ 'execute': 'qmp_capabilities' }" 'return'
 
 _launch_qemu \
-    -drive file="${TEST_IMG}.dest",cache=$CACHEMODE,driver=$IMGFMT,id=disk \
+    -drive file="${TEST_IMG}.dest",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk \
     -incoming "unix:${MIG_SOCKET}"
 dest=$QEMU_HANDLE
 _send_qemu_cmd $dest "{ 'execute': 'qmp_capabilities' }" 'return'
tests/qemu-iotests/185 +5 -5
index 454ff600cc..53aa4e89ed 100755
@@ -54,7 +54,7 @@ echo
 qemu_comm_method="qmp"
 
 _launch_qemu \
-    -drive file="${TEST_IMG}.base",cache=$CACHEMODE,driver=$IMGFMT,id=disk
+    -drive file="${TEST_IMG}.base",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk
 h=$QEMU_HANDLE
 _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 
@@ -125,7 +125,7 @@ echo === Start active commit job and exit qemu ===
 echo
 
 _launch_qemu \
-    -drive file="${TEST_IMG}",cache=$CACHEMODE,driver=$IMGFMT,id=disk
+    -drive file="${TEST_IMG}",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk
 h=$QEMU_HANDLE
 _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 
@@ -147,7 +147,7 @@ echo === Start mirror job and exit qemu ===
 echo
 
 _launch_qemu \
-    -drive file="${TEST_IMG}",cache=$CACHEMODE,driver=$IMGFMT,id=disk
+    -drive file="${TEST_IMG}",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk
 h=$QEMU_HANDLE
 _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 
@@ -172,7 +172,7 @@ echo === Start backup job and exit qemu ===
 echo
 
 _launch_qemu \
-    -drive file="${TEST_IMG}",cache=$CACHEMODE,driver=$IMGFMT,id=disk
+    -drive file="${TEST_IMG}",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk
 h=$QEMU_HANDLE
 _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 
@@ -196,7 +196,7 @@ echo === Start streaming job and exit qemu ===
 echo
 
 _launch_qemu \
-    -drive file="${TEST_IMG}",cache=$CACHEMODE,driver=$IMGFMT,id=disk
+    -drive file="${TEST_IMG}",cache=$CACHEMODE,aio=$AIOMODE,driver=$IMGFMT,id=disk
 h=$QEMU_HANDLE
 _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 
tests/qemu-iotests/200 +1 -1
index 72d431f251..49f93c5df2 100755
@@ -66,7 +66,7 @@ echo === Starting QEMU VM ===
 echo
 qemu_comm_method="qmp"
 _launch_qemu -object iothread,id=iothread0 $virtio_scsi \
-             -drive file="${TEST_IMG}",media=disk,if=none,cache=$CACHEMODE,id=drive_sysdisk,format=$IMGFMT \
+             -drive file="${TEST_IMG}",media=disk,if=none,cache=$CACHEMODE,aio=$AIOMODE,id=drive_sysdisk,format=$IMGFMT \
              -device scsi-hd,drive=drive_sysdisk,bus=scsi0.0,id=sysdisk,bootindex=0
 h1=$QEMU_HANDLE
 
tests/qemu-iotests/201 +4 -4
index 7abf740fe4..448e72dd72 100755
@@ -58,20 +58,20 @@ qemu_comm_method="monitor"
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     _launch_qemu \
-        -drive "${TEST_IMG}",cache=${CACHEMODE},id=disk
+        -drive "${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,id=disk
 else
     _launch_qemu \
-        -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk
+        -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,driver=$IMGFMT,id=disk
 fi
 src=$QEMU_HANDLE
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     _launch_qemu \
-        -drive "${TEST_IMG}",cache=${CACHEMODE},id=disk \
+        -drive "${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,id=disk \
         -incoming "unix:${MIG_SOCKET}"
 else
     _launch_qemu \
-        -drive file="${TEST_IMG}",cache=${CACHEMODE},driver=$IMGFMT,id=disk \
+        -drive file="${TEST_IMG}",cache=${CACHEMODE},aio=$AIOMODE,driver=$IMGFMT,id=disk \
         -incoming "unix:${MIG_SOCKET}"
 fi
 dest=$QEMU_HANDLE
Signature · 1 line
2.21.0
#25 lore raw
In reply to Stefan Hajnoczi
Quoted 90 lines · Stefan Hajnoczi

v11:
* Drop fd registration because it breaks QEMU's file locking and will need to be resolved in a separate patch series
* Drop line-wrapping changes that accidentally broke several qemu-iotests

v10:
* Dropped kernel submission queue polling, it requires root and has additional limitations. It should be benchmarked and considered for inclusion later, maybe even together with kernel side changes.
* Add io_uring_register_files() return value to trace_luring_fd_register()
* Fix indentation in luring_fd_unregister()
* Set s->fd_reg.fd_array to NULL after g_free() to avoid dangling pointers
* Simplify fd registration code
* Add luring_fd_unregister() and call it from file-posix.c to prevent fd leaks
* Add trace_luring_fd_unregister() trace event
* Add missing space to qemu-img command-line documentation
* Update MAINTAINERS file [Julia]
* Rename MAX_EVENTS to MAX_ENTRIES [Julia]
* Define ioq_submit() before callers so the prototype isn't necessary [Julia]
* Declare variables at the beginning of the block in luring_init() [Julia]

This patch series is based on Aarushi Mehta's v9 patch series written for Google Summer of Code 2019:

https://lists.gnu.org/archive/html/qemu-devel/2019-08/msg00179.html

It adds a new AIO engine that uses the new Linux io_uring API. This is the successor to Linux AIO with a number of improvements:
1. Both O_DIRECT and buffered I/O work
2. fdatasync(2) is supported (no need for a separate thread pool!)
3. True async behavior so the syscall doesn't block (Linux AIO got there to some degree...)
4. Advanced performance optimizations are available (file registration, memory buffer registration, completion polling, submission polling).

Since Aarushi has been busy, I have taken up this patch series. Booting a guest works with -drive aio=io_uring and -drive aio=io_uring,cache=none with a raw file on XFS.

I currently recommend using -drive aio=io_uring only with host block devices (like NVMe devices). As of Linux v5.4-rc1 I still hit kernel bugs when using image files on ext4 or XFS.

Aarushi Mehta (15):
configure: permit use of io_uring qapi/block-core: add option for io_uring block/block: add BDRV flag for io_uring block/io_uring: implements interfaces for io_uring
stubs: add stubs for io_uring interface util/async: add aio interfaces for io_uring
blockdev: adds bdrv_parse_aio to use io_uring block/file-posix.c: extend to use io_uring
block: add trace events for io_uring block/io_uring: adds userspace completion polling
qemu-io: adds option to use aio engine
qemu-img: adds option to use aio engine for benchmarking
qemu-nbd: adds option for aio engines tests/qemu-iotests: enable testing with aio options tests/qemu-iotests: use AIOMODE with various tests

 MAINTAINERS                   |   9 +
 qapi/block-core.json          |   4 +-
 configure                     |  27 +++
 block/Makefile.objs           |   3 +
 stubs/Makefile.objs           |   1 +
 include/block/aio.h           |  16 +-
 include/block/block.h         |   2 +
 include/block/raw-aio.h       |  12 +
 block.c                       |  22 ++
 block/file-posix.c            |  99 ++++++--
 block/io_uring.c              | 433 ++++++++++++++++++++++++++++++++++
 blockdev.c                    |  12 +-
 qemu-img.c                    |  11 +-
 qemu-io.c                     |  25 +-
 qemu-nbd.c                    |  12 +-
 stubs/io_uring.c              |  32 +++
 util/async.c                  |  36 +++
 block/trace-events            |  12 +
 qemu-img-cmds.hx              |   4 +-
 qemu-img.texi                 |   5 +-
 qemu-nbd.texi                 |   4 +-
 tests/qemu-iotests/028        |   2 +-
 tests/qemu-iotests/058        |   2 +-
 tests/qemu-iotests/089        |   4 +-
 tests/qemu-iotests/091        |   4 +-
 tests/qemu-iotests/109        |   2 +-
 tests/qemu-iotests/147        |   5 +-
 tests/qemu-iotests/181        |   8 +-
 tests/qemu-iotests/183        |   4 +-
 tests/qemu-iotests/185        |  10 +-
Quoted 8 lines
 tests/qemu-iotests/200        |   2 +-
 tests/qemu-iotests/201        |   8 +-
 tests/qemu-iotests/check      |  15 +-
 tests/qemu-iotests/common.rc  |  14 ++
 tests/qemu-iotests/iotests.py |  12 +-
 35 files changed, 797 insertions(+), 76 deletions(-)
 create mode 100644 block/io_uring.c
 create mode 100644 stubs/io_uring.c

Fixed up commit description as requested by Markus.

Thanks, applied to my block-next tree for QEMU 4.3:
https://github.com/stefanha/qemu/commits/block-next

Stefan

#26 lore raw
Quoted 100 lines · Stefan Hajnoczi
Quoted 90 lines · Stefan Hajnoczi

v11:
* Drop fd registration because it breaks QEMU's file locking and will need to be resolved in a separate patch series
* Drop line-wrapping changes that accidentally broke several qemu-iotests

v10:
* Dropped kernel submission queue polling, it requires root and has additional limitations. It should be benchmarked and considered for inclusion later, maybe even together with kernel side changes.
* Add io_uring_register_files() return value to trace_luring_fd_register()
* Fix indentation in luring_fd_unregister()
* Set s->fd_reg.fd_array to NULL after g_free() to avoid dangling pointers
* Simplify fd registration code
* Add luring_fd_unregister() and call it from file-posix.c to prevent fd leaks
* Add trace_luring_fd_unregister() trace event
* Add missing space to qemu-img command-line documentation
* Update MAINTAINERS file [Julia]
* Rename MAX_EVENTS to MAX_ENTRIES [Julia]
* Define ioq_submit() before callers so the prototype isn't necessary [Julia]
* Declare variables at the beginning of the block in luring_init() [Julia]

This patch series is based on Aarushi Mehta's v9 patch series written for Google Summer of Code 2019:

https://lists.gnu.org/archive/html/qemu-devel/2019-08/msg00179.html

It adds a new AIO engine that uses the new Linux io_uring API. This is the successor to Linux AIO with a number of improvements:
1. Both O_DIRECT and buffered I/O work
2. fdatasync(2) is supported (no need for a separate thread pool!)
3. True async behavior so the syscall doesn't block (Linux AIO got there to some degree...)
4. Advanced performance optimizations are available (file registration, memory buffer registration, completion polling, submission polling).

Since Aarushi has been busy, I have taken up this patch series. Booting a guest works with -drive aio=io_uring and -drive aio=io_uring,cache=none with a raw file on XFS.

I currently recommend using -drive aio=io_uring only with host block devices (like NVMe devices). As of Linux v5.4-rc1 I still hit kernel bugs when using image files on ext4 or XFS.

Aarushi Mehta (15):
configure: permit use of io_uring qapi/block-core: add option for io_uring block/block: add BDRV flag for io_uring block/io_uring: implements interfaces for io_uring
stubs: add stubs for io_uring interface util/async: add aio interfaces for io_uring
blockdev: adds bdrv_parse_aio to use io_uring block/file-posix.c: extend to use io_uring
block: add trace events for io_uring block/io_uring: adds userspace completion polling
qemu-io: adds option to use aio engine
qemu-img: adds option to use aio engine for benchmarking
qemu-nbd: adds option for aio engines tests/qemu-iotests: enable testing with aio options tests/qemu-iotests: use AIOMODE with various tests

 MAINTAINERS                   |   9 +
 qapi/block-core.json          |   4 +-
 configure                     |  27 +++
 block/Makefile.objs           |   3 +
 stubs/Makefile.objs           |   1 +
 include/block/aio.h           |  16 +-
 include/block/block.h         |   2 +
 include/block/raw-aio.h       |  12 +
 block.c                       |  22 ++
 block/file-posix.c            |  99 ++++++--
 block/io_uring.c              | 433 ++++++++++++++++++++++++++++++++++
 blockdev.c                    |  12 +-
 qemu-img.c                    |  11 +-
 qemu-io.c                     |  25 +-
 qemu-nbd.c                    |  12 +-
 stubs/io_uring.c              |  32 +++
 util/async.c                  |  36 +++
 block/trace-events            |  12 +
 qemu-img-cmds.hx              |   4 +-
 qemu-img.texi                 |   5 +-
 qemu-nbd.texi                 |   4 +-
 tests/qemu-iotests/028        |   2 +-
 tests/qemu-iotests/058        |   2 +-
 tests/qemu-iotests/089        |   4 +-
 tests/qemu-iotests/091        |   4 +-
 tests/qemu-iotests/109        |   2 +-
 tests/qemu-iotests/147        |   5 +-
 tests/qemu-iotests/181        |   8 +-
 tests/qemu-iotests/183        |   4 +-
 tests/qemu-iotests/185        |  10 +-
Quoted 8 lines
 tests/qemu-iotests/200        |   2 +-
 tests/qemu-iotests/201        |   8 +-
 tests/qemu-iotests/check      |  15 +-
 tests/qemu-iotests/common.rc  |  14 ++
 tests/qemu-iotests/iotests.py |  12 +-
 35 files changed, 797 insertions(+), 76 deletions(-)
 create mode 100644 block/io_uring.c
 create mode 100644 stubs/io_uring.c
Quoted 4 lines

Fixed up commit description as requested by Markus.

Thanks, applied to my block-next tree for QEMU 4.3:
https://github.com/stefanha/qemu/commits/block-next

Kevin and Max have raised some issues that I will address before this gets merged into qemu.git/master.

Stefan