Skip to content

io_uring: add Linux io_uring AIO engine

Stefan Hajnoczi qemu-devel 5 versions – Applied

Status from this message: Thanks, applied to my block tree:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest15 patches 1 replyApplied
  2. v415 patches 2 repliesApplied
  3. v315 patches 6 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v215 patches11 repliesApplied
v116 patches0 repliesSuperseded

Changes in v5 from the cover letter

* Add back BDRV_O_IO_URING check that was dropped accidentally in v3 [Kevin]

Patches in v5 Applied

#Subject and filesSizeRepliesApplied
1/15configure: permit use of io_uringconfigure+27 -00 replies
2/15qapi/block-core: add option for io_uringqapi/block-core.json+3 -10 replies
3/15block/block: add BDRV flag for io_uringinclude/block/block.h+1 -00 replies
4/15block/io_uring: implements interfaces for io_uringMAINTAINERS block/Makefile.objs block/io_uring.c +2 more+439 -10 replies
5/15stubs: add stubs for io_uring interfaceMAINTAINERS stubs/Makefile.objs stubs/io_uring.c+34 -00 replies
6/15util/async: add aio interfaces for io_uringutil/async.c+36 -00 replies
7/15blockdev: adds bdrv_parse_aio to use io_uringblock.c blockdev.c include/block/block.h+27 -80 replies
8/15block/file-posix.c: extend to use io_uringblock/file-posix.c+79 -190 replies
9/15block: add trace events for io_uringblock/io_uring.c block/trace-events+32 -30 replies
10/15block/io_uring: adds userspace completion pollingblock/io_uring.c+16 -10 replies
11/15qemu-io: adds option to use aio engineqemu-io.c+21 -40 replies
12/15qemu-img: adds option to use aio engine for benchmarkingqemu-img-cmds.hx qemu-img.c qemu-img.texi+16 -40 replies
13/15qemu-nbd: adds option for aio enginesqemu-nbd.c qemu-nbd.texi+6 -100 replies
14/15tests/qemu-iotests: enable testing with aio optionstests/qemu-iotests/check tests/qemu-iotests/common.rc tests/qemu-iotests/iotests.py+38 -30 replies
15/15tests/qemu-iotests: use AIOMODE with various teststests/qemu-iotests/028 tests/qemu-iotests/058 tests/qemu-iotests/089 +8 more+26 -250 replies

Reviews trailers per patch

All 15 patches have a Reviewed-by or Acked-by.

PatchMaxim LevitskyStefano GarzarellaMarkus ArmbrusterEric Blake
1/15R-b A-b
2/15A-b A-b
3/15R-b A-b
4/15A-b
5/15A-b
6/15A-b
7/15A-b
8/15R-b A-b
9/15A-b
10/15A-b
11/15A-b
12/15A-b
13/15A-b A-b
14/15A-b
15/15A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v5 thread

Cover letter [PATCH v5 00/15] io_uring: add Linux io_uring AIO engine

v5:
* Add back BDRV_O_IO_URING check that was dropped accidentally in v3 [Kevin]

v4:
* Drop unnecessary changes in Patch 8 [Stefano]

v3:
* Reword BlockdevAioOptions QAPI schema commit description [Markus]
* Increase QAPI "Since: 4.2" to "Since: 5.0"
* Explain rationale for io_uring stubs in commit description [Kevin]
* Tried to use file.aio=io_uring instead of BDRV_O_IO_URING but it's really hard to make qemu-iotests work. Tests build blkdebug: and other graphs so the syntax for io_uring is dependent on the test case. I scrapped this approach and went back to a global flag.

v2:
* Drop fd registration because it breaks QEMU's file locking and will need to be resolved in a separate patch series
* Drop line-wrapping changes that accidentally broke several qemu-iotests

This patch series is based on Aarushi Mehta's v9 patch series written for Outreachy 2019:

https://lists.gnu.org/archive/html/qemu-devel/2019-08/msg00179.html

It adds a new AIO engine that uses the new Linux io_uring API. This is the successor to Linux AIO with a number of improvements:
1. Both O_DIRECT and buffered I/O work
2. fdatasync(2) is supported (no need for a separate thread pool!)
3. True async behavior so the syscall doesn't block (Linux AIO got there to some degree...)
4. Advanced performance optimizations are available (file registration, memory buffer registration, completion polling, submission polling).

Since Aarushi has been busy, I have taken up this patch series. Booting a guest works with -drive aio=io_uring and -drive aio=io_uring,cache=none with a raw file on XFS.

I currently recommend using -drive aio=io_uring only with host block devices (like NVMe devices). As of Linux v5.4-rc1 I still hit kernel bugs when using image files on ext4 or XFS.

Aarushi Mehta (15):
configure: permit use of io_uring qapi/block-core: add option for io_uring block/block: add BDRV flag for io_uring block/io_uring: implements interfaces for io_uring
stubs: add stubs for io_uring interface util/async: add aio interfaces for io_uring
blockdev: adds bdrv_parse_aio to use io_uring block/file-posix.c: extend to use io_uring
block: add trace events for io_uring block/io_uring: adds userspace completion polling
qemu-io: adds option to use aio engine
qemu-img: adds option to use aio engine for benchmarking
qemu-nbd: adds option for aio engines tests/qemu-iotests: enable testing with aio options tests/qemu-iotests: use AIOMODE with various tests

 MAINTAINERS                   |   9 +
 block.c                       |  22 ++
 block/Makefile.objs           |   3 +
 block/file-posix.c            |  98 ++++++--
 block/io_uring.c              | 433 ++++++++++++++++++++++++++++++++++
 block/trace-events            |  12 +
 blockdev.c                    |  12 +-
 configure                     |  27 +++
 include/block/aio.h           |  16 +-
 include/block/block.h         |   2 +
 include/block/raw-aio.h       |  12 +
 qapi/block-core.json          |   4 +-
 qemu-img-cmds.hx              |   4 +-
 qemu-img.c                    |  11 +-
 qemu-img.texi                 |   5 +-
 qemu-io.c                     |  25 +-
 qemu-nbd.c                    |  12 +-
 qemu-nbd.texi                 |   4 +-
 stubs/Makefile.objs           |   1 +
 stubs/io_uring.c              |  32 +++
 tests/qemu-iotests/028        |   2 +-
 tests/qemu-iotests/058        |   2 +-
 tests/qemu-iotests/089        |   4 +-
 tests/qemu-iotests/091        |   4 +-
 tests/qemu-iotests/109        |   2 +-
 tests/qemu-iotests/147        |   5 +-
 tests/qemu-iotests/181        |   8 +-
 tests/qemu-iotests/183        |   4 +-
 tests/qemu-iotests/185        |  10 +-
 tests/qemu-iotests/200        |   2 +-
 tests/qemu-iotests/201        |   8 +-
 tests/qemu-iotests/check      |  15 +-
 tests/qemu-iotests/common.rc  |  14 ++
 tests/qemu-iotests/iotests.py |  12 +-
 util/async.c                  |  36 +++
 35 files changed, 797 insertions(+), 75 deletions(-)
 create mode 100644 block/io_uring.c
 create mode 100644 stubs/io_uring.c
Signature · 1 line
2.24.1