Skip to content

migration/mapped-ram: Add direct-io support

Fabiano Rosas qemu-devel 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest16 patches 7 repliesIn review
  2. v218 patches 41 repliesSuperseded
  3. v19 patches 47 repliesSuperseded

v2 from the cover letter

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/16migration: Drop reference to QIOChannel if file seeking failsmigration/file.c+1 -01 reply
2/16migration: Fix file migration with fdsetmigration/file.c+9 -22 replies
3/16tests/qtest/migration: Fix file migration offset checktests/qtest/migration-test.c+48 -310 replies
4/16tests/qtest/migration: Add a precopy file test with fdsettests/qtest/migration-test.c+44 -00 replies
5/16monitor: Drop monitor_fdset_dup_fd_find/_remove()include/monitor/monitor.h monitor/fds.c stubs/fdset.c +1 more+7 -420 replies
6/16monitor: Introduce monitor_fdset_*freemonitor/fds.c+23 -80 replies
7/16monitor: Stop removing non-duplicated fdsmonitor/fds.c monitor/hmp.c monitor/monitor-internal.h +2 more+8 -130 replies
8/16monitor: Simplify fdset and fd removalmonitor/fds.c+6 -210 replies
9/16monitor: Report errors from monitor_fdset_dup_fd_addinclude/monitor/monitor.h monitor/fds.c stubs/fdset.c +1 more+12 -120 replies
10/16io: Stop using qemu_open_old in channel-fileio/channel-file.c+5 -30 replies
11/16migration: Add direct-io parameterinclude/qemu/osdep.h migration/migration-hmp-cmds.c migration/options.c +3 more+76 -30 replies
12/16migration/multifd: Add direct-io supportmigration/file.c migration/file.h migration/migration.c+51 -61 reply
13/16tests/qtest/migration: Add tests for file migration with direct-iotests/qtest/migration-helpers.c tests/qtest/migration-helpers.h tests/qtest/migration-test.c+114 -00 replies
14/16monitor: fdset: Match against O_DIRECTmonitor/fds.c+6 -10 replies
15/16migration: Add documentation for fdset with multifd + filedocs/devel/migration/main.rst docs/devel/migration/mapped-ram.rst+24 -60 replies
16/16tests/qtest/migration: Add a test for mapped-ram with passing of fdstests/qtest/migration-test.c+95 -33 replies

Reviews trailers per patch

All 16 patches have a Reviewed-by or Acked-by.

PatchPrasad PanditPeter XuDaniel P. BerrangéMarkus Armbruster
1/16R-b
2/16R-b R-b R-b
3/16R-b R-b
4/16R-b
5/16R-b
6/16R-b
7/16R-b
8/16R-b
9/16R-b
10/16R-b R-b
11/16R-b A-b
12/16R-b
13/16R-b
14/16R-b
15/16R-b
16/16R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v3 thread

Cover letter [PATCH v3 00/16] migration/mapped-ram: Add direct-io support

Hi,

Not many changes since v2:

- new patch 1: drop IOC reference on offset check failure

- dropped 2 patches adding direct-io to the single threaded migration. We have agreed to use multifd with 1 channel for that use-case.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/1335719164

Thanks

v2:
https://lore.kernel.org/r/[email protected]

v1:
https://lore.kernel.org/r/[email protected]

Hi everyone, here's the rest of the migration "mapped-ram" feature that didn't get merged for 9.0. This series adds support for direct I/O, the missing piece to get the desired performance improvements.

There's 3 parts to this:

1- The plumbing for the new "direct-io" migration parameter. With this
we can already use direct-io with the file transport + multifd + mapped-ram. Patches 1-3.

Due to the alignment requirements of O_DIRECT and the fact that multifd runs the channels in parallel with the migration thread, we must open the migration file two times, one with O_DIRECT set and another with it clear.

If the user is not passing in a file name which QEMU can open at will, we must then require that the user pass the two file descriptors with the flags already properly set. We'll use the already existing fdset + QMP add-fd infrastructure for this.

2- Changes to the fdset infrastructure to support O_DIRECT. We need
those to be able to select from the user-provided fdset the file descriptor that contains the O_DIRECT flag. Patches 4-5.

3- Some fdset validation to make sure the two-fds requirement is being
met. Patches 6-7.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/1269352083

Fabiano Rosas (15):
migration: Drop reference to QIOChannel if file seeking fails
migration: Fix file migration with fdset tests/qtest/migration: Fix file migration offset check tests/qtest/migration: Add a precopy file test with fdset
monitor: Introduce monitor_fdset_*free
monitor: Stop removing non-duplicated fds
monitor: Simplify fdset and fd removal
monitor: Report errors from monitor_fdset_dup_fd_add
io: Stop using qemu_open_old in channel-file
migration: Add direct-io parameter
migration/multifd: Add direct-io support tests/qtest/migration: Add tests for file migration with direct-io
monitor: fdset: Match against O_DIRECT
migration: Add documentation for fdset with multifd + file tests/qtest/migration: Add a test for mapped-ram with passing of fds

Peter Xu (1):
monitor: Drop monitor_fdset_dup_fd_find/_remove()

 docs/devel/migration/main.rst       |  24 ++-
 docs/devel/migration/mapped-ram.rst |   6 +-
 include/monitor/monitor.h           |   3 +-
 include/qemu/osdep.h                |   2 +
 io/channel-file.c                   |   8 +-
 migration/file.c                    |  45 ++++-
 migration/file.h                    |   1 -
 migration/migration-hmp-cmds.c      |  11 ++
 migration/migration.c               |  23 +++
 migration/options.c                 |  35 ++++
 migration/options.h                 |   1 +
 monitor/fds.c                       |  96 +++++-----
 monitor/hmp.c                       |   2 -
 monitor/monitor-internal.h          |   1 -
 monitor/monitor.c                   |   1 -
 monitor/qmp.c                       |   2 -
 qapi/migration.json                 |  21 ++-
 stubs/fdset.c                       |   7 +-
 tests/qtest/migration-helpers.c     |  44 +++++
 tests/qtest/migration-helpers.h     |   8 +
 tests/qtest/migration-test.c        | 263 +++++++++++++++++++++++++---
 util/osdep.c                        |  34 ++--
 22 files changed, 508 insertions(+), 130 deletions(-)

base-commit: 05ad1440b8428b0ade9b8e5c01469adb8fbf83e3

Signature · 1 line
2.35.3