Skip to content

migration: Postcopy Preempt-Full

Peter Xu qemu-devel 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest15 patches 15 repliesIn review
  2. v114 patches 29 repliesSuperseded

Changes in v2 from the cover letter

- Add r-bs
- Rename release_lock to preempt_active [Dave]
- Create MigrationAtomicStats structure, maintaining the 4 stats fields that may need atomic access since they'll be accessed by multiple threads. define them with Stat64. Add rich comment. [Dave]
- One more patch added (patch 1) to fix another deadlock; not easy to reproduce but actually quite obvious..

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15migration: Take bitmap mutex when completing ram migrationmigration/ram.c+2 -02 replies
2/15migration: Add postcopy_preempt_active()migration/ram.c+7 -21 reply
3/15migration: Cleanup xbzrle zero page cache update logicmigration/ram.c+1 -51 reply
4/15migration: Trivial cleanup save_page_header() on same block checkmigration/ram.c+3 -21 reply
5/15migration: Remove RAMState.f references in compression codemigration/ram.c+7 -51 reply
6/15migration: Yield bitmap_mutex properly when sending/sleepingmigration/ram.c+30 -115 replies
7/15migration: Use atomic ops properly for page accountingsmigration/migration.c migration/multifd.c migration/ram.c +1 more+51 -231 reply
8/15migration: Teach PSS about host pagemigration/ram.c+76 -190 replies
9/15migration: Introduce pss_channelmigration/ram.c+38 -320 replies
10/15migration: Add pss_init()migration/ram.c+9 -30 replies
11/15migration: Make PageSearchStatus part of RAMStatemigration/ram.c+61 -510 replies
12/15migration: Move last_sent_block into PageSearchStatusmigration/ram.c+41 -301 reply
13/15migration: Send requested page directly in rp-return threadmigration/migration.c migration/ram.c+131 -160 replies
14/15migration: Remove old preempt code around state maintainancemigration/migration.c migration/migration.h migration/ram.c+3 -2971 reply
15/15migration: Drop rs->fmigration/ram.c+4 -81 reply

Reviews trailers per patch

All 15 patches have a Reviewed-by or Acked-by.

PatchDr. David Alan GilbertJuan Quintela
1/15R-b R-b
2/15R-b R-b
3/15R-b R-b
4/15R-b R-b
5/15R-b R-b
6/15R-b R-b R-b
7/15R-b R-b
8/15R-b
9/15R-b
10/15R-b
11/15R-b
12/15R-b R-b
13/15R-b
14/15R-b R-b
15/15R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 00/15] migration: Postcopy Preempt-Full

Based-on: <[email protected]>
[PATCH v2 0/5] migration: Bug fixes (prepare for preempt-full)

Tree is here:
https://github.com/xzpeter/qemu/tree/preempt-full

RFC: https://lore.kernel.org/r/[email protected]
v1:  https://lore.kernel.org/r/[email protected]

This patchset is v2 version of preempt-full series.

v2 changelog:
- Add r-bs
- Rename release_lock to preempt_active [Dave]
- Create MigrationAtomicStats structure, maintaining the 4 stats fields that may need atomic access since they'll be accessed by multiple threads. define them with Stat64. Add rich comment. [Dave]
- One more patch added (patch 1) to fix another deadlock; not easy to reproduce but actually quite obvious..

The existing preempt code has reduced ramdom page req latency over 10Gbps network from ~12ms to ~500us which has already landed.

This preempt-full series can further reduces that ~500us to ~230us per my initial test.

Note that no new capability is needed, IOW it's fully compatible with the existing preempt mode. So the naming is actually not important but just to identify the difference on the binaries.

The logic of the series is simple: send urgent pages in rp-return thread rather than migration thread. It also means rp-thread will take over the ownership of the newly created preempt channel. It can slow down rp-return thread on receiving page requests, but I don't really see a major issue with it so far but only benefits.

For detailed performance numbers, please refer to the rfc cover letter.

Please have a look, thanks.

Peter Xu (15):
migration: Take bitmap mutex when completing ram migration
migration: Add postcopy_preempt_active()
migration: Cleanup xbzrle zero page cache update logic
migration: Trivial cleanup save_page_header() on same block check
migration: Remove RAMState.f references in compression code
migration: Yield bitmap_mutex properly when sending/sleeping
migration: Use atomic ops properly for page accountings
migration: Teach PSS about host page
migration: Introduce pss_channel
migration: Add pss_init()
migration: Make PageSearchStatus part of RAMState
migration: Move last_sent_block into PageSearchStatus
migration: Send requested page directly in rp-return thread
migration: Remove old preempt code around state maintainance
migration: Drop rs->f

 migration/migration.c |  47 +--
 migration/migration.h |   7 -
 migration/multifd.c   |   4 +-
 migration/ram.c       | 720 +++++++++++++++++++-----------------------
 migration/ram.h       |  20 ++
 5 files changed, 379 insertions(+), 419 deletions(-)
Signature · 1 line
2.37.3