Migration 20211031 patches
The following changes since commit af531756d25541a1b3b3d9a14e72e7fedd941a2e:
Merge remote-tracking branch 'remotes/philmd/tags/renesas-20211030' into staging (2021-10-30 11:31:41 -0700)
are available in the Git repository at:
https://github.com/juanquintela/qemu.git tags/migration-20211031-pull-request
for you to fetch changes up to 826b8bc80cb191557a4ce7cf0e155b436d2d1afa:
migration/dirtyrate: implement dirty-bitmap dirtyrate calculation (2021-11-01 22:56:44 +0100)
----------------------------------------------------------------
Migration Pull request
Hi
this includes pending bits of migration patches.
- virtio-mem support by David Hildenbrand
- dirtyrate improvements by Hyman Huang
- fix rdma wrid by Li Zhijian
- dump-guest-memory fixes by Peter Xu
Pleas apply.
Thanks, Juan.
----------------------------------------------------------------
David Hildenbrand (8):
memory: Introduce replay_discarded callback for RamDiscardManager
virtio-mem: Implement replay_discarded RamDiscardManager callback migration/ram: Handle RAMBlocks with a RamDiscardManager on the
migration source
virtio-mem: Drop precopy notifier
migration/postcopy: Handle RAMBlocks with a RamDiscardManager on the
destination
migration: Simplify alignment and alignment checks migration/ram: Factor out populating pages readable in
ram_block_populate_pages()
migration/ram: Handle RAMBlocks with a RamDiscardManager on background
snapshots
Hyman Huang(黄勇) (6):
KVM: introduce dirty_pages and kvm_dirty_ring_enabled
memory: make global_dirty_tracking a bitmask migration/dirtyrate: introduce struct and adjust DirtyRateStat migration/dirtyrate: adjust order of registering thread migration/dirtyrate: move init step of calculation to main thread migration/dirtyrate: implement dirty-ring dirtyrate calculation
Hyman Huang(黄勇) (2):
memory: introduce total_dirty_pages to stat dirty pages migration/dirtyrate: implement dirty-bitmap dirtyrate calculation
Li Zhijian (1):
migration/rdma: Fix out of order wrid
Peter Xu (3):
migration: Make migration blocker work for snapshots too
migration: Add migrate_add_blocker_internal()
dump-guest-memory: Block live migration
qapi/migration.json | 48 ++++- include/exec/memory.h | 41 +++- include/exec/ram_addr.h | 13 +- include/hw/core/cpu.h | 1 + include/hw/virtio/virtio-mem.h | 3 - include/migration/blocker.h | 16 ++ include/sysemu/kvm.h | 1 + migration/dirtyrate.h | 21 +- migration/ram.h | 1 + accel/kvm/kvm-all.c | 7 + accel/stubs/kvm-stub.c | 5 + dump/dump.c | 19 ++ hw/i386/xen/xen-hvm.c | 4 +- hw/virtio/virtio-mem.c | 92 ++++++--- migration/dirtyrate.c | 367 ++++++++++++++++++++++++++++++--- migration/migration.c | 30 +-- migration/postcopy-ram.c | 40 +++- migration/ram.c | 180 ++++++++++++++-- migration/rdma.c | 138 +++++++++---- softmmu/memory.c | 43 +++- hmp-commands.hx | 8 +- migration/trace-events | 2 + softmmu/trace-events | 1 + 23 files changed, 909 insertions(+), 172 deletions(-)
Signature · 1 line
2.33.1
[PULL 01/20] migration/rdma: Fix out of order wrid
From: Li Zhijian <[email protected]>
destination:
../qemu/build/qemu-system-x86_64 -enable-kvm -netdev tap,id=hn0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device e1000,netdev=hn0,mac=50:52:54:00:11:22 -boot c -drive if=none,file=./Fedora-rdma-server-migration.qcow2,id=drive-virtio-disk0 -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -m 2048 -smp 2 -device piix3-usb-uhci -device usb-tablet -monitor stdio -vga qxl -spice streaming-video=filter,port=5902,disable-ticketing -incoming rdma:192.168.22.23:8888
qemu-system-x86_64: -spice streaming-video=filter,port=5902,disable-ticketing: warning: short-form boolean option 'disable-ticketing' deprecated
Please use disable-ticketing=on instead
QEMU 6.0.50 monitor - type 'help' for more information
(qemu) trace-event qemu_rdma_block_for_wrid_miss on
(qemu) dest_init RDMA Device opened: kernel name rxe_eth0 uverbs device name uverbs2, infiniband_verbs class device path /sys/class/infiniband_verbs/uverbs2, infiniband class device path /sys/class/infiniband/rxe_eth0, transport: (2) Ethernetqemu_rdma_block_for_wrid_miss A Wanted wrid CONTROL SEND (2000) but got CONTROL RECV (4000)
source:
../qemu/build/qemu-system-x86_64 -enable-kvm -netdev tap,id=hn0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device e1000,netdev=hn0,mac=50:52:54:00:11:22 -boot c -drive if=none,file=./Fedora-rdma-server.qcow2,id=drive-virtio-disk0 -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -m 2048 -smp 2 -device piix3-usb-uhci -device usb-tablet -monitor stdio -vga qxl -spice streaming-video=filter,port=5901,disable-ticketing -S
qemu-system-x86_64: -spice streaming-video=filter,port=5901,disable-ticketing: warning: short-form boolean option 'disable-ticketing' deprecated
Please use disable-ticketing=on instead
QEMU 6.0.50 monitor - type 'help' for more information
(qemu)
(qemu) trace-event qemu_rdma_block_for_wrid_miss on
(qemu) migrate -d rdma:192.168.22.23:8888source_resolve_host RDMA Device opened: kernel name rxe_eth0 uverbs device name uverbs2, infiniband_verbs class device path /sys/class/infiniband_verbs/uverbs2, infiniband class device path /sys/class/infiniband/rxe_eth0, transport: (2) Ethernet
(qemu) qemu_rdma_block_for_wrid_miss A Wanted wrid WRITE RDMA (1) but got CONTROL RECV (4000)
NOTE: we use soft RoCE as the rdma device. [root@iaas-rpma images]# rdma link show rxe_eth0/1 link rxe_eth0/1 state ACTIVE physical_state LINK_UP netdev eth0
This migration could not be completed when out of order(OOO) CQ event occurs. The send queue and receive queue shared a same completion queue, and qemu_rdma_block_for_wrid() will drop the CQs it's not interested in. But the dropped CQs by qemu_rdma_block_for_wrid() could be later CQs it wants. So in this case, qemu_rdma_block_for_wrid() will block forever.
OOO cases will occur in both source side and destination side. And a forever blocking happens on only SEND and RECV are out of order. OOO between 'WRITE RDMA' and 'RECV' doesn't matter.
below the OOO sequence:
source destination
rdma_write_one() qemu_rdma_registration_handle()
1. S1: post_recv X D1: post_recv Y
2. wait for recv CQ event X
3. D2: post_send X ---------------+
4. wait for send CQ send event X (D2) | 5. recv CQ event X reaches (D2) | 6. +-S2: post_send Y | 7. | wait for send CQ event Y | 8. | recv CQ event Y (S2) (drop it) | 9. +-send CQ event Y reaches (S2) |
10. send CQ event X reaches (D2) -----+
11. wait recv CQ event Y (dropped by (8))
Although a hardware IB works fine in my a hundred of runs, the IB specification doesn't guaratee the CQ order in such case.
Here we introduce a independent send completion queue to distinguish ibv_post_send completion queue from the original mixed completion queue. It helps us to poll the specific CQE we are really interested in.
--- migration/rdma.c | 138 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 101 insertions(+), 37 deletions(-)
migration/rdma.c +101 -37
@@ -358,9 +358,11 @@ typedef struct RDMAContext {struct ibv_context *verbs;struct rdma_event_channel *channel;struct ibv_qp *qp;/* queue pair */ -struct ibv_comp_channel *comp_channel;/* completion channel */ +struct ibv_comp_channel *recv_comp_channel;/* recv completion channel */ +struct ibv_comp_channel *send_comp_channel;/* send completion channel */ struct ibv_pd *pd;/* protection domain */ -struct ibv_cq *cq;/* completion queue */ +struct ibv_cq *recv_cq;/* recvieve completion queue */ +struct ibv_cq *send_cq;/* send completion queue */ /* * If a previous write failed (perhaps because of a failed
@@ -1059,21 +1061,34 @@ static int qemu_rdma_alloc_pd_cq(RDMAContext *rdma)return -1 ; } -/* create completion channel */ - rdma->comp_channel =ibv_create_comp_channel (rdma->verbs); -if (!rdma->comp_channel) { -error_report ("failed to allocate completion channel" ); +/* create receive completion channel */ + rdma->recv_comp_channel =ibv_create_comp_channel (rdma->verbs); +if (!rdma->recv_comp_channel) { +error_report ("failed to allocate receive completion channel" ); goto err_alloc_pd_cq; }/* -* Completion queue can be filled by both read and write work requests, -* so must reflect the sum of both possible queue sizes. +* Completion queue can be filled by read work requests. */ - rdma->cq =ibv_create_cq (rdma->verbs, (RDMA_SIGNALED_SEND_MAX *3 ), -NULL , rdma->comp_channel,0 ); -if (!rdma->cq) { -error_report ("failed to allocate completion queue" ); + rdma->recv_cq =ibv_create_cq (rdma->verbs, (RDMA_SIGNALED_SEND_MAX *3 ), +NULL , rdma->recv_comp_channel,0 ); +if (!rdma->recv_cq) { +error_report ("failed to allocate receive completion queue" ); + goto err_alloc_pd_cq; + } + +/* create send completion channel */ + rdma->send_comp_channel =ibv_create_comp_channel (rdma->verbs); +if (!rdma->send_comp_channel) { +error_report ("failed to allocate send completion channel" ); + goto err_alloc_pd_cq; + } + + rdma->send_cq =ibv_create_cq (rdma->verbs, (RDMA_SIGNALED_SEND_MAX *3 ), +NULL , rdma->send_comp_channel,0 ); +if (!rdma->send_cq) { +error_report ("failed to allocate send completion queue" ); goto err_alloc_pd_cq; }
@@ -1083,11 +1098,19 @@ err_alloc_pd_cq:if (rdma->pd) {ibv_dealloc_pd (rdma->pd); } -if (rdma->comp_channel) { -ibv_destroy_comp_channel (rdma->comp_channel); +if (rdma->recv_comp_channel) { +ibv_destroy_comp_channel (rdma->recv_comp_channel); + } +if (rdma->send_comp_channel) { +ibv_destroy_comp_channel (rdma->send_comp_channel); + } +if (rdma->recv_cq) { +ibv_destroy_cq (rdma->recv_cq); + rdma->recv_cq =NULL ; } rdma->pd =NULL ; - rdma->comp_channel =NULL ; + rdma->recv_comp_channel =NULL ; + rdma->send_comp_channel =NULL ;return -1 ; }
@@ -1104,8 +1127,8 @@ static int qemu_rdma_alloc_qp(RDMAContext *rdma) attr.cap.max_recv_wr =3 ; attr.cap.max_send_sge =1 ; attr.cap.max_recv_sge =1 ; - attr.send_cq = rdma->cq; - attr.recv_cq = rdma->cq; + attr.send_cq = rdma->send_cq; + attr.recv_cq = rdma->recv_cq; attr.qp_type =IBV_QPT_RC ; ret =rdma_create_qp (rdma->cm_id, rdma->pd, &attr);
@@ -1496,14 +1519,14 @@ static void qemu_rdma_signal_unregister(RDMAContext *rdma, uint64_t index,* (of any kind) has completed. * Return the work request ID that completed. */ -static uint64_t qemu_rdma_poll (RDMAContext *rdma,uint64_t *wr_id_out, -uint32_t *byte_len) +static uint64_t qemu_rdma_poll (RDMAContext *rdma,struct ibv_cq *cq, +uint64_t *wr_id_out,uint32_t *byte_len) {int ret;struct ibv_wc wc;uint64_t wr_id; - ret =ibv_poll_cq (rdma->cq,1 , &wc); + ret =ibv_poll_cq (cq,1 , &wc);if (!ret) { *wr_id_out =RDMA_WRID_NONE ;
@@ -1575,7 +1598,8 @@ static uint64_t qemu_rdma_poll(RDMAContext *rdma, uint64_t *wr_id_out,/* Wait for activity on the completion channel. * Returns 0 on success, none-0 on error. */ -static int qemu_rdma_wait_comp_channel (RDMAContext *rdma) +static int qemu_rdma_wait_comp_channel (RDMAContext *rdma, +struct ibv_comp_channel *comp_channel) {struct rdma_cm_event *cm_event;int ret =-1 ;
@@ -1586,7 +1610,7 @@ static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)*/ if (rdma->migration_started_on_destination &&migration_incoming_get_current ()->state ==MIGRATION_STATUS_ACTIVE ) { -yield_until_fd_readable (rdma->comp_channel->fd); +yield_until_fd_readable (comp_channel->fd); }else {/* This is the source side, we're in a separate thread * or destination prior to migration_fd_process_incoming()
@@ -1597,7 +1621,7 @@ static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)*/ while (!rdma->error_state && !rdma->received_error) {GPollFD pfds[2 ]; - pfds[0 ].fd = rdma->comp_channel->fd; + pfds[0 ].fd = comp_channel->fd; pfds[0 ].events =G_IO_IN |G_IO_HUP |G_IO_ERR ; pfds[0 ].revents =0 ;
@@ -1655,6 +1679,17 @@ static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)return rdma->error_state; } +static struct ibv_comp_channel *to_channel (RDMAContext *rdma,int wrid) +{ +return wrid <RDMA_WRID_RECV_CONTROL ? rdma->send_comp_channel : + rdma->recv_comp_channel; +} + +static struct ibv_cq *to_cq (RDMAContext *rdma,int wrid) +{ +return wrid <RDMA_WRID_RECV_CONTROL ? rdma->send_cq : rdma->recv_cq; +} +/* * Block until the next work request has completed. *
@@ -1675,13 +1710,15 @@ static int qemu_rdma_block_for_wrid(RDMAContext *rdma, int wrid_requested,struct ibv_cq *cq;void *cq_ctx;uint64_t wr_id =RDMA_WRID_NONE , wr_id_in; +struct ibv_comp_channel *ch =to_channel (rdma, wrid_requested); +struct ibv_cq *poll_cq =to_cq (rdma, wrid_requested); -if (ibv_req_notify_cq (rdma->cq,0 )) { +if (ibv_req_notify_cq (poll_cq,0 )) {return -1 ; }/* poll cq first */ while (wr_id != wrid_requested) { - ret =qemu_rdma_poll (rdma, &wr_id_in, byte_len); + ret =qemu_rdma_poll (rdma, poll_cq, &wr_id_in, byte_len);if (ret <0 ) {return ret; }
@@ -1702,12 +1739,12 @@ static int qemu_rdma_block_for_wrid(RDMAContext *rdma, int wrid_requested, }while (1 ) { - ret =qemu_rdma_wait_comp_channel (rdma); + ret =qemu_rdma_wait_comp_channel (rdma, ch);if (ret) { goto err_block_for_wrid; } - ret =ibv_get_cq_event (rdma->comp_channel, &cq, &cq_ctx); + ret =ibv_get_cq_event (ch, &cq, &cq_ctx);if (ret) {perror ("ibv_get_cq_event" ); goto err_block_for_wrid;
@@ -1721,7 +1758,7 @@ static int qemu_rdma_block_for_wrid(RDMAContext *rdma, int wrid_requested, }while (wr_id != wrid_requested) { - ret =qemu_rdma_poll (rdma, &wr_id_in, byte_len); + ret =qemu_rdma_poll (rdma, poll_cq, &wr_id_in, byte_len);if (ret <0 ) { goto err_block_for_wrid; }
@@ -2437,13 +2474,21 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)rdma_destroy_qp (rdma->cm_id); rdma->qp =NULL ; } -if (rdma->cq) { -ibv_destroy_cq (rdma->cq); - rdma->cq =NULL ; +if (rdma->recv_cq) { +ibv_destroy_cq (rdma->recv_cq); + rdma->recv_cq =NULL ; } -if (rdma->comp_channel) { -ibv_destroy_comp_channel (rdma->comp_channel); - rdma->comp_channel =NULL ; +if (rdma->send_cq) { +ibv_destroy_cq (rdma->send_cq); + rdma->send_cq =NULL ; + } +if (rdma->recv_comp_channel) { +ibv_destroy_comp_channel (rdma->recv_comp_channel); + rdma->recv_comp_channel =NULL ; + } +if (rdma->send_comp_channel) { +ibv_destroy_comp_channel (rdma->send_comp_channel); + rdma->send_comp_channel =NULL ; }if (rdma->pd) {ibv_dealloc_pd (rdma->pd);
@@ -3115,10 +3160,14 @@ static void qio_channel_rdma_set_aio_fd_handler(QIOChannel *ioc, {QIOChannelRDMA *rioc =QIO_CHANNEL_RDMA (ioc);if (io_read) { -aio_set_fd_handler (ctx, rioc->rdmain->comp_channel->fd, +aio_set_fd_handler (ctx, rioc->rdmain->recv_comp_channel->fd, + false, io_read, io_write,NULL , opaque); +aio_set_fd_handler (ctx, rioc->rdmain->send_comp_channel->fd, false, io_read, io_write,NULL , opaque); }else { -aio_set_fd_handler (ctx, rioc->rdmaout->comp_channel->fd, +aio_set_fd_handler (ctx, rioc->rdmaout->recv_comp_channel->fd, + false, io_read, io_write,NULL , opaque); +aio_set_fd_handler (ctx, rioc->rdmaout->send_comp_channel->fd, false, io_read, io_write,NULL , opaque); } }
@@ -3332,7 +3381,22 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void *opaque,*/ while (1 ) {uint64_t wr_id, wr_id_in; -int ret =qemu_rdma_poll (rdma, &wr_id_in,NULL ); +int ret =qemu_rdma_poll (rdma, rdma->recv_cq, &wr_id_in,NULL ); +if (ret <0 ) { +error_report ("rdma migration: polling error! %d" , ret); + goto err; + } + + wr_id = wr_id_in &RDMA_WRID_TYPE_MASK ; + +if (wr_id ==RDMA_WRID_NONE ) { +break ; + } + } + +while (1 ) { +uint64_t wr_id, wr_id_in; +int ret =qemu_rdma_poll (rdma, rdma->send_cq, &wr_id_in,NULL );if (ret <0 ) {error_report ("rdma migration: polling error! %d" , ret); goto err;
Signature · 1 line
2.33.1
[PULL 02/20] KVM: introduce dirty_pages and kvm_dirty_ring_enabled
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
dirty_pages is used to calculate dirtyrate via dirty ring, when enabled, kvm-reaper will increase the dirty pages after gfns being dirtied.
kvm_dirty_ring_enabled shows if kvm-reaper is working. dirtyrate thread could use it to check if measurement can base on dirty ring feature.
Message-Id: <fee5fb2ab17ec2159405fc54a3cff8e02322f816.1624040308.git.huangy81@chinatelecom.cn>
--- include/hw/core/cpu.h | 1 + include/sysemu/kvm.h | 1 + accel/kvm/kvm-all.c | 7 +++++++ accel/stubs/kvm-stub.c | 5 +++++ 4 files changed, 14 insertions(+)
include/hw/core/cpu.h +1 -0
@@ -381,6 +381,7 @@ struct CPUState {struct kvm_run *kvm_run;struct kvm_dirty_gfn *kvm_dirty_gfns;uint32_t kvm_fetch_index; +uint64_t dirty_pages;/* Used for events with 'vcpu' and *without* the 'disabled' properties */ DECLARE_BITMAP (trace_dstate_delayed,CPU_TRACE_DSTATE_MAX_EVENTS );
include/sysemu/kvm.h +1 -0
@@ -547,4 +547,5 @@ bool kvm_cpu_check_are_resettable(void);bool kvm_arch_cpu_check_are_resettable (void ); +bool kvm_dirty_ring_enabled (void );#endif
accel/kvm/kvm-all.c +7 -0
@@ -469,6 +469,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp) cpu->kvm_fd = ret; cpu->kvm_state = s; cpu->vcpu_dirty = true; + cpu->dirty_pages =0 ; mmap_size =kvm_ioctl (s,KVM_GET_VCPU_MMAP_SIZE ,0 );if (mmap_size <0 ) {
@@ -743,6 +744,7 @@ static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CPUState *cpu) count++; } cpu->kvm_fetch_index = fetch; + cpu->dirty_pages += count;return count; }
@@ -2296,6 +2298,11 @@ bool kvm_vcpu_id_is_valid(int vcpu_id)return vcpu_id >=0 && vcpu_id <kvm_max_vcpu_id (s); } +bool kvm_dirty_ring_enabled (void ) +{ +return kvm_state->kvm_dirty_ring_size ? true : false; +} +static int kvm_init (MachineState *ms) {MachineClass *mc =MACHINE_GET_CLASS (ms);
accel/stubs/kvm-stub.c +5 -0
@@ -147,4 +147,9 @@ bool kvm_arm_supports_user_irq(void) {return false; } + +bool kvm_dirty_ring_enabled (void ) +{ +return false; +}#endif
Signature · 1 line
2.33.1
[PULL 03/20] memory: make global_dirty_tracking a bitmask
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
since dirty ring has been introduced, there are two methods to track dirty pages of vm. it seems that "logging" has a hint on the method, so rename the global_dirty_log to global_dirty_tracking would make description more accurate.
dirty rate measurement may start or stop dirty tracking during calculation. this conflict with migration because stop dirty tracking make migration leave dirty pages out then that'll be a problem.
make global_dirty_tracking a bitmask can let both migration and dirty rate measurement work fine. introduce GLOBAL_DIRTY_MIGRATION and GLOBAL_DIRTY_DIRTY_RATE to distinguish what current dirty tracking aims for, migration or dirty rate.
Message-Id: <9c9388657cfa0301bd2c1cfa36e7cf6da4aeca19.1624040308.git.huangy81@chinatelecom.cn>
--- include/exec/memory.h | 20 +++++++++++++++++--- include/exec/ram_addr.h | 4 ++-- hw/i386/xen/xen-hvm.c | 4 ++-- migration/ram.c | 15 +++++++++++---- softmmu/memory.c | 32 +++++++++++++++++++++----------- softmmu/trace-events | 1 + 6 files changed, 54 insertions(+), 22 deletions(-)
include/exec/memory.h +17 -3
@@ -61,7 +61,17 @@ static inline void fuzz_dma_read_cb(size_t addr, }#endif -extern bool global_dirty_log; +/* Possible bits for global_dirty_log_{start|stop} */ + +/* Dirty tracking enabled because migration is running */ +#define GLOBAL_DIRTY_MIGRATION (1U << 0) + +/* Dirty tracking enabled because measuring dirty rate */ +#define GLOBAL_DIRTY_DIRTY_RATE (1U << 1) + +#define GLOBAL_DIRTY_MASK (0x3) + +extern unsigned int global_dirty_tracking;typedef struct MemoryRegionOps MemoryRegionOps ;
@@ -2388,13 +2398,17 @@ void memory_listener_unregister(MemoryListener *listener);/** * memory_global_dirty_log_start: begin dirty logging for all regions +* +* @flags: purpose of starting dirty log, migration or dirty rate */ -void memory_global_dirty_log_start (void ); +void memory_global_dirty_log_start (unsigned int flags);/** * memory_global_dirty_log_stop: end dirty logging for all regions +* +* @flags: purpose of stopping dirty log, migration or dirty rate */ -void memory_global_dirty_log_stop (void ); +void memory_global_dirty_log_stop (unsigned int flags);void mtree_info (bool flatview,bool dispatch_tree,bool owner,bool disabled);
include/exec/ram_addr.h +2 -2
@@ -369,7 +369,7 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,qatomic_or (&blocks[DIRTY_MEMORY_VGA ][idx][offset], temp); -if (global_dirty_log) { +if (global_dirty_tracking) {qatomic_or ( &blocks[DIRTY_MEMORY_MIGRATION ][idx][offset], temp);
@@ -392,7 +392,7 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap, }else {uint8_t clients =tcg_enabled () ?DIRTY_CLIENTS_ALL :DIRTY_CLIENTS_NOCODE ; -if (!global_dirty_log) { +if (!global_dirty_tracking) { clients &= ~(1 <<DIRTY_MEMORY_MIGRATION ); }
hw/i386/xen/xen-hvm.c +2 -2
@@ -1613,8 +1613,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length)void qmp_xen_set_global_dirty_log (bool enable,Error **errp) {if (enable) { -memory_global_dirty_log_start (); +memory_global_dirty_log_start (GLOBAL_DIRTY_MIGRATION ); }else { -memory_global_dirty_log_stop (); +memory_global_dirty_log_stop (GLOBAL_DIRTY_MIGRATION ); } }
migration/ram.c +11 -4
@@ -2216,7 +2216,14 @@ static void ram_save_cleanup(void *opaque)/* caller have hold iothread lock or is in a bh, so there is * no writing race against the migration bitmap */ -memory_global_dirty_log_stop (); +if (global_dirty_tracking &GLOBAL_DIRTY_MIGRATION ) { +/* +* do not stop dirty log without starting it, since +* memory_global_dirty_log_stop will assert that +* memory_global_dirty_log_start/stop used in pairs +*/ +memory_global_dirty_log_stop (GLOBAL_DIRTY_MIGRATION ); + } }RAMBLOCK_FOREACH_NOT_IGNORED (block) {
@@ -2678,7 +2685,7 @@ static void ram_init_bitmaps(RAMState *rs)ram_list_init_bitmaps ();/* We don't use dirty log with background snapshots */ if (!migrate_background_snapshot ()) { -memory_global_dirty_log_start (); +memory_global_dirty_log_start (GLOBAL_DIRTY_MIGRATION );migration_bitmap_sync_precopy (rs); } }
@@ -3434,7 +3441,7 @@ void colo_incoming_start_dirty_log(void)/* Discard this dirty bitmap record */ bitmap_zero (block->bmap, block->max_length >>TARGET_PAGE_BITS ); } -memory_global_dirty_log_start (); +memory_global_dirty_log_start (GLOBAL_DIRTY_MIGRATION ); } ram_state->migration_dirty_pages =0 ;qemu_mutex_unlock_ramlist ();
@@ -3446,7 +3453,7 @@ void colo_release_ram_cache(void) {RAMBlock *block; -memory_global_dirty_log_stop (); +memory_global_dirty_log_stop (GLOBAL_DIRTY_MIGRATION );RAMBLOCK_FOREACH_NOT_IGNORED (block) {g_free (block->bmap); block->bmap =NULL ;
softmmu/memory.c +21 -11
@@ -39,7 +39,7 @@static unsigned memory_region_transaction_depth;static bool memory_region_update_pending;static bool ioeventfd_update_pending; -bool global_dirty_log; +unsigned int global_dirty_tracking; static QTAILQ_HEAD(, MemoryListener) memory_listeners =QTAILQ_HEAD_INITIALIZER (memory_listeners);
@@ -1821,7 +1821,7 @@ uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)uint8_t mask = mr->dirty_log_mask;RAMBlock *rb = mr->ram_block; -if (global_dirty_log && ((rb &&qemu_ram_is_migratable (rb)) || +if (global_dirty_tracking && ((rb &&qemu_ram_is_migratable (rb)) ||memory_region_is_iommu (mr))) { mask |= (1 <<DIRTY_MEMORY_MIGRATION ); }
@@ -2760,14 +2760,18 @@ void memory_global_after_dirty_log_sync(void)static VMChangeStateEntry *vmstate_change; -void memory_global_dirty_log_start (void ) +void memory_global_dirty_log_start (unsigned int flags) {if (vmstate_change) {qemu_del_vm_change_state_handler (vmstate_change); vmstate_change =NULL ; } - global_dirty_log = true; +assert (flags && !(flags & (~GLOBAL_DIRTY_MASK ))); +assert (!(global_dirty_tracking & flags)); + global_dirty_tracking |= flags; + +trace_global_dirty_changed (global_dirty_tracking);MEMORY_LISTENER_CALL_GLOBAL (log_global_start, Forward);
@@ -2777,9 +2781,13 @@ void memory_global_dirty_log_start(void)memory_region_transaction_commit (); } -static void memory_global_dirty_log_do_stop (void ) +static void memory_global_dirty_log_do_stop (unsigned int flags) { - global_dirty_log = false; +assert (flags && !(flags & (~GLOBAL_DIRTY_MASK ))); +assert ((global_dirty_tracking & flags) == flags); + global_dirty_tracking &= ~flags; + +trace_global_dirty_changed (global_dirty_tracking);/* Refresh DIRTY_MEMORY_MIGRATION bit. */ memory_region_transaction_begin ();
@@ -2792,8 +2800,9 @@ static void memory_global_dirty_log_do_stop(void)static void memory_vm_change_state_handler (void *opaque,bool running,RunState state) { +unsigned int flags = (unsigned int )(uintptr_t )opaque;if (running) { -memory_global_dirty_log_do_stop (); +memory_global_dirty_log_do_stop (flags);if (vmstate_change) {qemu_del_vm_change_state_handler (vmstate_change);
@@ -2802,18 +2811,19 @@ static void memory_vm_change_state_handler(void *opaque, bool running, } } -void memory_global_dirty_log_stop (void ) +void memory_global_dirty_log_stop (unsigned int flags) {if (!runstate_is_running ()) {if (vmstate_change) {return ; } vmstate_change =qemu_add_vm_change_state_handler ( - memory_vm_change_state_handler,NULL ); + memory_vm_change_state_handler, + (void *)(uintptr_t )flags);return ; } -memory_global_dirty_log_do_stop (); +memory_global_dirty_log_do_stop (flags); } static void listener_add_address_space(MemoryListener *listener,
@@ -2825,7 +2835,7 @@ static void listener_add_address_space(MemoryListener *listener,if (listener->begin) { listener->begin (listener); } -if (global_dirty_log) { +if (global_dirty_tracking) {if (listener->log_global_start) { listener->log_global_start (listener); }
softmmu/trace-events +1 -0
@@ -19,6 +19,7 @@ memory_region_sync_dirty(const char *mr, const char *listener, int global) "mr ' flatview_new(void *view, void *root) "%p (root %p)" flatview_destroy(void *view, void *root) "%p (root %p)" flatview_destroy_rcu(void *view, void *root) "%p (root %p)" +global_dirty_changed(unsigned int bitmask) "bitmask 0x%"PRIx32 # softmmu.c vm_stop_flush_all(int ret) "ret %d"
Signature · 1 line
2.33.1
[PULL 04/20] migration/dirtyrate: introduce struct and adjust DirtyRateStat
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
introduce "DirtyRateMeasureMode" to specify what method should be used to calculate dirty rate, introduce "DirtyRateVcpu" to store dirty rate for each vcpu.
use union to store stat data of specific mode
Message-Id: <661c98c40f40e163aa58334337af8f3ddf41316a.1624040308.git.huangy81@chinatelecom.cn>
--- qapi/migration.json | 30 +++++++++++++++++++++++++++ migration/dirtyrate.h | 21 +++++++++++++++---- migration/dirtyrate.c | 48 +++++++++++++++++++++++++------------------ 3 files changed, 75 insertions(+), 24 deletions(-)
qapi/migration.json +30 -0
@@ -1731,6 +1731,21 @@ { 'event': 'UNPLUG_PRIMARY', 'data': { 'device-id': 'str' } } +## +# @DirtyRateVcpu: +# +# Dirty rate of vcpu. +# +# @id: vcpu index. +# +# @dirty-rate: dirty rate. +# +# Since: 6.1 +# +## +{ 'struct': 'DirtyRateVcpu', + 'data': { 'id': 'int', 'dirty-rate': 'int64' } } + ## # @DirtyRateStatus: #
@@ -1748,6 +1763,21 @@ { 'enum': 'DirtyRateStatus', 'data': [ 'unstarted', 'measuring', 'measured'] } +## +# @DirtyRateMeasureMode: +# +# An enumeration of mode of measuring dirtyrate. +# +# @page-sampling: calculate dirtyrate by sampling pages. +# +# @dirty-ring: calculate dirtyrate by via dirty ring. +# +# Since: 6.1 +# +## +{ 'enum': 'DirtyRateMeasureMode', + 'data': ['page-sampling', 'dirty-ring'] } + ## # @DirtyRateInfo: #
migration/dirtyrate.h +17 -4
@@ -43,6 +43,7 @@struct DirtyRateConfig {uint64_t sample_pages_per_gigabytes;/* sample pages per GB */ int64_t sample_period_seconds;/* time duration between two sampling */ +DirtyRateMeasureMode mode;/* mode of dirtyrate measurement */ };/*
@@ -58,17 +59,29 @@ struct RamblockDirtyInfo {uint32_t *hash_result;/* array of hash result for sampled pages */ }; -/* -* Store calculation statistics for each measure. -*/ -struct DirtyRateStat { +typedef struct SampleVMStat {uint64_t total_dirty_samples;/* total dirty sampled page */ uint64_t total_sample_count;/* total sampled pages */ uint64_t total_block_mem_MB;/* size of total sampled pages in MB */ +}SampleVMStat ; + +typedef struct VcpuStat { +int nvcpu;/* number of vcpu */ +DirtyRateVcpu *rates;/* array of dirty rate for each vcpu */ +}VcpuStat ; + +/* +* Store calculation statistics for each measure. +*/ +struct DirtyRateStat {int64_t dirty_rate;/* dirty rate in MB/s */ int64_t start_time;/* calculation start time in units of second */ int64_t calc_time;/* time duration of two sampling in units of second */ uint64_t sample_pages;/* sample pages per GB */ +union { +SampleVMStat page_sampling; +VcpuStat dirty_ring; + }; };void *get_dirtyrate_thread (void *arg);
migration/dirtyrate.c +28 -20
91 lines not on this page · raw message
Signature · 1 line
2.33.1
Re: [PULL 04/20] migration/dirtyrate: introduce struct and adjust DirtyRateStat
Quoted 27 lines · Juan Quintela
From: Hyman Huang(黄勇) <[email protected]>
introduce "DirtyRateMeasureMode" to specify what method should be used to calculate dirty rate, introduce "DirtyRateVcpu" to store dirty rate for each vcpu.
use union to store stat data of specific mode
Signed-off-by Hyman Huang(黄勇) <[email protected]>Message-Id: <661c98c40f40e163aa58334337af8f3ddf41316a.1624040308.git.huangy81@chinatelecom.cn>
Reviewed-by Peter Xu <[email protected]>Reviewed-by Juan Quintela <[email protected]>Signed-off-by Juan Quintela <[email protected]>--- qapi/migration.json | 30 +++++++++++++++++++++++++++ migration/dirtyrate.h | 21 +++++++++++++++---- migration/dirtyrate.c | 48 +++++++++++++++++++++++++------------------ 3 files changed, 75 insertions(+), 24 deletions(-)qapi/migration.json +0
-0@@ -1731,6 +1731,21 @@ { 'event': 'UNPLUG_PRIMARY', 'data': { 'device-id': 'str' } }
qapi/migration.json line 1743
Quoted 10 lines
qapi/migration.json +10
-0+## +# @DirtyRateVcpu: +# +# Dirty rate of vcpu. +# +# @id: vcpu index. +# +# @dirty-rate: dirty rate. +# +# Since: 6.1
I'm a bit late on the review, since this pull request is already in. We'll want a followup patch that changes this to mention 6.2, to correctly match the release that will first have it. Such a followup is safe during freeze, since it is doc-only.
Quoted 8 lines
qapi/migration.json +5
-0+# +## +{ 'struct': 'DirtyRateVcpu', + 'data': { 'id': 'int', 'dirty-rate': 'int64' } } + ## # @DirtyRateStatus: #
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
[PULL 05/20] migration/dirtyrate: adjust order of registering thread
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
registering get_dirtyrate thread in advance so that both page-sampling and dirty-ring mode can be covered.
Message-Id: <d7727581a8e86d4a42fc3eacf7f310419b9ebf7e.1624040308.git.huangy81@chinatelecom.cn>
--- migration/dirtyrate.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
migration/dirtyrate.c +3 -2
@@ -352,7 +352,6 @@ static void calculate_dirtyrate(struct DirtyRateConfig config)int64_t msec =0 ;int64_t initial_time; -rcu_register_thread ();rcu_read_lock (); initial_time =qemu_clock_get_ms (QEMU_CLOCK_REALTIME );if (!record_ramblock_hash_info (&block_dinfo, config, &block_count)) {
@@ -375,7 +374,6 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) out:rcu_read_unlock ();free_ramblock_dirty_info (block_dinfo, block_count); -rcu_unregister_thread (); }void *get_dirtyrate_thread (void *arg)
@@ -383,6 +381,7 @@ void *get_dirtyrate_thread(void *arg)struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg;int ret;int64_t start_time; +rcu_register_thread (); ret =dirtyrate_set_state (&CalculatingState,DIRTY_RATE_STATUS_UNSTARTED ,DIRTY_RATE_STATUS_MEASURING );
@@ -401,6 +400,8 @@ void *get_dirtyrate_thread(void *arg)if (ret ==-1 ) {error_report ("change dirtyrate state failed." ); } + +rcu_unregister_thread ();return NULL ; }
Signature · 1 line
2.33.1
[PULL 06/20] migration/dirtyrate: move init step of calculation to main thread
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
since main thread may "query dirty rate" at any time, it's better to move init step into main thead so that synchronization overhead between "main" and "get_dirtyrate" can be reduced.
Message-Id: <109f8077518ed2f13068e3bfb10e625e964780f1.1624040308.git.huangy81@chinatelecom.cn>
--- migration/dirtyrate.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
migration/dirtyrate.c +5 -4
@@ -380,7 +380,6 @@ void *get_dirtyrate_thread(void *arg) {struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg;int ret; -int64_t start_time;rcu_register_thread (); ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTARTED,
@@ -390,9 +389,6 @@ void *get_dirtyrate_thread(void *arg)return NULL ; } - start_time =qemu_clock_get_ms (QEMU_CLOCK_REALTIME ) /1000 ; -init_dirtyrate_stat (start_time, config); -calculate_dirtyrate (config); ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_MEASURING,
@@ -411,6 +407,7 @@ void qmp_calc_dirty_rate(int64_t calc_time, bool has_sample_pages,static struct DirtyRateConfig config;QemuThread thread;int ret; +int64_t start_time;/* * If the dirty rate is already being measured, don't attempt to start.
@@ -451,6 +448,10 @@ void qmp_calc_dirty_rate(int64_t calc_time, bool has_sample_pages, config.sample_period_seconds = calc_time; config.sample_pages_per_gigabytes = sample_pages; config.mode =DIRTY_RATE_MEASURE_MODE_PAGE_SAMPLING ; + + start_time =qemu_clock_get_ms (QEMU_CLOCK_REALTIME ) /1000 ; +init_dirtyrate_stat (start_time, config); +qemu_thread_create (&thread,"get_dirtyrate" , get_dirtyrate_thread, (void *)&config,QEMU_THREAD_DETACHED ); }
Signature · 1 line
2.33.1
[PULL 07/20] migration/dirtyrate: implement dirty-ring dirtyrate calculation
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
use dirty ring feature to implement dirtyrate calculation.
introduce mode option in qmp calc_dirty_rate to specify what method should be used when calculating dirtyrate, either page-sampling or dirty-ring should be passed.
introduce "dirty_ring:-r" option in hmp calc_dirty_rate to indicate dirty ring method should be used for calculation.
Message-Id: <7db445109bd18125ce8ec86816d14f6ab5de6a7d.1624040308.git.huangy81@chinatelecom.cn>
--- qapi/migration.json | 16 +++- migration/dirtyrate.c | 208 +++++++++++++++++++++++++++++++++++++++-- hmp-commands.hx | 7 +- migration/trace-events | 2 + 4 files changed, 218 insertions(+), 15 deletions(-)
qapi/migration.json +14 -2
@@ -1796,6 +1796,12 @@ # @sample-pages: page count per GB for sample dirty pages # the default value is 512 (since 6.1) # +# @mode: mode containing method of calculate dirtyrate includes +# 'page-sampling' and 'dirty-ring' (Since 6.1) +# +# @vcpu-dirty-rate: dirtyrate for each vcpu if dirty-ring +# mode specified (Since 6.1) +# # Since: 5.2 # ##
@@ -1804,7 +1810,9 @@ 'status': 'DirtyRateStatus', 'start-time': 'int64', 'calc-time': 'int64', - 'sample-pages': 'uint64'} } + 'sample-pages': 'uint64', + 'mode': 'DirtyRateMeasureMode', + '*vcpu-dirty-rate': [ 'DirtyRateVcpu' ] } } ## # @calc-dirty-rate:
@@ -1816,6 +1824,9 @@ # @sample-pages: page count per GB for sample dirty pages # the default value is 512 (since 6.1) # +# @mode: mechanism of calculating dirtyrate includes +# 'page-sampling' and 'dirty-ring' (Since 6.1) +# # Since: 5.2 # # Example:
@@ -1824,7 +1835,8 @@ # ## { 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64', - '*sample-pages': 'int'} } + '*sample-pages': 'int', + '*mode': 'DirtyRateMeasureMode'} } ## # @query-dirty-rate:
migration/dirtyrate.c +198 -10
302 lines not on this page · raw message
hmp-commands.hx +4 -3
@@ -1737,8 +1737,9 @@ ERST { .name = "calc_dirty_rate", - .args_type = "second:l,sample_pages_per_GB:l?", - .params = "second [sample_pages_per_GB]", - .help = "start a round of guest dirty rate measurement", + .args_type = "dirty_ring:-r,second:l,sample_pages_per_GB:l?", + .params = "[-r] second [sample_pages_per_GB]", + .help = "start a round of guest dirty rate measurement (using -d to" + "\n\t\t\t specify dirty ring as the method of calculation)", .cmd = hmp_calc_dirty_rate, },
migration/trace-events +2 -0
@@ -333,6 +333,8 @@ get_ramblock_vfn_hash(const char *idstr, uint64_t vfn, uint32_t crc) "ramblock n calc_page_dirty_rate(const char *idstr, uint32_t new_crc, uint32_t old_crc) "ramblock name: %s, new crc: %" PRIu32 ", old crc: %" PRIu32 skip_sample_ramblock(const char *idstr, uint64_t ramblock_size) "ramblock name: %s, ramblock size: %" PRIu64 find_page_matched(const char *idstr) "ramblock %s addr or size changed" +dirtyrate_calculate(int64_t dirtyrate) "dirty rate: %" PRIi64 " MB/s" +dirtyrate_do_calculate_vcpu(int idx, uint64_t rate) "vcpu[%d]: %"PRIu64 " MB/s" # block.c migration_block_init_shared(const char *blk_device_name) "Start migration for %s with shared base image"
Signature · 1 line
2.33.1
Re: [PULL 07/20] migration/dirtyrate: implement dirty-ring dirtyrate calculation
Hi Juan,
Quoted 4 lines · Juan Quintela
From: Hyman Huang(黄勇) <[email protected]>
use dirty ring feature to implement dirtyrate calculation.
Quoted 8 lines
introduce mode option in qmp
calc_dirty_rateto specify what method should be used when calculating dirtyrate, either page-sampling or dirty-ring should be passed.introduce "dirty_ring:-r" option in hmp
calc_dirty_rateto indicate dirty ring method should be used for calculation.Signed-off-by Hyman Huang(黄勇) <[email protected]>
Just noticing in the git history some commits from your pull request have the author name (from + S-o-b) mojibaked.
Quoted 10 lines
Message-Id: <7db445109bd18125ce8ec86816d14f6ab5de6a7d.1624040308.git.huangy81@chinatelecom.cn>
Reviewed-by Peter Xu <[email protected]>Reviewed-by Juan Quintela <[email protected]>Signed-off-by Juan Quintela <[email protected]>--- qapi/migration.json | 16 +++- migration/dirtyrate.c | 208 +++++++++++++++++++++++++++++++++++++++-- hmp-commands.hx | 7 +- migration/trace-events | 2 + 4 files changed, 218 insertions(+), 15 deletions(-)
Re: [PULL 07/20] migration/dirtyrate: implement dirty-ring dirtyrate calculation
Quoted 11 lines · Philippe Mathieu-Daudé
Hi Juan,
Quoted 8 lines · Juan Quintela
From: Hyman Huang(黄勇) <[email protected]>
use dirty ring feature to implement dirtyrate calculation.
introduce mode option in qmp
calc_dirty_rateto specify what method should be used when calculating dirtyrate, either page-sampling or dirty-ring should be passed.
Quoted 7 lines
Quoted 4 lines
introduce "dirty_ring:-r" option in hmp
calc_dirty_rateto indicate dirty ring method should be used for calculation.Signed-off-by Hyman Huang(黄勇) <[email protected]>Just noticing in the git history some commits from your pull request have the author name (from + S-o-b) mojibaked.
Didn't knew the term.
I noticed that after the fact. Still have to debug *why* my setup decided to break some of the patches and not the others. Obviosly, I have no clue about how to write/read/whatever anything that is not latin1, so if something breaks, I got in a difficult place O:-)
Thanks, Juan.
Quoted 10 lines
Quoted 10 lines
<7db445109bd18125ce8ec86816d14f6ab5de6a7d.1624040308.git.huangy81@chinatelecom.cn>
Reviewed-by Peter Xu <[email protected]>Reviewed-by Juan Quintela <[email protected]>Signed-off-by Juan Quintela <[email protected]>--- qapi/migration.json | 16 +++- migration/dirtyrate.c | 208 +++++++++++++++++++++++++++++++++++++++-- hmp-commands.hx | 7 +- migration/trace-events | 2 + 4 files changed, 218 insertions(+), 15 deletions(-)
[PULL 08/20] migration: Make migration blocker work for snapshots too
In reply to Juan QuintelaFrom: Peter Xu <[email protected]>
save_snapshot() checks migration blocker, which looks sane. At the meantime we should also teach the blocker add helper to fail if during a snapshot, just like for migrations.
--- migration/migration.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
migration/migration.c +8 -7
@@ -2058,15 +2058,16 @@ int migrate_add_blocker(Error *reason, Error **errp)return -EACCES ; } -if (migration_is_idle ()) { - migration_blockers =g_slist_prepend (migration_blockers, reason); -return 0 ; +/* Snapshots are similar to migrations, so check RUN_STATE_SAVE_VM too. */ +if (runstate_check (RUN_STATE_SAVE_VM ) || !migration_is_idle ()) { +error_propagate_prepend (errp,error_copy (reason), +"disallowing migration blocker " +"(migration/snapshot in progress) for: " ); +return -EBUSY ; } -error_propagate_prepend (errp,error_copy (reason), -"disallowing migration blocker " -"(migration in progress) for: " ); -return -EBUSY ; + migration_blockers =g_slist_prepend (migration_blockers, reason); +return 0 ; }void migrate_del_blocker (Error *reason)
Signature · 1 line
2.33.1
[PULL 09/20] migration: Add migrate_add_blocker_internal()
In reply to Juan QuintelaFrom: Peter Xu <[email protected]>
An internal version that removes -only-migratable implications. It can be used for temporary migration blockers like dump-guest-memory.
--- include/migration/blocker.h | 16 ++++++++++++++++ migration/migration.c | 21 +++++++++++++-------- 2 files changed, 29 insertions(+), 8 deletions(-)
include/migration/blocker.h +16 -0
@@ -25,6 +25,22 @@*/ int migrate_add_blocker (Error *reason,Error **errp); +/** +* @migrate_add_blocker_internal - prevent migration from proceeding without +* only-migrate implications +* +* @reason - an error to be returned whenever migration is attempted +* +* @errp - [out] The reason (if any) we cannot block migration right now. +* +* @returns - 0 on success, -EBUSY on failure, with errp set. +* +* Some of the migration blockers can be temporary (e.g., for a few seconds), +* so it shouldn't need to conflict with "-only-migratable". For those cases, +* we can call this function rather than @migrate_add_blocker(). +*/ +int migrate_add_blocker_internal (Error *reason,Error **errp); +/** * @migrate_del_blocker - remove a blocking error from migration *
migration/migration.c +13 -8
@@ -2049,15 +2049,8 @@ void migrate_init(MigrationState *s) s->threshold_size =0 ; } -int migrate_add_blocker (Error *reason,Error **errp) +int migrate_add_blocker_internal (Error *reason,Error **errp) { -if (only_migratable) { -error_propagate_prepend (errp,error_copy (reason), -"disallowing migration blocker " -"(--only-migratable) for: " ); -return -EACCES ; - } -/* Snapshots are similar to migrations, so check RUN_STATE_SAVE_VM too. */ if (runstate_check (RUN_STATE_SAVE_VM ) || !migration_is_idle ()) { error_propagate_prepend(errp, error_copy(reason),
@@ -2070,6 +2063,18 @@ int migrate_add_blocker(Error *reason, Error **errp)return 0 ; } +int migrate_add_blocker (Error *reason,Error **errp) +{ +if (only_migratable) { +error_propagate_prepend (errp,error_copy (reason), +"disallowing migration blocker " +"(--only-migratable) for: " ); +return -EACCES ; + } + +return migrate_add_blocker_internal (reason, errp); +} +void migrate_del_blocker (Error *reason) { migration_blockers =g_slist_remove (migration_blockers, reason);
Signature · 1 line
2.33.1
[PULL 10/20] dump-guest-memory: Block live migration
In reply to Juan QuintelaFrom: Peter Xu <[email protected]>
Both dump-guest-memory and live migration caches vm state at the beginning. Either of them entering the other one will cause race on the vm state, and even more severe on that (please refer to the crash report in the bug link).
Let's block live migration in dump-guest-memory, and that'll also block dump-guest-memory if it detected that we're during a live migration.
Side note: migrate_del_blocker() can be called even if the blocker is not inserted yet, so it's safe to unconditionally delete that blocker in dump_cleanup (g_slist_remove allows no-entry-found case).
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1996609
--- dump/dump.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
dump/dump.c +19 -0
@@ -29,6 +29,7 @@#include "qemu/error-report.h" #include "qemu/main-loop.h" #include "hw/misc/vmcoreinfo.h" +#include "migration/blocker.h" #ifdef TARGET_X86_64 #include "win_dump.h"
@@ -47,6 +48,8 @@#define MAX_GUEST_NOTE_SIZE (1 << 20)/* 1MB should be enough */ +static Error *dump_migration_blocker; +#define ELF_NOTE_SIZE (hdr_size, name_size, desc_size) \ ((DIV_ROUND_UP((hdr_size), 4) + \ DIV_ROUND_UP((name_size), 4) + \
@@ -101,6 +104,7 @@ static int dump_cleanup(DumpState *s)qemu_mutex_unlock_iothread (); } } +migrate_del_blocker (dump_migration_blocker);return 0 ; }
@@ -2005,6 +2009,21 @@ void qmp_dump_guest_memory(bool paging, const char *file,return ; } +if (!dump_migration_blocker) { +error_setg (&dump_migration_blocker, +"Live migration disabled: dump-guest-memory in progress" ); + } + +/* +* Allows even for -only-migratable, but forbid migration during the +* process of dump guest memory. +*/ +if (migrate_add_blocker_internal (dump_migration_blocker, errp)) { +/* Remember to release the fd before passing it over to dump state */ +close (fd); +return ; + } + s = &dump_state_global;dump_state_prepare (s);
Signature · 1 line
2.33.1
[PULL 11/20] memory: Introduce replay_discarded callback for RamDiscardManager
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
Introduce replay_discarded callback similar to our existing replay_populated callback, to be used my migration code to never migrate discarded memory.
--- include/exec/memory.h | 21 +++++++++++++++++++++ softmmu/memory.c | 11 +++++++++++ 2 files changed, 32 insertions(+)
include/exec/memory.h +21 -0
@@ -550,6 +550,7 @@ static inline void ram_discard_listener_init(RamDiscardListener *rdl, }typedef int (*ReplayRamPopulate )(MemoryRegionSection *section,void *opaque); +typedef void (*ReplayRamDiscard )(MemoryRegionSection *section,void *opaque);/* * RamDiscardManagerClass:
@@ -638,6 +639,21 @@ struct RamDiscardManagerClass {MemoryRegionSection *section, ReplayRamPopulate replay_fn, void *opaque); +/** +* @replay_discarded: +* +* Call the #ReplayRamDiscard callback for all discarded parts within the +* #MemoryRegionSection via the #RamDiscardManager. +* +* @rdm: the #RamDiscardManager +* @section: the #MemoryRegionSection +* @replay_fn: the #ReplayRamDiscard callback +* @opaque: pointer to forward to the callback +*/ +void (*replay_discarded)(const RamDiscardManager *rdm, +MemoryRegionSection *section, +ReplayRamDiscard replay_fn,void *opaque); +/** * @register_listener: *
@@ -682,6 +698,11 @@ int ram_discard_manager_replay_populated(const RamDiscardManager *rdm,ReplayRamPopulate replay_fn, void *opaque); +void ram_discard_manager_replay_discarded (const RamDiscardManager *rdm, +MemoryRegionSection *section, +ReplayRamDiscard replay_fn, +void *opaque); +void ram_discard_manager_register_listener (RamDiscardManager *rdm,RamDiscardListener *rdl,MemoryRegionSection *section);
softmmu/memory.c +11 -0
@@ -2081,6 +2081,17 @@ int ram_discard_manager_replay_populated(const RamDiscardManager *rdm,return rdmc->replay_populated (rdm, section, replay_fn, opaque); } +void ram_discard_manager_replay_discarded (const RamDiscardManager *rdm, +MemoryRegionSection *section, +ReplayRamDiscard replay_fn, +void *opaque) +{ +RamDiscardManagerClass *rdmc =RAM_DISCARD_MANAGER_GET_CLASS (rdm); + +g_assert (rdmc->replay_discarded); + rdmc->replay_discarded (rdm, section, replay_fn, opaque); +} +void ram_discard_manager_register_listener (RamDiscardManager *rdm,RamDiscardListener *rdl,MemoryRegionSection *section)
Signature · 1 line
2.33.1
[PULL 13/20] migration/ram: Handle RAMBlocks with a RamDiscardManager on the migration source
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
We don't want to migrate memory that corresponds to discarded ranges as managed by a RamDiscardManager responsible for the mapped memory region of the RAMBlock. The content of these pages is essentially stale and without any guarantees for the VM ("logically unplugged").
Depending on the underlying memory type, even reading memory might populate memory on the source, resulting in an undesired memory consumption. Of course, on the destination, even writing a zeropage consumes memory, which we also want to avoid (similar to free page hinting).
Currently, virtio-mem tries achieving that goal (not migrating "unplugged" memory that was discarded) by going via qemu_guest_free_page_hint() - but it's hackish and incomplete.
For example, background snapshots still end up reading all memory, as they don't do bitmap syncs. Postcopy recovery code will re-add previously cleared bits to the dirty bitmap and migrate them.
Let's consult the RamDiscardManager after setting up our dirty bitmap initially and when postcopy recovery code reinitializes it: clear corresponding bits in the dirty bitmaps (e.g., of the RAMBlock and inside KVM). It's important to fixup the dirty bitmap *after* our initial bitmap sync, such that the corresponding dirty bits in KVM are actually cleared.
As colo is incompatible with discarding of RAM and inhibits it, we don't have to bother.
Note: if a misbehaving guest would use discarded ranges after migration started we would still migrate that memory: however, then we already populated that memory on the migration source.
--- migration/ram.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+)
migration/ram.c +77 -0
@@ -858,6 +858,60 @@ static inline bool migration_bitmap_clear_dirty(RAMState *rs,return ret; } +static void dirty_bitmap_clear_section (MemoryRegionSection *section, +void *opaque) +{ +const hwaddr offset = section->offset_within_region; +const hwaddr size =int128_get64 (section->size); +const unsigned long start = offset >>TARGET_PAGE_BITS ; +const unsigned long npages = size >>TARGET_PAGE_BITS ; +RAMBlock *rb = section->mr->ram_block; +uint64_t *cleared_bits = opaque; + +/* +* We don't grab ram_state->bitmap_mutex because we expect to run +* only when starting migration or during postcopy recovery where +* we don't have concurrent access. +*/ +if (!migration_in_postcopy () && !migrate_background_snapshot ()) { +migration_clear_memory_region_dirty_bitmap_range (rb, start, npages); + } + *cleared_bits +=bitmap_count_one_with_offset (rb->bmap, start, npages); +bitmap_clear (rb->bmap, start, npages); +} + +/* +* Exclude all dirty pages from migration that fall into a discarded range as +* managed by a RamDiscardManager responsible for the mapped memory region of +* the RAMBlock. Clear the corresponding bits in the dirty bitmaps. +* +* Discarded pages ("logically unplugged") have undefined content and must +* not get migrated, because even reading these pages for migration might +* result in undesired behavior. +* +* Returns the number of cleared bits in the RAMBlock dirty bitmap. +* +* Note: The result is only stable while migrating (precopy/postcopy). +*/ +static uint64_t ramblock_dirty_bitmap_clear_discarded_pages (RAMBlock *rb) +{ +uint64_t cleared_bits =0 ; + +if (rb->mr && rb->bmap &&memory_region_has_ram_discard_manager (rb->mr)) { +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (rb->mr); +MemoryRegionSection section = { + .mr = rb->mr, + .offset_within_region =0 , + .size =int128_make64 (qemu_ram_get_used_length (rb)), + }; + +ram_discard_manager_replay_discarded (rdm, §ion, + dirty_bitmap_clear_section, + &cleared_bits); + } +return cleared_bits; +} +/* Called with RCU critical section */ static void ramblock_sync_dirty_bitmap (RAMState *rs,RAMBlock *rb) {
@@ -2675,6 +2729,19 @@ static void ram_list_init_bitmaps(void) } } +static void migration_bitmap_clear_discarded_pages (RAMState *rs) +{ +unsigned long pages; +RAMBlock *rb; + +RCU_READ_LOCK_GUARD (); + +RAMBLOCK_FOREACH_NOT_IGNORED (rb) { + pages =ramblock_dirty_bitmap_clear_discarded_pages (rb); + rs->migration_dirty_pages -= pages; + } +} +static void ram_init_bitmaps (RAMState *rs) {/* For memory_global_dirty_log_start below. */
@@ -2691,6 +2758,12 @@ static void ram_init_bitmaps(RAMState *rs) }qemu_mutex_unlock_ramlist ();qemu_mutex_unlock_iothread (); + +/* +* After an eventual first bitmap sync, fixup the initial bitmap +* containing all 1s to exclude any discarded pages from migration. +*/ +migration_bitmap_clear_discarded_pages (rs); }static int ram_init_all (RAMState **rsp)
@@ -4119,6 +4192,10 @@ int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *block)*/ bitmap_complement (block->bmap, block->bmap, nbits); +/* Clear dirty bits of discarded ranges that we don't want to migrate. */ +ramblock_dirty_bitmap_clear_discarded_pages (block); + +/* We'll recalculate migration_dirty_pages in ram_state_resume_prepare(). */ trace_ram_dirty_bitmap_reload_complete (block->idstr);/*
Signature · 1 line
2.33.1
[PULL 14/20] virtio-mem: Drop precopy notifier
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
Migration code now properly handles RAMBlocks which are indirectly managed by a RamDiscardManager. No need for manual handling via the free page optimization interface, let's get rid of it.
--- include/hw/virtio/virtio-mem.h | 3 --- hw/virtio/virtio-mem.c | 34 ---------------------------------- 2 files changed, 37 deletions(-)
include/hw/virtio/virtio-mem.h +0 -3
@@ -65,9 +65,6 @@ struct VirtIOMEM {/* notifiers to notify when "size" changes */ NotifierList size_change_notifiers; -/* don't migrate unplugged memory */ -NotifierWithReturn precopy_notifier; -/* listeners to notify on plug/unplug activity. */ QLIST_HEAD(, RamDiscardListener) rdl_list; };
hw/virtio/virtio-mem.c +0 -34
@@ -776,7 +776,6 @@ static void virtio_mem_device_realize(DeviceState *dev, Error **errp)host_memory_backend_set_mapped (vmem->memdev, true);vmstate_register_ram (&vmem->memdev->mr,DEVICE (vmem));qemu_register_reset (virtio_mem_system_reset, vmem); -precopy_add_notifier (&vmem->precopy_notifier);/* * Set ourselves as RamDiscardManager before the plug handler maps the
@@ -796,7 +795,6 @@ static void virtio_mem_device_unrealize(DeviceState *dev)* found via an address space anymore. Unset ourselves. */ memory_region_set_ram_discard_manager (&vmem->memdev->mr,NULL ); -precopy_remove_notifier (&vmem->precopy_notifier);qemu_unregister_reset (virtio_mem_system_reset, vmem);vmstate_unregister_ram (&vmem->memdev->mr,DEVICE (vmem));host_memory_backend_set_mapped (vmem->memdev, false);
@@ -1089,43 +1087,11 @@ static void virtio_mem_set_block_size(Object *obj, Visitor *v, const char *name, vmem->block_size = value; } -static int virtio_mem_precopy_exclude_range_cb (const VirtIOMEM *vmem,void *arg, -uint64_t offset,uint64_t size) -{ -void *const host =qemu_ram_get_host_addr (vmem->memdev->mr.ram_block); - -qemu_guest_free_page_hint (host + offset, size); -return 0 ; -} - -static void virtio_mem_precopy_exclude_unplugged (VirtIOMEM *vmem) -{ -virtio_mem_for_each_unplugged_range (vmem,NULL , - virtio_mem_precopy_exclude_range_cb); -} - -static int virtio_mem_precopy_notify (NotifierWithReturn *n,void *data) -{ -VirtIOMEM *vmem =container_of (n, VirtIOMEM, precopy_notifier); -PrecopyNotifyData *pnd = data; - -switch (pnd->reason) { -case PRECOPY_NOTIFY_AFTER_BITMAP_SYNC : -virtio_mem_precopy_exclude_unplugged (vmem); -break ; -default : -break ; - } - -return 0 ; -} -static void virtio_mem_instance_init (Object *obj) {VirtIOMEM *vmem =VIRTIO_MEM (obj);notifier_list_init (&vmem->size_change_notifiers); - vmem->precopy_notifier.notify = virtio_mem_precopy_notify;QLIST_INIT (&vmem->rdl_list); object_property_add(obj, VIRTIO_MEM_SIZE_PROP, "size", virtio_mem_get_size,
Signature · 1 line
2.33.1
[PULL 15/20] migration/postcopy: Handle RAMBlocks with a RamDiscardManager on the destination
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
Currently, when someone (i.e., the VM) accesses discarded parts inside a RAMBlock with a RamDiscardManager managing the corresponding mapped memory region, postcopy will request migration of the corresponding page from the source. The source, however, will never answer, because it refuses to migrate such pages with undefined content ("logically unplugged"): the pages are never dirty, and get_queued_page() will consequently skip processing these postcopy requests.
Especially reading discarded ("logically unplugged") ranges is supposed to work in some setups (for example with current virtio-mem), although it barely ever happens: still, not placing a page would currently stall the VM, as it cannot make forward progress.
Let's check the state via the RamDiscardManager (the state e.g., of virtio-mem is migrated during precopy) and avoid sending a request that will never get answered. Place a fresh zero page instead to keep the VM working. This is the same behavior that would happen automatically without userfaultfd being active, when accessing virtual memory regions without populated pages -- "populate on demand".
For now, there are valid cases (as documented in the virtio-mem spec) where a VM might read discarded memory; in the future, we will disallow that. Then, we might want to handle that case differently, e.g., warning the user that the VM seems to be mis-behaving.
--- migration/ram.h | 1 + migration/postcopy-ram.c | 31 +++++++++++++++++++++++++++---- migration/ram.c | 21 +++++++++++++++++++++ 3 files changed, 49 insertions(+), 4 deletions(-)
migration/ram.h +1 -0
@@ -72,6 +72,7 @@ void ramblock_recv_bitmap_set_range(RAMBlock *rb, void *host_addr, size_t nr);int64_t ramblock_recv_bitmap_send (QEMUFile *file,const char *block_name);int ram_dirty_bitmap_reload (MigrationState *s,RAMBlock *rb); +bool ramblock_page_is_discarded (RAMBlock *rb,ram_addr_t start);/* ram cache */ int colo_init_ram_cache (void );
migration/postcopy-ram.c +27 -4
@@ -671,6 +671,29 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd,return ret; } +static int postcopy_request_page (MigrationIncomingState *mis,RAMBlock *rb, +ram_addr_t start,uint64_t haddr) +{ +void *aligned = (void *)(uintptr_t )ROUND_DOWN (haddr,qemu_ram_pagesize (rb)); + +/* +* Discarded pages (via RamDiscardManager) are never migrated. On unlikely +* access, place a zeropage, which will also set the relevant bits in the +* recv_bitmap accordingly, so we won't try placing a zeropage twice. +* +* Checking a single bit is sufficient to handle pagesize > TPS as either +* all relevant bits are set or not. +*/ +assert (QEMU_IS_ALIGNED (start,qemu_ram_pagesize (rb))); +if (ramblock_page_is_discarded (rb, start)) { +bool received =ramblock_recv_bitmap_test_byte_offset (rb, start); + +return received ?0 :postcopy_place_page_zero (mis, aligned, rb); + } + +return migrate_send_rp_req_pages (mis, rb, start, haddr); +} +/* * Callback from shared fault handlers to ask for a page, * the page must be specified by a RAMBlock and an offset in that rb
@@ -690,7 +713,7 @@ int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset);return postcopy_wake_shared (pcfd, client_addr, rb); } -migrate_send_rp_req_pages (mis, rb, aligned_rbo, client_addr); +postcopy_request_page (mis, rb, aligned_rbo, client_addr);return 0 ; }
@@ -984,8 +1007,8 @@ retry:* Send the request to the source - we want to request one * of our host page sizes (which is >= TPS) */ - ret =migrate_send_rp_req_pages (mis, rb, rb_offset, - msg.arg.pagefault.address); + ret =postcopy_request_page (mis, rb, rb_offset, + msg.arg.pagefault.address);if (ret) {/* May be network failure, try to wait for recovery */ if (ret == -EIO &&postcopy_pause_fault_thread (mis)) {
@@ -993,7 +1016,7 @@ retry: goto retry; }else {/* This is a unavoidable fault */ -error_report ("%s: migrate_send_rp_req_pages() get %d" , +error_report ("%s: postcopy_request_page() get %d" , __func__, ret);break ; }
migration/ram.c +21 -0
@@ -912,6 +912,27 @@ static uint64_t ramblock_dirty_bitmap_clear_discarded_pages(RAMBlock *rb)return cleared_bits; } +/* +* Check if a host-page aligned page falls into a discarded range as managed by +* a RamDiscardManager responsible for the mapped memory region of the RAMBlock. +* +* Note: The result is only stable while migrating (precopy/postcopy). +*/ +bool ramblock_page_is_discarded (RAMBlock *rb,ram_addr_t start) +{ +if (rb->mr &&memory_region_has_ram_discard_manager (rb->mr)) { +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (rb->mr); +MemoryRegionSection section = { + .mr = rb->mr, + .offset_within_region = start, + .size =int128_make64 (qemu_ram_pagesize (rb)), + }; + +return !ram_discard_manager_is_populated (rdm, §ion); + } +return false; +} +/* Called with RCU critical section */ static void ramblock_sync_dirty_bitmap (RAMState *rs,RAMBlock *rb) {
Signature · 1 line
2.33.1
[PULL 16/20] migration: Simplify alignment and alignment checks
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
Let's use QEMU_ALIGN_DOWN() and friends to make the code a bit easier to read.
--- migration/migration.c | 6 +++--- migration/postcopy-ram.c | 9 ++++----- migration/ram.c | 2 +- 3 files changed, 8 insertions(+), 9 deletions(-)
migration/migration.c +3 -3
@@ -391,7 +391,7 @@ int migrate_send_rp_message_req_pages(MigrationIncomingState *mis,int migrate_send_rp_req_pages (MigrationIncomingState *mis,RAMBlock *rb,ram_addr_t start,uint64_t haddr) { -void *aligned = (void *)(uintptr_t )(haddr & (-qemu_ram_pagesize (rb))); +void *aligned = (void *)(uintptr_t )ROUND_DOWN (haddr,qemu_ram_pagesize (rb));bool received = false; WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) {
@@ -2637,8 +2637,8 @@ static void migrate_handle_rp_req_pages(MigrationState *ms, const char* rbname,* Since we currently insist on matching page sizes, just sanity check * we're being asked for whole host pages. */ -if (start & (our_host_ps -1 ) || - (len & (our_host_ps -1 ))) { +if (!QEMU_IS_ALIGNED (start, our_host_ps) || + !QEMU_IS_ALIGNED (len, our_host_ps)) {error_report ("%s: Misaligned page request, start: " RAM_ADDR_FMT " len: %zd" , __func__, start, len);mark_source_rp_bad (ms);
migration/postcopy-ram.c +4 -5
@@ -402,7 +402,7 @@ bool postcopy_ram_supported_by_host(MigrationIncomingState *mis) strerror(errno)); goto out; } -g_assert (((size_t )testarea & (pagesize -1 )) ==0 ); +g_assert (QEMU_PTR_IS_ALIGNED (testarea, pagesize)); reg_struct.range.start = (uintptr_t )testarea; reg_struct.range.len = pagesize;
@@ -660,7 +660,7 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd,struct uffdio_range range;int ret;trace_postcopy_wake_shared (client_addr,qemu_ram_get_idstr (rb)); - range.start = client_addr & ~(pagesize -1 ); + range.start =ROUND_DOWN (client_addr, pagesize); range.len = pagesize; ret =ioctl (pcfd->fd,UFFDIO_WAKE , &range);if (ret) {
@@ -702,8 +702,7 @@ static int postcopy_request_page(MigrationIncomingState *mis, RAMBlock *rb,int postcopy_request_shared_page (struct PostCopyFD *pcfd,RAMBlock *rb,uint64_t client_addr,uint64_t rb_offset) { -size_t pagesize =qemu_ram_pagesize (rb); -uint64_t aligned_rbo = rb_offset & ~(pagesize -1 ); +uint64_t aligned_rbo =ROUND_DOWN (rb_offset,qemu_ram_pagesize (rb));MigrationIncomingState *mis =migration_incoming_get_current (); trace_postcopy_request_shared_page(pcfd->idstr, qemu_ram_get_idstr(rb),
@@ -993,7 +992,7 @@ static void *postcopy_ram_fault_thread(void *opaque)break ; } - rb_offset &= ~(qemu_ram_pagesize (rb) -1 ); + rb_offset =ROUND_DOWN (rb_offset,qemu_ram_pagesize (rb)); trace_postcopy_ram_fault_thread_request(msg.arg.pagefault.address, qemu_ram_get_idstr(rb), rb_offset,
migration/ram.c +1 -1
@@ -811,7 +811,7 @@ static void migration_clear_memory_region_dirty_bitmap(RAMBlock *rb,assert (shift >=6 ); size =1ULL << (TARGET_PAGE_BITS + shift); - start = (((ram_addr_t )page) <<TARGET_PAGE_BITS ) & (-size); + start =QEMU_ALIGN_DOWN ((ram_addr_t )page <<TARGET_PAGE_BITS , size);trace_migration_bitmap_clear_dirty (rb->idstr, start, size, page);memory_region_clear_dirty_bitmap (rb->mr, start, size); }
Signature · 1 line
2.33.1
[PULL 17/20] migration/ram: Factor out populating pages readable in ram_block_populate_pages()
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
Let's factor out prefaulting/populating to make further changes easier to review and add a comment what we are actually expecting to happen. While at it, use the actual page size of the ramblock, which defaults to qemu_real_host_page_size for anonymous memory. Further, rename ram_block_populate_pages() to ram_block_populate_read() as well, to make it clearer what we are doing.
In the future, we might want to use MADV_POPULATE_READ to speed up population.
--- migration/ram.c | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)
migration/ram.c +22 -13
@@ -1639,26 +1639,35 @@ out:return ret; } +static inline void populate_read_range (RAMBlock *block,ram_addr_t offset, +ram_addr_t size) +{ +/* +* We read one byte of each page; this will preallocate page tables if +* required and populate the shared zeropage on MAP_PRIVATE anonymous memory +* where no page was populated yet. This might require adaption when +* supporting other mappings, like shmem. +*/ +for (; offset < size; offset += block->page_size) { +char tmp = *((char *)block->host + offset); + +/* Don't optimize the read out */ + asmvolatile ("" :"+r" (tmp)); + } +} +/* -* ram_block_populate_pages: populate memory in the RAM block by reading -* an integer from the beginning of each page. +* ram_block_populate_read: preallocate page tables and populate pages in the +* RAM block by reading a byte of each page. * * Since it's solely used for userfault_fd WP feature, here we just * hardcode page size to qemu_real_host_page_size. * * @block: RAM block to populate */ -static void ram_block_populate_pages (RAMBlock *block) +static void ram_block_populate_read (RAMBlock *block) { -char *ptr = (char *) block->host; - -for (ram_addr_t offset =0 ; offset < block->used_length; - offset += qemu_real_host_page_size) { -char tmp = *(ptr + offset); - -/* Don't optimize the read out */ - asmvolatile ("" :"+r" (tmp)); - } +populate_read_range (block,0 , block->used_length); }/*
@@ -1684,7 +1693,7 @@ void ram_write_tracking_prepare(void)* UFFDIO_WRITEPROTECT_MODE_WP mode setting would silently skip * pages with pte_none() entries in page table. */ -ram_block_populate_pages (block); +ram_block_populate_read (block); } }
Signature · 1 line
2.33.1
[PULL 18/20] migration/ram: Handle RAMBlocks with a RamDiscardManager on background snapshots
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
We already don't ever migrate memory that corresponds to discarded ranges as managed by a RamDiscardManager responsible for the mapped memory region of the RAMBlock.
virtio-mem uses this mechanism to logically unplug parts of a RAMBlock. Right now, we still populate zeropages for the whole usable part of the RAMBlock, which is undesired because:
1. Even populating the shared zeropage will result in memory getting consumed for page tables.
2. Memory backends without a shared zeropage (like hugetlbfs and shmem) will populate an actual, fresh page, resulting in an unintended memory consumption.
Discarded ("logically unplugged") parts have to remain discarded. As these pages are never part of the migration stream, there is no need to track modifications via userfaultfd WP reliably for these parts.
Further, any writes to these ranges by the VM are invalid and the behavior is undefined.
Note that Linux only supports userfaultfd WP on private anonymous memory for now, which usually results in the shared zeropage getting populated. The issue will become more relevant once userfaultfd WP supports shmem and hugetlb.
--- migration/ram.c | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-)
migration/ram.c +36 -2
@@ -1656,6 +1656,17 @@ static inline void populate_read_range(RAMBlock *block, ram_addr_t offset, } } +static inline int populate_read_section (MemoryRegionSection *section, +void *opaque) +{ +const hwaddr size =int128_get64 (section->size); +hwaddr offset = section->offset_within_region; +RAMBlock *block = section->mr->ram_block; + +populate_read_range (block, offset, size); +return 0 ; +} +/* * ram_block_populate_read: preallocate page tables and populate pages in the * RAM block by reading a byte of each page.
@@ -1665,9 +1676,32 @@ static inline void populate_read_range(RAMBlock *block, ram_addr_t offset,* * @block: RAM block to populate */ -static void ram_block_populate_read (RAMBlock *block) +static void ram_block_populate_read (RAMBlock *rb) { -populate_read_range (block,0 , block->used_length); +/* +* Skip populating all pages that fall into a discarded range as managed by +* a RamDiscardManager responsible for the mapped memory region of the +* RAMBlock. Such discarded ("logically unplugged") parts of a RAMBlock +* must not get populated automatically. We don't have to track +* modifications via userfaultfd WP reliably, because these pages will +* not be part of the migration stream either way -- see +* ramblock_dirty_bitmap_exclude_discarded_pages(). +* +* Note: The result is only stable while migrating (precopy/postcopy). +*/ +if (rb->mr &&memory_region_has_ram_discard_manager (rb->mr)) { +RamDiscardManager *rdm =memory_region_get_ram_discard_manager (rb->mr); +MemoryRegionSection section = { + .mr = rb->mr, + .offset_within_region =0 , + .size = rb->mr->size, + }; + +ram_discard_manager_replay_populated (rdm, §ion, + populate_read_section,NULL ); + }else { +populate_read_range (rb,0 , rb->used_length); + } }/*
Signature · 1 line
2.33.1
[PULL 19/20] memory: introduce total_dirty_pages to stat dirty pages
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
introduce global var total_dirty_pages to stat dirty pages along with memory_global_dirty_log_sync.
--- include/exec/ram_addr.h | 9 +++++++++ migration/dirtyrate.c | 7 +++++++ 2 files changed, 16 insertions(+)
include/exec/ram_addr.h +9 -0
@@ -26,6 +26,8 @@#include "exec/ramlist.h" #include "exec/ramblock.h" +extern uint64_t total_dirty_pages; +/** * clear_bmap_size: calculate clear bitmap size *
@@ -373,6 +375,10 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,qatomic_or ( &blocks[DIRTY_MEMORY_MIGRATION ][idx][offset], temp); +if (unlikely ( + global_dirty_tracking &GLOBAL_DIRTY_DIRTY_RATE )) { + total_dirty_pages +=ctpopl (temp); + } }if (tcg_enabled ()) {
@@ -403,6 +409,9 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,for (i =0 ; i < len; i++) {if (bitmap[i] !=0 ) { c =leul_to_cpu (bitmap[i]); +if (unlikely (global_dirty_tracking &GLOBAL_DIRTY_DIRTY_RATE )) { + total_dirty_pages +=ctpopl (c); + } do { j = ctzl(c); c &= ~(1ul << j);
migration/dirtyrate.c +7 -0
@@ -28,6 +28,13 @@#include "sysemu/runstate.h" #include "exec/memory.h" +/* +* total_dirty_pages is procted by BQL and is used +* to stat dirty pages during the period of two +* memory_global_dirty_log_sync +*/ +uint64_t total_dirty_pages; + typedef struct DirtyPageRecord { uint64_t start_pages; uint64_t end_pages;
Signature · 1 line
2.33.1
[PULL 20/20] migration/dirtyrate: implement dirty-bitmap dirtyrate calculation
In reply to Juan QuintelaFrom: Hyman Huang(黄勇) <[email protected]>
introduce dirty-bitmap mode as the third method of calc-dirty-rate. implement dirty-bitmap dirtyrate calculation, which can be used to measuring dirtyrate in the absence of dirty-ring.
introduce "dirty_bitmap:-b" option in hmp calc_dirty_rate to indicate dirty bitmap method should be used for calculation.
--- qapi/migration.json | 6 ++- migration/dirtyrate.c | 112 ++++++++++++++++++++++++++++++++++++++---- hmp-commands.hx | 9 ++-- 3 files changed, 112 insertions(+), 15 deletions(-)
qapi/migration.json +4 -2
@@ -1770,13 +1770,15 @@ # # @page-sampling: calculate dirtyrate by sampling pages. # -# @dirty-ring: calculate dirtyrate by via dirty ring. +# @dirty-ring: calculate dirtyrate by dirty ring. +# +# @dirty-bitmap: calculate dirtyrate by dirty bitmap. # # Since: 6.1 # ## { 'enum': 'DirtyRateMeasureMode', - 'data': ['page-sampling', 'dirty-ring'] } + 'data': ['page-sampling', 'dirty-ring', 'dirty-bitmap'] } ## # @DirtyRateInfo:
migration/dirtyrate.c +103 -9
155 lines not on this page · raw message
hmp-commands.hx +5 -4
@@ -1737,9 +1737,10 @@ ERST { .name = "calc_dirty_rate", - .args_type = "dirty_ring:-r,second:l,sample_pages_per_GB:l?", - .params = "[-r] second [sample_pages_per_GB]", - .help = "start a round of guest dirty rate measurement (using -d to" - "\n\t\t\t specify dirty ring as the method of calculation)", + .args_type = "dirty_ring:-r,dirty_bitmap:-b,second:l,sample_pages_per_GB:l?", + .params = "[-r] [-b] second [sample_pages_per_GB]", + .help = "start a round of guest dirty rate measurement (using -r to" + "\n\t\t\t specify dirty ring as the method of calculation and" + "\n\t\t\t -b to specify dirty bitmap as method of calculation)", .cmd = hmp_calc_dirty_rate, },
Signature · 1 line
2.33.1
[PULL 12/20] virtio-mem: Implement replay_discarded RamDiscardManager callback
In reply to Juan QuintelaFrom: David Hildenbrand <[email protected]>
Implement it similar to the replay_populated callback.
--- hw/virtio/virtio-mem.c | 58 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+)
hw/virtio/virtio-mem.c +58 -0
@@ -228,6 +228,38 @@ static int virtio_mem_for_each_plugged_section(const VirtIOMEM *vmem,return ret; } +static int virtio_mem_for_each_unplugged_section (const VirtIOMEM *vmem, +MemoryRegionSection *s, +void *arg, +virtio_mem_section_cb cb) +{ +unsigned long first_bit, last_bit; +uint64_t offset, size; +int ret =0 ; + + first_bit = s->offset_within_region / vmem->bitmap_size; + first_bit =find_next_zero_bit (vmem->bitmap, vmem->bitmap_size, first_bit); +while (first_bit < vmem->bitmap_size) { +MemoryRegionSection tmp = *s; + + offset = first_bit * vmem->block_size; + last_bit =find_next_bit (vmem->bitmap, vmem->bitmap_size, + first_bit +1 ) -1 ; + size = (last_bit - first_bit +1 ) * vmem->block_size; + +if (!virito_mem_intersect_memory_section (&tmp, offset, size)) { +break ; + } + ret =cb (&tmp, arg); +if (ret) { +break ; + } + first_bit =find_next_zero_bit (vmem->bitmap, vmem->bitmap_size, + last_bit +2 ); + } +return ret; +} +static int virtio_mem_notify_populate_cb (MemoryRegionSection *s,void *arg) {RamDiscardListener *rdl = arg;
@@ -1170,6 +1202,31 @@ static int virtio_mem_rdm_replay_populated(const RamDiscardManager *rdm, virtio_mem_rdm_replay_populated_cb); } +static int virtio_mem_rdm_replay_discarded_cb (MemoryRegionSection *s, +void *arg) +{ +struct VirtIOMEMReplayData *data = arg; + + ((ReplayRamDiscard )data->fn)(s, data->opaque); +return 0 ; +} + +static void virtio_mem_rdm_replay_discarded (const RamDiscardManager *rdm, +MemoryRegionSection *s, +ReplayRamDiscard replay_fn, +void *opaque) +{ +const VirtIOMEM *vmem =VIRTIO_MEM (rdm); +struct VirtIOMEMReplayData data = { + .fn = replay_fn, + .opaque = opaque, + }; + +g_assert (s->mr == &vmem->memdev->mr); +virtio_mem_for_each_unplugged_section (vmem, s, &data, + virtio_mem_rdm_replay_discarded_cb); +} +static void virtio_mem_rdm_register_listener (RamDiscardManager *rdm,RamDiscardListener *rdl,MemoryRegionSection *s)
@@ -1234,6 +1291,7 @@ static void virtio_mem_class_init(ObjectClass *klass, void *data) rdmc->get_min_granularity = virtio_mem_rdm_get_min_granularity; rdmc->is_populated = virtio_mem_rdm_is_populated; rdmc->replay_populated = virtio_mem_rdm_replay_populated; + rdmc->replay_discarded = virtio_mem_rdm_replay_discarded; rdmc->register_listener = virtio_mem_rdm_register_listener; rdmc->unregister_listener = virtio_mem_rdm_unregister_listener; }
Signature · 1 line
2.33.1
Richard Henderson
AppliedQuoted 80 lines · Juan Quintela
The following changes since commit
af531756d25541a1b3b3d9a14e72e7fedd941a2e:Merge remote-tracking branch 'remotes/philmd/tags/renesas-20211030' into staging (2021-10-30 11:31:41 -0700)are available in the Git repository at:
https://github.com/juanquintela/qemu.git tags/migration-20211031-pull-request
for you to fetch changes up to
826b8bc80cb191557a4ce7cf0e155b436d2d1afa:migration/dirtyrate: implement dirty-bitmap dirtyrate calculation (2021-11-01 22:56:44 +0100)----------------------------------------------------------------
Migration Pull requestHi
this includes pending bits of migration patches.
- virtio-mem support by David Hildenbrand
- dirtyrate improvements by Hyman Huang
- fix rdma wrid by Li Zhijian
- dump-guest-memory fixes by Peter XuPleas apply.
Thanks, Juan.
----------------------------------------------------------------
David Hildenbrand (8):
memory: Introducereplay_discardedcallback for RamDiscardManager
virtio-mem: Implementreplay_discardedRamDiscardManager callback migration/ram: Handle RAMBlocks with a RamDiscardManager on the
migration source
virtio-mem: Drop precopy notifier
migration/postcopy: Handle RAMBlocks with a RamDiscardManager on the
destination
migration: Simplify alignment and alignment checks migration/ram: Factor out populating pages readable in
ram_block_populate_pages()
migration/ram: Handle RAMBlocks with a RamDiscardManager on background
snapshotsHyman Huang(黄勇) (6):
KVM: introducedirty_pagesandkvm_dirty_ring_enabled
memory: makeglobal_dirty_trackinga bitmask migration/dirtyrate: introduce struct and adjust DirtyRateStat migration/dirtyrate: adjust order of registering thread migration/dirtyrate: move init step of calculation to main thread migration/dirtyrate: implement dirty-ring dirtyrate calculationHyman Huang(黄勇) (2):
memory: introducetotal_dirty_pagesto stat dirty pages migration/dirtyrate: implement dirty-bitmap dirtyrate calculationLi Zhijian (1):
migration/rdma: Fix out of order wridPeter Xu (3):
migration: Make migration blocker work for snapshots too
migration: Addmigrate_add_blocker_internal()
dump-guest-memory: Block live migrationqapi/migration.json | 48 ++++- include/exec/memory.h | 41 +++- include/exec/ram_addr.h | 13 +- include/hw/core/cpu.h | 1 + include/hw/virtio/virtio-mem.h | 3 - include/migration/blocker.h | 16 ++ include/sysemu/kvm.h | 1 + migration/dirtyrate.h | 21 +- migration/ram.h | 1 + accel/kvm/kvm-all.c | 7 + accel/stubs/kvm-stub.c | 5 + dump/dump.c | 19 ++ hw/i386/xen/xen-hvm.c | 4 +- hw/virtio/virtio-mem.c | 92 ++++++--- migration/dirtyrate.c | 367 ++++++++++++++++++++++++++++++--- migration/migration.c | 30 +--
Quoted 8 lines
migration/postcopy-ram.c | 40 +++- migration/ram.c | 180 ++++++++++++++-- migration/rdma.c | 138 +++++++++---- softmmu/memory.c | 43 +++- hmp-commands.hx | 8 +- migration/trace-events | 2 + softmmu/trace-events | 1 + 23 files changed, 909 insertions(+), 172 deletions(-)
Applied, thanks.
r~