support dirtyrate measurement with dirty bitmap
v2 cover support dirtyrate measurement with dirty bitmap
From: Hyman Huang(黄勇) <[email protected]>
v2:
- drop the DIRTY_MEMORY_DIRTY_RATE dirty bits
- reuse the DIRTY_MEMORY_MIGRATION dirty bits to stat the dirty pages.
- introduce global var DirtyRateIncreasedPages to stat the increased dirty pages
- simplify the logic of calculation. skip the 1'round of log sync unconditionally
changes of this version are based on Peter's advice, like the version 1, it is posted for the sake of RFC. ideally, this patshset may be merged after the commit:
"migration/dirtyrate: implement dirty-ring dirtyrate calculation"
v1:
the dirtyrate measurement implemented by page-sampling originally, it is not accurate in some scenarios, so we have introduced dirty-ring based dirtyrate measurement(maybe it will be merged soon), it fix the accuracy of page-sampling, and more importantly, it is at the granualrity of vcpu.
dirty-ring method can be used when dirty-ring enable, as supplementary, we introduce dirty-bitmap method to calculating dirtyrate when dirty log enable, so that we can also get the accurate dirtyrate if needed in the absence of dirty-ring.
three things has done to implement the measurement:
- introduce a fresh new dirty bits named DIRTY_MEMORY_DIRTY_RATE, which is used to store dirty bitmap after fetching it from kvm. why we do not reuse the existing DIRTY_MEMORY_MIGRATION dirty bits is we do not want to interfere with migration of and let implementation clear, this is also the reason why dirty_memory be split.
DIRTY_MEMORY_DIRTY_RATE dirty bits will be filled when memory_global_dirty_log_sync executed if GLOBAL_DIRTY_DIRTY_RATE bit be set in the global_dirty_tracking flag.
- introduce kvm_get_manual_dirty_log_protect function so that we can probe the protect caps of kvm when calculating.
- implement dirtyrate measurement with dirty bitmap with following step:
1. start the dirty log.
2. probe the protect cap, if KVM_DIRTY_LOG_INITIALLY_SET enable, skip skip the 1'R and do the reset page protection manually, since kvm file bitmap with 1 bits if this cap is enabled.
3. clear the DIRTY_MEMORY_DIRTY_RATE dirty bits, prepare to store the dirty bitmap.
4. start memory_global_dirty_log_sync and fetch dirty bitmap from kvm
5. reap the DIRTY_MEMORY_DIRTY_RATE dirty bits and do the calculation.
this patchset rebases on the commit "migration/dirtyrate: implement dirty-ring dirtyrate calculation", since the above feature has not been merged, so we post this patch for the sake of RFC. ideally, this patshset may be merged after it.
Please, review, thanks !
Best Regards !
Hyman Huang(黄勇) (3):
KVM: introduce kvm_get_manual_dirty_log_protect
memory: introduce DirtyRateIncreasedPages and util function migration/dirtyrate: implement dirty-bitmap dirtyrate calculation
accel/kvm/kvm-all.c | 6 +++ hmp-commands.hx | 9 ++-- include/exec/ram_addr.h | 87 ++++++++++++++++++++++++++---- include/sysemu/kvm.h | 1 + migration/dirtyrate.c | 138 ++++++++++++++++++++++++++++++++++++++++++++---- migration/trace-events | 1 + qapi/migration.json | 6 ++- softmmu/physmem.c | 35 ++++++++++++ 8 files changed, 259 insertions(+), 24 deletions(-)
Signature · 1 line
1.8.3.1
PATCH v2 1/3 KVM: introduce kvm_get_manual_dirty_log_protect
From: Hyman Huang(黄勇) <[email protected]>
introduce kvm_get_manual_dirty_log_protect for measureing dirtyrate via dirty bitmap. calculation of dirtyrate need to sync dirty log and depends on the features of dirty log.
--- accel/kvm/kvm-all.c | 6 ++++++ include/sysemu/kvm.h | 1 + 2 files changed, 7 insertions(+)
accel/kvm/kvm-all.c +6 -0
@@ -245,6 +245,12 @@ int kvm_get_max_memslots(void)return s->nr_slots; } +uint64_t kvm_get_manual_dirty_log_protect (void ) +{ +KVMState *s =KVM_STATE (current_accel ()); +return s->manual_dirty_log_protect; +} +/* Called with KVMMemoryListener.slots_lock held */ static KVMSlot *kvm_get_free_slot (KVMMemoryListener *kml) {
include/sysemu/kvm.h +1 -0
@@ -533,6 +533,7 @@ int kvm_set_one_reg(CPUState *cs, uint64_t id, void *source);int kvm_get_one_reg (CPUState *cs,uint64_t id,void *target);struct ppc_radix_page_info *kvm_get_radix_page_info (void );int kvm_get_max_memslots (void ); +uint64_t kvm_get_manual_dirty_log_protect (void );/* Notify resamplefd for EOI of specific interrupts. */ void kvm_resample_fd_notify (int gsi);
Signature · 1 line
1.8.3.1
PATCH v2 2/3 memory: introduce DirtyRateIncreasedPages and util function
In reply to [email protected]From: Hyman Huang(黄勇) <[email protected]>
introduce DirtyRateIncreasedPages to stat the increased dirty pages during the calculation time along with ramblock_sync_dirty_bitmap.
introduce util functions to setup the DIRTY_MEMORY_MIGRATION dirty bits for the convenience of tracking dirty bitmap when calculating dirtyrate.
--- include/exec/ram_addr.h | 87 ++++++++++++++++++++++++++++++++++++++++++++----- softmmu/physmem.c | 35 ++++++++++++++++++++ 2 files changed, 113 insertions(+), 9 deletions(-)
include/exec/ram_addr.h +78 -9
@@ -26,6 +26,8 @@#include "exec/ramlist.h" #include "exec/ramblock.h" +static uint64_t DirtyRateIncreasedPages =0 ; +/** * clear_bmap_size: calculate clear bitmap size *
@@ -422,6 +424,9 @@ bool cpu_physical_memory_test_and_clear_dirty(ram_addr_t start,ram_addr_t length, unsigned client); +void cpu_physical_memory_dirtyrate_clear_bit (ram_addr_t start, +ram_addr_t length); +DirtyBitmapSnapshot *cpu_physical_memory_snapshot_and_clear_dirty (MemoryRegion *mr,hwaddr offset,hwaddr length,unsigned client);
@@ -449,6 +454,8 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,uint64_t num_dirty =0 ;unsigned long *dest = rb->bmap; +assert (global_dirty_tracking); +/* start address and length is aligned at the start of a word? */ if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (start + rb->offset) &&
@@ -466,12 +473,20 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,for (k = page; k < page + nr; k++) {if (src[idx][offset]) { -unsigned long bits =qatomic_xchg (&src[idx][offset],0 ); -unsigned long new_dirty; - new_dirty = ~dest[k]; - dest[k] |= bits; - new_dirty &= bits; - num_dirty +=ctpopl (new_dirty); +unsigned long bits; +if (global_dirty_tracking &GLOBAL_DIRTY_DIRTY_RATE ) { + bits =qatomic_read (&src[idx][offset]); + DirtyRateIncreasedPages +=ctpopl (bits); + } + +if (global_dirty_tracking &GLOBAL_DIRTY_MIGRATION ) { +unsigned long new_dirty; + bits =qatomic_xchg (&src[idx][offset],0 ); + new_dirty = ~dest[k]; + dest[k] |= bits; + new_dirty &= bits; + num_dirty +=ctpopl (new_dirty); + } }if (++offset >=BITS_TO_LONGS (DIRTY_MEMORY_BLOCK_SIZE )) {
@@ -500,9 +515,15 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb, start + addr + offset, TARGET_PAGE_SIZE, DIRTY_MEMORY_MIGRATION)) { -long k = (start + addr) >>TARGET_PAGE_BITS ; -if (!test_and_set_bit (k, dest)) { - num_dirty++; +if (global_dirty_tracking &GLOBAL_DIRTY_DIRTY_RATE ) { + DirtyRateIncreasedPages++; + } + +if (global_dirty_tracking &GLOBAL_DIRTY_MIGRATION ) { +long k = (start + addr) >>TARGET_PAGE_BITS ; +if (!test_and_set_bit (k, dest)) { + num_dirty++; + } } } }
@@ -510,5 +531,53 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,return num_dirty; } + +static inline +void cpu_physical_memory_dirtyrate_clear_dirty_bits (RAMBlock *rb) +{ +ram_addr_t addr; +ram_addr_t length = rb->used_length; +unsigned long word =BIT_WORD (rb->offset >>TARGET_PAGE_BITS ); + +/* start address and length is aligned at the start of a word? */ +if (((word *BITS_PER_LONG ) <<TARGET_PAGE_BITS ) == rb->offset && + !(length & ((BITS_PER_LONG <<TARGET_PAGE_BITS ) -1 ))) { +int k; +int nr =BITS_TO_LONGS (length >>TARGET_PAGE_BITS ); +unsigned long *const *src; +unsigned long idx = (word *BITS_PER_LONG ) /DIRTY_MEMORY_BLOCK_SIZE ; +unsigned long offset =BIT_WORD ((word *BITS_PER_LONG ) % +DIRTY_MEMORY_BLOCK_SIZE ); + + src =qatomic_rcu_read ( + &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION ])->blocks; + +for (k =0 ; k < nr; k++) { +if (src[idx][offset]) { +qatomic_set (&src[idx][offset],0 ); + } +if (++offset >=BITS_TO_LONGS (DIRTY_MEMORY_BLOCK_SIZE )) { + offset =0 ; + idx++; + } + } + }else { +ram_addr_t offset = rb->offset; + +for (addr =0 ; addr < length; addr +=TARGET_PAGE_SIZE ) { +cpu_physical_memory_dirtyrate_clear_bit (addr + offset, +TARGET_PAGE_SIZE ); + } + } + +return ; +} + +static inline +void cpu_physical_memory_dirtyrate_reset_protect (RAMBlock *rb) +{ +memory_region_clear_dirty_bitmap (rb->mr,0 , rb->used_length); +cpu_physical_memory_dirtyrate_clear_dirty_bits (rb); +}#endif #endif
softmmu/physmem.c +35 -0
@@ -1068,6 +1068,41 @@ bool cpu_physical_memory_test_and_clear_dirty(ram_addr_t start,return dirty; } +void cpu_physical_memory_dirtyrate_clear_bit (ram_addr_t start, +ram_addr_t length) +{ +DirtyMemoryBlocks *blocks; +unsigned long end, page; +RAMBlock *ramblock; + +if (length ==0 ) { +return ; + } + + end =TARGET_PAGE_ALIGN (start + length) >>TARGET_PAGE_BITS ; + page = start >>TARGET_PAGE_BITS ; + + WITH_RCU_READ_LOCK_GUARD() { + blocks = +qatomic_rcu_read (&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION ]); + ramblock =qemu_get_ram_block (start); +/* Range sanity check on the ramblock */ +assert (start >= ramblock->offset && + start + length <= ramblock->offset + ramblock->used_length); +while (page < end) { +unsigned long idx = page /DIRTY_MEMORY_BLOCK_SIZE ; +unsigned long offset = page %DIRTY_MEMORY_BLOCK_SIZE ; +unsigned long num =MIN (end - page, +DIRTY_MEMORY_BLOCK_SIZE - offset); + +clear_bit (num, blocks->blocks[idx]); + page += num; + } + } + +return ; +} +DirtyBitmapSnapshot *cpu_physical_memory_snapshot_and_clear_dirty (MemoryRegion *mr,hwaddr offset,hwaddr length,unsigned client) {
Signature · 1 line
1.8.3.1
Re: [PATCH v2 2/3] memory: introduce DirtyRateIncreasedPages and util function
Quoted 20 lines · On Tue, Jul 13, 2021 at 12:56:51AM +0800, [email protected] wrote:
From: Hyman Huang(黄勇) <[email protected]>
introduce DirtyRateIncreasedPages to stat the increased dirty pages during the calculation time along with
ramblock_sync_dirty_bitmap.introduce util functions to setup the
DIRTY_MEMORY_MIGRATIONdirty bits for the convenience of tracking dirty bitmap when calculating dirtyrate.Signed-off-by Hyman Huang(黄勇) <[email protected]>--- include/exec/ram_addr.h | 87 ++++++++++++++++++++++++++++++++++++++++++++----- softmmu/physmem.c | 35 ++++++++++++++++++++ 2 files changed, 113 insertions(+), 9 deletions(-)include/exec/ram_addr.h +0
-0
include/exec/ram_addr.h line 29
Quoted 4 lines
include/exec/ram_addr.h +1
-0@@ -26,6 +26,8 @@#include "exec/ramlist.h" #include "exec/ramblock.h" +static uint64_t DirtyRateIncreasedPages =0 ;
This looks odd already itself; per my understanding defining a static in a header file will define one variable for each .c file including it. This could lead to each .c file increase its own variable and other .c files read zeros, iiuc.
If we need a global we should define it in a .c file and here we define with "extern" to reference it in other .c files.
Quoted 40 lines
include/exec/ram_addr.h +13
-6+/** * clear_bmap_size: calculate clear bitmap size * @@ -422,6 +424,9 @@ bool cpu_physical_memory_test_and_clear_dirty(ram_addr_t start,ram_addr_t length, unsigned client); +void cpu_physical_memory_dirtyrate_clear_bit (ram_addr_t start, +ram_addr_t length); +DirtyBitmapSnapshot *cpu_physical_memory_snapshot_and_clear_dirty (MemoryRegion *mr,hwaddr offset,hwaddr length,unsigned client);@@ -449,6 +454,8 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,uint64_t num_dirty =0 ;unsigned long *dest = rb->bmap; +assert (global_dirty_tracking); +/* start address and length is aligned at the start of a word? */ if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (start + rb->offset) &&@@ -466,12 +473,20 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,for (k = page; k < page + nr; k++) {if (src[idx][offset]) { -unsigned long bits =qatomic_xchg (&src[idx][offset],0 ); -unsigned long new_dirty; - new_dirty = ~dest[k]; - dest[k] |= bits; - new_dirty &= bits; - num_dirty +=ctpopl (new_dirty); +unsigned long bits; +if (global_dirty_tracking &GLOBAL_DIRTY_DIRTY_RATE ) { + bits =qatomic_read (&src[idx][offset]); + DirtyRateIncreasedPages +=ctpopl (bits); + } + +if (global_dirty_tracking &GLOBAL_DIRTY_MIGRATION ) {
include/exec/ram_addr.h uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb, · line 492
Quoted 10 lines
include/exec/ram_addr.h +7
-0+unsigned long new_dirty; + bits =qatomic_xchg (&src[idx][offset],0 ); + new_dirty = ~dest[k]; + dest[k] |= bits; + new_dirty &= bits; + num_dirty +=ctpopl (new_dirty); + } }if (++offset >=BITS_TO_LONGS (DIRTY_MEMORY_BLOCK_SIZE )) {
Please see my other reply in previous version's cover letter.
IMHO cpu_physical_memory_sync_dirty_bitmap() should be kept untouched. Then below functions are not needed either.
Thanks,
Quoted 122 lines
include/exec/ram_addr.h +57
-3@@ -500,9 +515,15 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb, start + addr + offset, TARGET_PAGE_SIZE, DIRTY_MEMORY_MIGRATION)) { -long k = (start + addr) >>TARGET_PAGE_BITS ; -if (!test_and_set_bit (k, dest)) { - num_dirty++; +if (global_dirty_tracking &GLOBAL_DIRTY_DIRTY_RATE ) { + DirtyRateIncreasedPages++; + } + +if (global_dirty_tracking &GLOBAL_DIRTY_MIGRATION ) { +long k = (start + addr) >>TARGET_PAGE_BITS ; +if (!test_and_set_bit (k, dest)) { + num_dirty++; + } } } }@@ -510,5 +531,53 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,return num_dirty; } + +static inline +void cpu_physical_memory_dirtyrate_clear_dirty_bits (RAMBlock *rb) +{ +ram_addr_t addr; +ram_addr_t length = rb->used_length; +unsigned long word =BIT_WORD (rb->offset >>TARGET_PAGE_BITS ); + +/* start address and length is aligned at the start of a word? */ +if (((word *BITS_PER_LONG ) <<TARGET_PAGE_BITS ) == rb->offset && + !(length & ((BITS_PER_LONG <<TARGET_PAGE_BITS ) -1 ))) { +int k; +int nr =BITS_TO_LONGS (length >>TARGET_PAGE_BITS ); +unsigned long *const *src; +unsigned long idx = (word *BITS_PER_LONG ) /DIRTY_MEMORY_BLOCK_SIZE ; +unsigned long offset =BIT_WORD ((word *BITS_PER_LONG ) % +DIRTY_MEMORY_BLOCK_SIZE ); + + src =qatomic_rcu_read ( + &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION ])->blocks; + +for (k =0 ; k < nr; k++) { +if (src[idx][offset]) { +qatomic_set (&src[idx][offset],0 ); + } +if (++offset >=BITS_TO_LONGS (DIRTY_MEMORY_BLOCK_SIZE )) { + offset =0 ; + idx++; + } + } + }else { +ram_addr_t offset = rb->offset; + +for (addr =0 ; addr < length; addr +=TARGET_PAGE_SIZE ) { +cpu_physical_memory_dirtyrate_clear_bit (addr + offset, +TARGET_PAGE_SIZE ); + } + } + +return ; +} + +static inline +void cpu_physical_memory_dirtyrate_reset_protect (RAMBlock *rb) +{ +memory_region_clear_dirty_bitmap (rb->mr,0 , rb->used_length); +cpu_physical_memory_dirtyrate_clear_dirty_bits (rb); +}#endif #endif softmmu/physmem.c +35
-0@@ -1068,6 +1068,41 @@ bool cpu_physical_memory_test_and_clear_dirty(ram_addr_t start,return dirty; } +void cpu_physical_memory_dirtyrate_clear_bit (ram_addr_t start, +ram_addr_t length) +{ +DirtyMemoryBlocks *blocks; +unsigned long end, page; +RAMBlock *ramblock; + +if (length ==0 ) { +return ; + } + + end =TARGET_PAGE_ALIGN (start + length) >>TARGET_PAGE_BITS ; + page = start >>TARGET_PAGE_BITS ; + + WITH_RCU_READ_LOCK_GUARD() { + blocks = +qatomic_rcu_read (&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION ]); + ramblock =qemu_get_ram_block (start); +/* Range sanity check on the ramblock */ +assert (start >= ramblock->offset && + start + length <= ramblock->offset + ramblock->used_length); +while (page < end) { +unsigned long idx = page /DIRTY_MEMORY_BLOCK_SIZE ; +unsigned long offset = page %DIRTY_MEMORY_BLOCK_SIZE ; +unsigned long num =MIN (end - page, +DIRTY_MEMORY_BLOCK_SIZE - offset); + +clear_bit (num, blocks->blocks[idx]); + page += num; + } + } + +return ; +} +DirtyBitmapSnapshot *cpu_physical_memory_snapshot_and_clear_dirty (MemoryRegion *mr,hwaddr offset,hwaddr length,unsigned client) {Signature · 1 line
1.8.3.1
Signature · 1 line
Peter Xu
PATCH v2 3/3 migration/dirtyrate: implement dirty-bitmap dirtyrate calculation
In reply to [email protected]From: Hyman Huang(黄勇) <[email protected]>
introduce dirty-bitmap mode as the third method of calc-dirty-rate. implement dirty-bitmap dirtyrate calculation, which can be used to measuring dirtyrate in the absence of dirty-ring.
introduce "dirty_bitmap:-b" option in hmp calc_dirty_rate to indicate dirty bitmap method should be used for calculation.
--- hmp-commands.hx | 9 ++-- migration/dirtyrate.c | 134 +++++++++++++++++++++++++++++++++++++++++++++---- migration/trace-events | 1 + qapi/migration.json | 6 ++- 4 files changed, 135 insertions(+), 15 deletions(-)
hmp-commands.hx +5 -4
@@ -1738,9 +1738,10 @@ ERST { .name = "calc_dirty_rate", - .args_type = "dirty_ring:-r,second:l,sample_pages_per_GB:l?", - .params = "[-r] second [sample_pages_per_GB]", - .help = "start a round of guest dirty rate measurement (using -d to" - "\n\t\t\t specify dirty ring as the method of calculation)", + .args_type = "dirty_ring:-r,dirty_bitmap:-b,second:l,sample_pages_per_GB:l?", + .params = "[-r] [-b] second [sample_pages_per_GB]", + .help = "start a round of guest dirty rate measurement (using -r to" + "\n\t\t\t specify dirty ring as the method of calculation and" + "\n\t\t\t -b to specify dirty bitmap as method of calculation)", .cmd = hmp_calc_dirty_rate, },
migration/dirtyrate.c +125 -9
177 lines not on this page · raw message
migration/trace-events +1 -0
@@ -333,6 +333,7 @@ skip_sample_ramblock(const char *idstr, uint64_t ramblock_size) "ramblock name: find_page_matched(const char *idstr) "ramblock %s addr or size changed" dirtyrate_calculate(int64_t dirtyrate) "dirty rate: %" PRIi64 " MB/s" dirtyrate_do_calculate_vcpu(int idx, uint64_t rate) "vcpu[%d]: %"PRIu64 " MB/s" +dirtyrate_do_calculate_bitmap(uint64_t pages, int64_t time, uint64_t rate) "%"PRIu64 " pages has increased in %"PRIi64 " s, rate %"PRIu64 "MB/s" # block.c migration_block_init_shared(const char *blk_device_name) "Start migration for %s with shared base image"
qapi/migration.json +4 -2
@@ -1747,13 +1747,15 @@ # # @page-sampling: calculate dirtyrate by sampling pages. # -# @dirty-ring: calculate dirtyrate by via dirty ring. +# @dirty-ring: calculate dirtyrate by dirty ring. +# +# @dirty-bitmap: calculate dirtyrate by dirty bitmap. # # Since: 6.1 # ## { 'enum': 'DirtyRateMeasureMode', - 'data': ['page-sampling', 'dirty-ring'] } + 'data': ['page-sampling', 'dirty-ring', 'dirty-bitmap'] } ## # @DirtyRateInfo:
Signature · 1 line
1.8.3.1