KVM: ppc64: Enable ring-based dirty memory tracking
Subsystems (4)
PATCH 1/1 KVM: ppc64: Enable ring-based dirty memory tracking
- Enable CONFIG_HAVE_KVM_DIRTY_RING_ACQ_REL as ppc64 is weakly ordered.
- Enable CONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAP because the kvmppc_xive_native_set_attr is called in the context of an ioctl syscall and will call kvmppc_xive_native_eq_sync for setting the KVM_DEV_XIVE_EQ_SYNC attribute which will call mark_dirty_page() when there isn't a running vcpu. Implemented the kvm_arch_allow_write_without_running_vcpu to always return true to allow mark_page_dirty_in_slot to mark the page dirty in the memslot->dirty_bitmap in this case.
- Set KVM_DIRTY_LOG_PAGE_OFFSET for the ring buffer's physical page offset.
- Implement the kvm_arch_mmu_enable_log_dirty_pt_masked function required for the generic KVM code to call.
- Add a check to kvmppc_vcpu_run_hv for checking whether the dirty ring is soft full.
- Implement the kvm_arch_flush_remote_tlbs_memslot function to support the CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT config option.
On testing with live migration it was found that there is around 150-180 ms improvment in overall migration time with this patch.
--- Documentation/virt/kvm/api.rst | 2 +- arch/powerpc/include/uapi/asm/kvm.h | 2 ++ arch/powerpc/kvm/Kconfig | 2 ++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 42 +++++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 3 +++ include/linux/kvm_dirty_ring.h | 5 ++++ 6 files changed, 55 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +1 -1
@@ -8114,7 +8114,7 @@ regardless of what has actually been exposed through the CPUID leaf. 8.29 KVM_CAP_DIRTY_LOG_RING/KVM_CAP_DIRTY_LOG_RING_ACQ_REL ---------------------------------------------------------- -:Architectures: x86, arm64 +:Architectures: x86, arm64, ppc64 :Parameters: args[0] - size of the dirty log ring KVM is capable of tracking dirty memory using ring buffers that are
arch/powerpc/include/uapi/asm/kvm.h +2 -0
@@ -33,6 +33,8 @@/* Not always available, but if it is, this is the correct offset. */ #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 +struct kvm_regs {__u64 pc;__u64 cr;
arch/powerpc/kvm/Kconfig +2 -0
@@ -26,6 +26,8 @@ config KVM select IRQ_BYPASS_MANAGER select HAVE_KVM_IRQ_BYPASS select INTERVAL_TREE + select HAVE_KVM_DIRTY_RING_ACQ_REL + select NEED_KVM_DIRTY_RING_WITH_BITMAP config KVM_BOOK3S_HANDLER bool
arch/powerpc/kvm/book3s_64_mmu_hv.c +42 -0
45 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +3 -0
@@ -4804,6 +4804,9 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)return -EINTR ; } +if (kvm_dirty_ring_check_request (vcpu)) +return 0 ; +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM /* * Don't allow entry with a suspended transaction, because
include/linux/kvm_dirty_ring.h +5 -0
@@ -69,6 +69,11 @@ static inline void kvm_dirty_ring_free(struct kvm_dirty_ring *ring) { } +static inline bool kvm_dirty_ring_check_request (struct kvm_vcpu *vcpu) +{ +return false; +} +#else /* CONFIG_HAVE_KVM_DIRTY_RING */ int kvm_cpu_dirty_log_size (void );
Signature · 1 line
2.39.2
Hi Nick/Gavin/Everyone,
Quoted 23 lines · Kautuk Consul
- Enable
CONFIG_HAVE_KVM_DIRTY_RING_ACQ_RELas ppc64 is weakly ordered.
- EnableCONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAPbecause thekvmppc_xive_native_set_attris called in the context of an ioctl syscall and will callkvmppc_xive_native_eq_syncfor setting theKVM_DEV_XIVE_EQ_SYNCattribute which will callmark_dirty_page()when there isn't a running vcpu. Implemented thekvm_arch_allow_write_without_running_vcputo always return true to allowmark_page_dirty_in_slotto mark the page dirty in thememslot->dirty_bitmapin this case.
- SetKVM_DIRTY_LOG_PAGE_OFFSETfor the ring buffer's physical page offset.
- Implement thekvm_arch_mmu_enable_log_dirty_pt_maskedfunction required for the generic KVM code to call.
- Add a check tokvmppc_vcpu_run_hvfor checking whether the dirty ring is soft full.
- Implement thekvm_arch_flush_remote_tlbs_memslotfunction to support theCONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECTconfig option.On testing with live migration it was found that there is around 150-180 ms improvment in overall migration time with this patch.
Signed-off-by Kautuk Consul <[email protected]>
Quoted 9 lines
--- Documentation/virt/kvm/api.rst | 2 +- arch/powerpc/include/uapi/asm/kvm.h | 2 ++ arch/powerpc/kvm/Kconfig | 2 ++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 42 +++++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 3 +++ include/linux/kvm_dirty_ring.h | 5 ++++ 6 files changed, 55 insertions(+), 1 deletion(-)
Any review comments on this ?
Quoted 122 lines
Documentation/virt/kvm/api.rst +1
-1@@ -8114,7 +8114,7 @@ regardless of what has actually been exposed through the CPUID leaf. 8.29 KVM_CAP_DIRTY_LOG_RING/KVM_CAP_DIRTY_LOG_RING_ACQ_REL ---------------------------------------------------------- -:Architectures: x86, arm64 +:Architectures: x86, arm64, ppc64 :Parameters: args[0] - size of the dirty log ring KVM is capable of tracking dirty memory using ring buffers that arearch/powerpc/include/uapi/asm/kvm.h +2
-0@@ -33,6 +33,8 @@/* Not always available, but if it is, this is the correct offset. */ #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 +struct kvm_regs {__u64 pc;__u64 cr;arch/powerpc/kvm/Kconfig +2
-0@@ -26,6 +26,8 @@ config KVM select IRQ_BYPASS_MANAGER select HAVE_KVM_IRQ_BYPASS select INTERVAL_TREE + select HAVE_KVM_DIRTY_RING_ACQ_REL + select NEED_KVM_DIRTY_RING_WITH_BITMAP config KVM_BOOK3S_HANDLER boolarch/powerpc/kvm/book3s_64_mmu_hv.c +42
-0@@ -2147,3 +2147,45 @@ void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu) vcpu->arch.hflags |=BOOK3S_HFLAG_SLB ; } + +/* +* kvm_arch_mmu_enable_log_dirty_pt_masked - enable dirty logging for selected +* dirty pages. +* +* It write protects selected pages to enable dirty logging for them. +*/ +void kvm_arch_mmu_enable_log_dirty_pt_masked (struct kvm *kvm, +struct kvm_memory_slot *slot, +gfn_t gfn_offset, +unsigned long mask) +{ +phys_addr_t base_gfn = slot->base_gfn + gfn_offset; +phys_addr_t start = (base_gfn +__ffs (mask)) <<PAGE_SHIFT ; +phys_addr_t end = (base_gfn +__fls (mask) +1 ) <<PAGE_SHIFT ; + +while (start < end) { +pte_t *ptep; +unsigned int shift; + + ptep =find_kvm_secondary_pte (kvm, start, &shift); + + *ptep =__pte (pte_val (*ptep) & ~(_PAGE_WRITE)); + + start +=PAGE_SIZE ; + } +} + +#ifdef CONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAP +bool kvm_arch_allow_write_without_running_vcpu (struct kvm *kvm) +{ +return true; +} +#endif + +#ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT +void kvm_arch_flush_remote_tlbs_memslot (struct kvm *kvm, +const struct kvm_memory_slot *memslot) +{ +kvm_flush_remote_tlbs (kvm); +} +#endif arch/powerpc/kvm/book3s_hv.c +3
-0@@ -4804,6 +4804,9 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)return -EINTR ; } +if (kvm_dirty_ring_check_request (vcpu)) +return 0 ; +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM /* * Don't allow entry with a suspended transaction, because include/linux/kvm_dirty_ring.h +5
-0@@ -69,6 +69,11 @@ static inline void kvm_dirty_ring_free(struct kvm_dirty_ring *ring) { } +static inline bool kvm_dirty_ring_check_request (struct kvm_vcpu *vcpu) +{ +return false; +} +#else /* CONFIG_HAVE_KVM_DIRTY_RING */ int kvm_cpu_dirty_log_size (void );Signature · 1 line
2.39.2
Hi Everyone,
Quoted 19 lines · Kautuk Consul
- Enable
CONFIG_HAVE_KVM_DIRTY_RING_ACQ_RELas ppc64 is weakly ordered.
- EnableCONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAPbecause thekvmppc_xive_native_set_attris called in the context of an ioctl syscall and will callkvmppc_xive_native_eq_syncfor setting theKVM_DEV_XIVE_EQ_SYNCattribute which will callmark_dirty_page()when there isn't a running vcpu. Implemented thekvm_arch_allow_write_without_running_vcputo always return true to allowmark_page_dirty_in_slotto mark the page dirty in thememslot->dirty_bitmapin this case.
- SetKVM_DIRTY_LOG_PAGE_OFFSETfor the ring buffer's physical page offset.
- Implement thekvm_arch_mmu_enable_log_dirty_pt_maskedfunction required for the generic KVM code to call.
- Add a check tokvmppc_vcpu_run_hvfor checking whether the dirty ring is soft full.
- Implement thekvm_arch_flush_remote_tlbs_memslotfunction to support theCONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECTconfig option.
Quoted 4 lines
On testing with live migration it was found that there is around 150-180 ms improvment in overall migration time with this patch.
Signed-off-by Kautuk Consul <[email protected]>
Can someone review this ?
On 8/6/23 10:34 pm, Kautuk Consul wrote:
Need at least a little context in the commit message itself:
"Enable ring-based dirty memory tracking on ppc64:"
Quoted 10 lines
- Enable
CONFIG_HAVE_KVM_DIRTY_RING_ACQ_RELas ppc64 is weakly ordered.
- EnableCONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAPbecause thekvmppc_xive_native_set_attris called in the context of an ioctl syscall and will callkvmppc_xive_native_eq_syncfor setting theKVM_DEV_XIVE_EQ_SYNCattribute which will callmark_dirty_page()when there isn't a running vcpu. Implemented thekvm_arch_allow_write_without_running_vcputo always return true to allowmark_page_dirty_in_slotto mark the page dirty in thememslot->dirty_bitmapin this case.
Should kvm_arch_allow_write_without_running_vcpu() only return true in the context of kvmppc_xive_native_eq_sync()?
Quoted 82 lines
- Set
KVM_DIRTY_LOG_PAGE_OFFSETfor the ring buffer's physical page offset.
- Implement thekvm_arch_mmu_enable_log_dirty_pt_maskedfunction required for the generic KVM code to call.
- Add a check tokvmppc_vcpu_run_hvfor checking whether the dirty ring is soft full.
- Implement thekvm_arch_flush_remote_tlbs_memslotfunction to support theCONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECTconfig option.On testing with live migration it was found that there is around 150-180 ms improvment in overall migration time with this
Signed-off-by Kautuk Consul <[email protected]>--- Documentation/virt/kvm/api.rst | 2 +- arch/powerpc/include/uapi/asm/kvm.h | 2 ++ arch/powerpc/kvm/Kconfig | 2 ++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 42 +++++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 3 +++ include/linux/kvm_dirty_ring.h | 5 ++++ 6 files changed, 55 insertions(+), 1 deletion(-)Documentation/virt/kvm/api.rst +1
-1@@ -8114,7 +8114,7 @@ regardless of what has actually been exposed through the CPUID leaf. 8.29 KVM_CAP_DIRTY_LOG_RING/KVM_CAP_DIRTY_LOG_RING_ACQ_REL ---------------------------------------------------------- -:Architectures: x86, arm64 +:Architectures: x86, arm64, ppc64 :Parameters: args[0] - size of the dirty log ring KVM is capable of tracking dirty memory using ring buffers that arearch/powerpc/include/uapi/asm/kvm.h +2
-0@@ -33,6 +33,8 @@/* Not always available, but if it is, this is the correct offset. */ #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 +struct kvm_regs {__u64 pc;__u64 cr;arch/powerpc/kvm/Kconfig +2
-0@@ -26,6 +26,8 @@ config KVM select IRQ_BYPASS_MANAGER select HAVE_KVM_IRQ_BYPASS select INTERVAL_TREE + select HAVE_KVM_DIRTY_RING_ACQ_REL + select NEED_KVM_DIRTY_RING_WITH_BITMAP config KVM_BOOK3S_HANDLER boolarch/powerpc/kvm/book3s_64_mmu_hv.c +13
-0@@ -2147,3 +2147,45 @@ void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu) vcpu->arch.hflags |=BOOK3S_HFLAG_SLB ; } + +/* +* kvm_arch_mmu_enable_log_dirty_pt_masked - enable dirty logging for selected +* dirty pages. +* +* It write protects selected pages to enable dirty logging for them. +*/ +void kvm_arch_mmu_enable_log_dirty_pt_masked (struct kvm *kvm, +struct kvm_memory_slot *slot, +gfn_t gfn_offset, +unsigned long mask) +{ +phys_addr_t base_gfn = slot->base_gfn + gfn_offset;
arch/powerpc/kvm/book3s_64_mmu_hv.c void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu) · line 2172
Quoted 10 lines
arch/powerpc/kvm/book3s_64_mmu_hv.c +10
-0+phys_addr_t start = (base_gfn +__ffs (mask)) <<PAGE_SHIFT ; +phys_addr_t end = (base_gfn +__fls (mask) +1 ) <<PAGE_SHIFT ; + +while (start < end) { +pte_t *ptep; +unsigned int shift; + + ptep =find_kvm_secondary_pte (kvm, start, &shift); + + *ptep =__pte (pte_val (*ptep) & ~(_PAGE_WRITE));
On rpt I think you'd need to use kvmppc_radix_update_pte()?
Quoted 48 lines
arch/powerpc/kvm/book3s_64_mmu_hv.c +18
-0+ + start += PAGE_SIZE > + } +} + +#ifdef CONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAP +bool kvm_arch_allow_write_without_running_vcpu (struct kvm *kvm) +{ +return true; +} +#endif + +#ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT +void kvm_arch_flush_remote_tlbs_memslot (struct kvm *kvm, +const struct kvm_memory_slot *memslot) +{ +kvm_flush_remote_tlbs (kvm); +} +#endif arch/powerpc/kvm/book3s_hv.c +3
-0@@ -4804,6 +4804,9 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)return -EINTR ; } +if (kvm_dirty_ring_check_request (vcpu)) +return 0 ; +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM /* * Don't allow entry with a suspended transaction, because include/linux/kvm_dirty_ring.h +5
-0@@ -69,6 +69,11 @@ static inline void kvm_dirty_ring_free(struct kvm_dirty_ring *ring) { } +static inline bool kvm_dirty_ring_check_request (struct kvm_vcpu *vcpu) +{ +return false; +} + > #else /* CONFIG_HAVE_KVM_DIRTY_RING */int kvm_cpu_dirty_log_size (void );
Hi Jordan,
Quoted 7 lines · Jordan Niethe
On 8/6/23 10:34 pm, Kautuk Consul wrote:
Need at least a little context in the commit message itself:
"Enable ring-based dirty memory tracking on ppc64:"
Sure will take this in the v2 patch.
Quoted 12 lines
Quoted 11 lines
- Enable
CONFIG_HAVE_KVM_DIRTY_RING_ACQ_RELas ppc64 is weakly ordered.
- EnableCONFIG_NEED_KVM_DIRTY_RING_WITH_BITMAPbecause thekvmppc_xive_native_set_attris called in the context of an ioctl syscall and will callkvmppc_xive_native_eq_syncfor setting theKVM_DEV_XIVE_EQ_SYNCattribute which will callmark_dirty_page()when there isn't a running vcpu. Implemented thekvm_arch_allow_write_without_running_vcputo always return true to allowmark_page_dirty_in_slotto mark the page dirty in thememslot->dirty_bitmapin this case.
Quoted 2 lines
Should
kvm_arch_allow_write_without_running_vcpu()only return true in the context ofkvmppc_xive_native_eq_sync()?
Not required. Reason is: kvm_arch_allow_write_without_running_vcpu() is anyway used only for avoiding the WARN_ON_ONCE in mark_page_dirty_in_slot(). The memslot->dirty_bitmap in mark_page_dirty_in_slot() will be anyway used only when the KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP is set and the vcpu returned by kvm_get_running_vcpu() is NULL which is what happens only when kvmppc_xive_native_eq_sync is called via the ioctl syscall I mentioned.
Quoted 2 lines
arch/powerpc/kvm/book3s_64_mmu_hv.c void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu) · line 2172
Quoted 1 line
arch/powerpc/kvm/book3s_64_mmu_hv.c +1
-0+ *ptep =__pte (pte_val (*ptep) & ~(_PAGE_WRITE));On rpt I think you'd need to use
kvmppc_radix_update_pte()?
Sure. I'll add a check for radix_enabled() and call kvmppc_radix_update_pte() or a similar function in the v2 patch for this.