the arm cache coherency cluster
Subsystems (4)
In reply to this message I'll send two series' one for KVM and one for QEMU. The two series' are their respective component complements, and attempt to implement cache coherency for arm guests using emulated devices, where the emulator (qemu) uses cached memory for the device memory, but the guest uses uncached - as device memory is generally used. Right now I've just focused on VGA vram.
This approach starts as the "add a new memslot flag" approach, and then turns into the "make qemu do some cache maintenance" approach with the final patch of each series (6/6). It stops short of the "add syscalls..." approach. Below is a summary of all the approaches discussed so far, to my knowledge.
"MAIR manipulating"
Posted[1] by Ard. Works. No performance degradation. Potential issues with device assignment and the guest getting confused.
"add a new memslot flag"
This posting (not counting patches 6/6). Works. Huge performance degradation.
"make qemu do some cache maintenance"
This posting (patches 6/6). We can only do so much in qemu without syscalls. This series does what it can. Almost works, probably could work, after playing 'find the missing flush'. This approach still requires the new memslot flag, as userspace can't invalidate the cache, only clean, or clean+invalidate. No noticeable performance degradation.
"add syscalls to make qemu do all cache maintenance" Variant 1: implement as kvm ioctls - to avoid trying to get
syscalls into the general kernel
Variant 2: add real syscalls, or maybe just ARM private SWIs
like __ARM_NR_cacheflush
This approach should work, and if we add an invalidate syscall, then we shouldn't need any kvm changes at all, i.e. no need for the memslot flag. I haven't experimented with this yet, but I'm starting to like the idea of variant 2, with a private SWI, so will try to pull something together soon for that.
"describe the problematic memory as cached to the guest" Not an ideal solution for virt. Could maybe be workable as a quirk for a specific device though.
re: $SUBJECT; Here 'cluster' is defined by the urban dictionary.
[1] http://thread.gmane.org/gmane.comp.emulators.kvm.arm.devel/34/
cover flush/invalidate on entry/exit
Userspace flags memory regions as incoherent and kvm flushes/ invalidates those regions on entry/exit from userspace.
Result before patch 6/6: restores coherency, waaaaay toooo sloooooow Result with patch 6/6: fast again - well, we removed the code...
Andrew Jones (6):
kvm: promote KVM_MEMSLOT_INCOHERENT to uapi
KVM: Introduce incoherent cache maintenance API
KVM: ARM: change __coherent_cache_guest_page interface
KVM: ARM: extend __coherent_cache_guest_page
KVM: ARM: implement kvm_*_incoherent_memory_regions
KVM: ARM: no need for kvm_arch_flush_incoherent
arch/arm/include/asm/kvm_mmu.h | 12 +++---- arch/arm/include/uapi/asm/kvm.h | 1 + arch/arm/kvm/arm.c | 4 +++ arch/arm/kvm/mmu.c | 72 +++++++++++++++++++++++++++++++++++++-- arch/arm64/include/asm/kvm_mmu.h | 12 ++++--- arch/arm64/include/uapi/asm/kvm.h | 1 + include/linux/kvm_host.h | 15 +++++++- include/uapi/linux/kvm.h | 1 + virt/kvm/kvm_main.c | 45 +++++++++++++++++++++++- 9 files changed, 147 insertions(+), 16 deletions(-)
Signature · 1 line
1.8.3.1
RFC PATCH 1/6 kvm: promote KVM_MEMSLOT_INCOHERENT to uapi
--- arch/arm/kvm/mmu.c | 9 +++++++-- include/linux/kvm_host.h | 1 - include/uapi/linux/kvm.h | 1 + 3 files changed, 8 insertions(+), 3 deletions(-)
arch/arm/kvm/mmu.c +7 -2
@@ -1260,7 +1260,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,if (!hugetlb && !force_pte) hugetlb =transparent_hugepage_adjust (&pfn, &fault_ipa); - fault_ipa_uncached = memslot->flags &KVM_MEMSLOT_INCOHERENT ; + fault_ipa_uncached = memslot->flags &KVM_MEM_INCOHERENT ;if (hugetlb) {pmd_t new_pmd =pfn_pmd (pfn, mem_type);
@@ -1784,15 +1784,20 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,int kvm_arch_create_memslot (struct kvm *kvm,struct kvm_memory_slot *slot,unsigned long npages) { +#if 1 /* * Readonly memslots are not incoherent with the caches by definition, * but in practice, they are used mostly to emulate ROMs or NOR flashes * that the guest may consider devices and hence map as uncached. * To prevent incoherency issues in these cases, tag all readonly * regions as incoherent. +* +* This heuristic can be removed after userspace has been updated to +* use KVM_MEM_INCOHERENT on readonly regions when necessary. */ if (slot->flags &KVM_MEM_READONLY ) - slot->flags |=KVM_MEMSLOT_INCOHERENT ; + slot->flags |=KVM_MEM_INCOHERENT ; +#endif return 0 ; }
include/linux/kvm_host.h +0 -1
@@ -43,7 +43,6 @@* include/linux/kvm_h. */ #define KVM_MEMSLOT_INVALID (1UL << 16) -#define KVM_MEMSLOT_INCOHERENT (1UL << 17)/* Two fragments for cross MMIO pages. */ #define KVM_MAX_MMIO_FRAGMENTS 2
include/uapi/linux/kvm.h +1 -0
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {*/ #define KVM_MEM_LOG_DIRTY_PAGES (1UL << 0)#define KVM_MEM_READONLY (1UL << 1) +#define KVM_MEM_INCOHERENT (1UL << 2)/* for KVM_IRQ_LINE */ struct kvm_irq_level {
Signature · 1 line
1.8.3.1
RFC PATCH 2/6 KVM: Introduce incoherent cache maintenance API
In reply to Andrew JonesAdd two new memslot functions to the API
kvm_flush_incoherent_memory_regions
flush all KVM_MEM_INCOHERENT memslot addresses
kvm_invalidate_incoherent_memory_regions
invalidate all KVM_MEM_INCOHERENT memslot addresses
--- include/linux/kvm_host.h | 14 ++++++++++++++ virt/kvm/kvm_main.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 58 insertions(+), 1 deletion(-)
include/linux/kvm_host.h +14 -0
@@ -517,6 +517,8 @@ int kvm_set_memory_region(struct kvm *kvm, struct kvm_userspace_memory_region *mem);int __kvm_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem); +void kvm_flush_incoherent_memory_regions (struct kvm *kvm); +void kvm_invalidate_incoherent_memory_regions (struct kvm *kvm);void kvm_arch_free_memslot (struct kvm *kvm,struct kvm_memory_slot *free,struct kvm_memory_slot *dont); int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
@@ -705,6 +707,18 @@ static inline bool kvm_arch_has_noncoherent_dma(struct kvm *kvm) }#endif +#ifdef __KVM_HAVE_INCOHERENT_MEM +void kvm_arch_flush_incoherent (struct kvm *kvm,struct kvm_memory_slot *slot); +void kvm_arch_invalidate_incoherent (struct kvm *kvm,struct kvm_memory_slot *slot); +#else +void kvm_arch_flush_incoherent (struct kvm *kvm,struct kvm_memory_slot *slot) +{ +} +void kvm_arch_invalidate_incoherent (struct kvm *kvm,struct kvm_memory_slot *slot) +{ +} +#endif +static inline wait_queue_head_t *kvm_arch_vcpu_wq (struct kvm_vcpu *vcpu) {#ifdef __KVM_HAVE_ARCH_WQP
virt/kvm/kvm_main.c +44 -1
57 lines not on this page · raw message
Signature · 1 line
1.8.3.1
RFC PATCH 3/6 KVM: ARM: change __coherent_cache_guest_page interface
In reply to Andrew JonesRemove the vcpu parameter. We can do this by doing the same query in the caller of __coherent_cache_guest_page, and then folding the result into its ipa_uncached parameter, which we rename to need_flush.
A later patch will add a new caller for __coherent_cache_guest_page that does not have a vcpu parameter.
--- arch/arm/include/asm/kvm_mmu.h | 7 ++----- arch/arm/kvm/mmu.c | 3 ++- arch/arm64/include/asm/kvm_mmu.h | 7 +++---- 3 files changed, 7 insertions(+), 10 deletions(-)
arch/arm/include/asm/kvm_mmu.h +2 -5
@@ -183,9 +183,8 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)return (vcpu->arch.cp15[c1_SCTLR] &0b101 ) ==0b101 ; } -static inline void __coherent_cache_guest_page (struct kvm_vcpu *vcpu,pfn_t pfn, -unsigned long size, -bool ipa_uncached) +static inline void __coherent_cache_guest_page (pfn_t pfn,unsigned long size, +bool need_flush) {/* * If we are going to insert an instruction page and the icache is
@@ -205,8 +204,6 @@ static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,* and iterate over the range. */ -bool need_flush = !vcpu_has_cache_enabled (vcpu) || ipa_uncached; -VM_BUG_ON (size &PAGE_MASK );if (!need_flush && !icache_is_pipt ())
arch/arm/kvm/mmu.c +2 -1
@@ -1157,7 +1157,8 @@ void kvm_arch_mmu_write_protect_pt_masked(struct kvm *kvm,static void coherent_cache_guest_page (struct kvm_vcpu *vcpu,pfn_t pfn,unsigned long size,bool uncached) { -__coherent_cache_guest_page (vcpu, pfn, size, uncached); +bool need_flush = uncached || !vcpu_has_cache_enabled (vcpu); +__coherent_cache_guest_page (pfn, size, need_flush); } static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
arch/arm64/include/asm/kvm_mmu.h +3 -4
@@ -264,13 +264,12 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)return (vcpu_sys_reg (vcpu,SCTLR_EL1 ) &0b101 ) ==0b101 ; } -static inline void __coherent_cache_guest_page (struct kvm_vcpu *vcpu,pfn_t pfn, -unsigned long size, -bool ipa_uncached) +static inline void __coherent_cache_guest_page (pfn_t pfn,unsigned long size, +bool need_flush) {void *va =page_address (pfn_to_page (pfn)); -if (!vcpu_has_cache_enabled (vcpu) || ipa_uncached) +if (need_flush)kvm_flush_dcache_to_poc (va, size);if (!icache_is_aliasing ()) {/* PIPT */
Signature · 1 line
1.8.3.1
RFC PATCH 4/6 KVM: ARM: extend __coherent_cache_guest_page
In reply to Andrew JonesAlso support only invalidating, rather than always invalidate+clear.
--- arch/arm/include/asm/kvm_mmu.h | 7 +++++-- arch/arm/kvm/mmu.c | 2 +- arch/arm64/include/asm/kvm_mmu.h | 7 +++++-- 3 files changed, 11 insertions(+), 5 deletions(-)
arch/arm/include/asm/kvm_mmu.h +5 -2
@@ -176,7 +176,8 @@ static inline void *kvm_get_hwpgd(struct kvm *kvm)struct kvm ; -#define kvm_flush_dcache_to_poc (a,l) __cpuc_flush_dcache_area((a), (l)) +#define kvm_flush_dcache_to_poc (a,l) __cpuc_flush_dcache_area((a), (l)) +#define kvm_invalidate_cache_to_poc (a,l) dmac_unmap_area((a), (l), 0)static inline bool vcpu_has_cache_enabled (struct kvm_vcpu *vcpu) {
@@ -184,7 +185,7 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) }static inline void __coherent_cache_guest_page (pfn_t pfn,unsigned long size, -bool need_flush) +bool need_flush,bool invalidate) {/* * If we are going to insert an instruction page and the icache is
@@ -214,6 +215,8 @@ static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,if (need_flush)kvm_flush_dcache_to_poc (va,PAGE_SIZE ); +if (invalidate) +kvm_invalidate_cache_to_poc (va,PAGE_SIZE );if (icache_is_pipt ()) __cpuc_coherent_user_range((unsigned long)va,
arch/arm/kvm/mmu.c +1 -1
@@ -1158,7 +1158,7 @@ static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn, unsigned long size, bool uncached) { bool need_flush = uncached || !vcpu_has_cache_enabled(vcpu); -__coherent_cache_guest_page (pfn, size, need_flush); +__coherent_cache_guest_page (pfn, size, need_flush, false); } static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
arch/arm64/include/asm/kvm_mmu.h +5 -2
@@ -257,7 +257,8 @@ static inline bool kvm_page_empty(void *ptr)struct kvm ; -#define kvm_flush_dcache_to_poc (a,l) __flush_dcache_area((a), (l)) +#define kvm_flush_dcache_to_poc (a,l) __flush_dcache_area((a), (l)) +#define kvm_invalidate_cache_to_poc (a,l) __dma_unmap_area((a), (l), 0)static inline bool vcpu_has_cache_enabled (struct kvm_vcpu *vcpu) {
@@ -265,12 +266,14 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) }static inline void __coherent_cache_guest_page (pfn_t pfn,unsigned long size, -bool need_flush) +bool need_flush,bool invalidate) {void *va =page_address (pfn_to_page (pfn));if (need_flush)kvm_flush_dcache_to_poc (va, size); +if (invalidate) +kvm_invalidate_cache_to_poc (va, size);if (!icache_is_aliasing ()) {/* PIPT */ flush_icache_range((unsigned long)va,
Signature · 1 line
1.8.3.1
RFC PATCH 5/6 KVM: ARM: implement kvm_*_incoherent_memory_regions
In reply to Andrew JonesAdd the kvm_*_incoherent_memory_regions calls to arm's kvm_arch_vcpu_ioctl_run and implement the corresponding arch flush/invalidate functions.
--- arch/arm/include/uapi/asm/kvm.h | 1 + arch/arm/kvm/arm.c | 4 +++ arch/arm/kvm/mmu.c | 54 +++++++++++++++++++++++++++++++++++++++ arch/arm64/include/uapi/asm/kvm.h | 1 + 4 files changed, 60 insertions(+)
arch/arm/include/uapi/asm/kvm.h +1 -0
@@ -26,6 +26,7 @@#define __KVM_HAVE_GUEST_DEBUG#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_INCOHERENT_MEM#define KVM_REG_SIZE (id) \ (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
arch/arm/kvm/arm.c +4 -0
@@ -486,6 +486,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)return ret; } +kvm_flush_incoherent_memory_regions (vcpu->kvm); +if (vcpu->sigset_active)sigprocmask (SIG_SETMASK , &vcpu->sigset, &sigsaved);
@@ -556,6 +558,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) ret =handle_exit (vcpu, run, ret); } +kvm_invalidate_incoherent_memory_regions (vcpu->kvm); +if (vcpu->sigset_active)sigprocmask (SIG_SETMASK , &sigsaved,NULL );return ret;
arch/arm/kvm/mmu.c +54 -0
66 lines not on this page · raw message
arch/arm64/include/uapi/asm/kvm.h +1 -0
@@ -38,6 +38,7 @@#define __KVM_HAVE_GUEST_DEBUG#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_INCOHERENT_MEM#define KVM_REG_SIZE (id) \ (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
Signature · 1 line
1.8.3.1
RFC PATCH 6/6 KVM: ARM: no need for kvm_arch_flush_incoherent
In reply to Andrew Joneskvm_arch_flush_incoherent makes things too slow, and we don't need it. Userspace can flush for us, as the necessary cache maintenance instruction is not (necessarily) privileged.
--- arch/arm/kvm/mmu.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/arm/kvm/mmu.c +6 -0
@@ -1822,6 +1822,11 @@ int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,void kvm_arch_flush_incoherent (struct kvm *kvm,struct kvm_memory_slot *slot) { +/* +* We no longer need this function, now that userspace does the +* flushing. +*/ +#if 0 if (slot->flags &KVM_MEM_READONLY ) {/* * Readonly memory shouldn't be changing, and we do a
@@ -1840,6 +1845,7 @@ void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)*/ coherent_cache_memslot (slot, true); +#endif }void kvm_arch_invalidate_incoherent (struct kvm *kvm,struct kvm_memory_slot *slot)
Signature · 1 line
1.8.3.1
cover support KVM_MEM_INCOHERENT
In reply to Andrew JonesAdd support for the new KVM_MEM_INCOHERENT flag, and flag appropriate memory. (Only flags vram for now.)
Patch 6/6 doesn't appear to be complete. While the VGA output is 99% corruption free, it's not perfect, so it's missing flushes somewhere...
Andrew Jones (6):
memory: add incoherent cache flag
HACK: linux header update
kvm-all: put kvm_mem_flags to more work
kvm-all: set KVM_MEM_INCOHERENT
vga: flag vram as incoherent
memory: add clear_cache_to_poc
exec.c | 16 ++++++++++------ hw/display/vga.c | 1 + include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++ include/exec/memory.h | 23 +++++++++++++++++++++++ kvm-all.c | 28 ++++++++++++++++++---------- linux-headers/linux/kvm.h | 1 + memory.c | 15 +++++++++++++++ 7 files changed, 109 insertions(+), 16 deletions(-)
Signature · 1 line
1.8.3.1
RFC PATCH 1/6 memory: add incoherent cache flag
Add an incoherent cache flag, which indicates the region needs explicit cache maintenance.
--- include/exec/memory.h | 23 +++++++++++++++++++++++ memory.c | 15 +++++++++++++++ 2 files changed, 38 insertions(+)
include/exec/memory.h +23 -0
@@ -158,6 +158,7 @@ struct MemoryRegion {bool rom_device;bool warning_printed;/* For reservations */ bool flush_coalesced_mmio; +bool incoherent_cache;MemoryRegion *alias;hwaddr alias_offset;int32_t priority;
@@ -778,6 +779,28 @@ void memory_region_set_flush_coalesced(MemoryRegion *mr);void memory_region_clear_flush_coalesced (MemoryRegion *mr);/** +* memory_region_set_incoherent_cache: Flag this memory region as needing +* explicit cache maintenance. +* +* @mr: the memory region to be updated. +*/ +void memory_region_set_incoherent_cache (MemoryRegion *mr); + +/** +* memory_region_clear_incoherent_cache: Remove the incoherent cache flag. +* +* @mr: the memory region to be updated. +*/ +void memory_region_clear_incoherent_cache (MemoryRegion *mr); + +/** +* memory_region_has_incoherent_cache: Return the incoherent cache flag. +* +* @mr: the memory region to check. +*/ +bool memory_region_has_incoherent_cache (MemoryRegion *mr); + +/** * memory_region_add_eventfd: Request an eventfd to be triggered when a word * is written to a location. *
memory.c +15 -0
@@ -1549,6 +1549,21 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr) } } +void memory_region_set_incoherent_cache (MemoryRegion *mr) +{ + mr->incoherent_cache = true; +} + +void memory_region_clear_incoherent_cache (MemoryRegion *mr) +{ + mr->incoherent_cache = false; +} + +bool memory_region_has_incoherent_cache (MemoryRegion *mr) +{ +return mr->incoherent_cache; +} + void memory_region_add_eventfd(MemoryRegion *mr, hwaddr addr, unsigned size,
Signature · 1 line
1.8.3.1
RFC PATCH 2/6 HACK: linux header update
In reply to Andrew JonesShould do a proper update-linux-headers.sh update.
--- linux-headers/linux/kvm.h | 1 + 1 file changed, 1 insertion(+)
linux-headers/linux/kvm.h +1 -0
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {*/ #define KVM_MEM_LOG_DIRTY_PAGES (1UL << 0)#define KVM_MEM_READONLY (1UL << 1) +#define KVM_MEM_INCOHERENT (1UL << 2)/* for KVM_IRQ_LINE */ struct kvm_irq_level {
Signature · 1 line
1.8.3.1
PATCH 3/6 kvm-all: put kvm_mem_flags to more work
In reply to Andrew JonesCurrently kvm_mem_flags just translates bools to bits, let's make it also determine the bools first. This avoids its parameter list growing each time we add a flag.
---
Posted this, as it makes sense without this series.
http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01225.html
kvm-all.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-)
kvm-all.c +15 -10
@@ -294,10 +294,14 @@ err:* dirty pages logging control */ -static int kvm_mem_flags (KVMState *s,bool log_dirty,bool readonly) +static int kvm_mem_flags (MemoryRegion *mr) { +bool readonly = mr->readonly ||memory_region_is_romd (mr);int flags =0 ; - flags = log_dirty ?KVM_MEM_LOG_DIRTY_PAGES :0 ; + +if (memory_region_is_logging (mr)) { + flags |=KVM_MEM_LOG_DIRTY_PAGES ; + }if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; }
@@ -312,7 +316,10 @@ static int kvm_slot_dirty_pages_log_change(KVMSlot *mem, bool log_dirty) old_flags = mem->flags; - flags = (mem->flags & ~mask) |kvm_mem_flags (s, log_dirty, false); + flags = mem->flags & ~mask; +if (log_dirty) { + flags |=KVM_MEM_LOG_DIRTY_PAGES ; + } mem->flags = flags;/* If nothing changed effectively, no need to issue ioctl */
@@ -642,9 +649,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)KVMSlot *mem, old;int err;MemoryRegion *mr = section->mr; -bool log_dirty =memory_region_is_logging (mr);bool writeable = !mr->readonly && !mr->rom_device; -bool readonly_flag = mr->readonly ||memory_region_is_romd (mr);hwaddr start_addr = section->offset_within_address_space;ram_addr_t size =int128_get64 (section->size);void *ram =NULL ;
@@ -688,7 +693,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add) (ram - start_addr == mem->ram - mem->start_addr)) {/* The new slot fits into the existing one and comes with * identical parameters - update flags and done. */ -kvm_slot_dirty_pages_log_change (mem, log_dirty); +kvm_slot_dirty_pages_log_change (mem,memory_region_is_logging (mr));return ; }
@@ -721,7 +726,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add) mem->memory_size = old.memory_size; mem->start_addr = old.start_addr; mem->ram = old.ram; - mem->flags =kvm_mem_flags (s, log_dirty, readonly_flag); + mem->flags =kvm_mem_flags (mr); err =kvm_set_user_memory_region (s, mem);if (err) {
@@ -742,7 +747,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add) mem->memory_size = start_addr - old.start_addr; mem->start_addr = old.start_addr; mem->ram = old.ram; - mem->flags =kvm_mem_flags (s, log_dirty, readonly_flag); + mem->flags =kvm_mem_flags (mr); err =kvm_set_user_memory_region (s, mem);if (err) {
@@ -766,7 +771,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add) size_delta = mem->start_addr - old.start_addr; mem->memory_size = old.memory_size - size_delta; mem->ram = old.ram + size_delta; - mem->flags =kvm_mem_flags (s, log_dirty, readonly_flag); + mem->flags =kvm_mem_flags (mr); err =kvm_set_user_memory_region (s, mem);if (err) {
@@ -788,7 +793,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add) mem->memory_size = size; mem->start_addr = start_addr; mem->ram = ram; - mem->flags =kvm_mem_flags (s, log_dirty, readonly_flag); + mem->flags =kvm_mem_flags (mr); err =kvm_set_user_memory_region (s, mem);if (err) {
Signature · 1 line
1.8.3.1
RFC PATCH 4/6 kvm-all: set KVM_MEM_INCOHERENT
In reply to Andrew Jones--- kvm-all.c | 3 +++ 1 file changed, 3 insertions(+)
kvm-all.c +3 -0
@@ -305,6 +305,9 @@ static int kvm_mem_flags(MemoryRegion *mr)if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; } +if (memory_region_has_incoherent_cache (mr)) { + flags |=KVM_MEM_INCOHERENT ; + }return flags; }
Signature · 1 line
1.8.3.1
RFC PATCH 5/6 vga: flag vram as incoherent
In reply to Andrew Jones--- hw/display/vga.c | 1 + 1 file changed, 1 insertion(+)
hw/display/vga.c +1 -0
@@ -2135,6 +2135,7 @@ void vga_common_init(VGACommonState *s, Object *obj, bool global_vmstate) s->is_vbe_vmstate =1 ;memory_region_init_ram (&s->vram, obj,"vga.vram" , s->vram_size, &error_abort); +memory_region_set_incoherent_cache (&s->vram);vmstate_register_ram (&s->vram, global_vmstate ?NULL :DEVICE (obj));xen_register_framebuffer (&s->vram); s->vram_ptr =memory_region_get_ram_ptr (&s->vram);
Signature · 1 line
1.8.3.1
PATCH 6/6 memory: add clear_cache_to_poc
In reply to Andrew JonesAdd a function that flushes the cache to PoC. We need a new function because __builtin___clear_cache only flushes to PoU. Call this function each time an address in a memory region that has been flagged as having an incoherent cache is written. For starters we only implement it for ARM. Most other architectures don't need it anyway.
---
Currently only implemented for aarch64, doesn't completely work yet.
exec.c | 16 ++++++++++------ include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 6 deletions(-)
exec.c +10 -6
@@ -2261,7 +2261,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,#else -static void invalidate_and_set_dirty (hwaddr addr, +static void invalidate_and_set_dirty (MemoryRegion *mr,hwaddr addr,hwaddr length) {if (cpu_physical_memory_range_includes_clean (addr, length)) {
@@ -2269,6 +2269,10 @@ static void invalidate_and_set_dirty(hwaddr addr,cpu_physical_memory_set_dirty_range_nocode (addr, length); }xen_modified_memory (addr, length); +if (memory_region_has_incoherent_cache (mr)) { +char *start =qemu_get_ram_ptr (addr); +clear_cache_to_poc (start, start + length); + } }static int memory_access_size (MemoryRegion *mr,unsigned l,hwaddr addr)
@@ -2348,7 +2352,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,/* RAM case */ ptr =qemu_get_ram_ptr (addr1);memcpy (ptr, buf, l); -invalidate_and_set_dirty (addr1, l); +invalidate_and_set_dirty (mr, addr1, l); } }else {if (!memory_access_is_direct (mr, is_write)) {
@@ -2437,7 +2441,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as,switch (type) {case WRITE_DATA :memcpy (ptr, buf, l); -invalidate_and_set_dirty (addr1, l); +invalidate_and_set_dirty (mr, addr1, l);break ;case FLUSH_CACHE :flush_icache_range ((uintptr_t )ptr, (uintptr_t )ptr + l);
@@ -2622,7 +2626,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len, mr =qemu_ram_addr_from_host (buffer, &addr1);assert (mr !=NULL );if (is_write) { -invalidate_and_set_dirty (addr1, access_len); +invalidate_and_set_dirty (mr, addr1, access_len); }if (xen_enabled ()) {xen_invalidate_map_cache_entry (buffer);
@@ -2904,7 +2908,7 @@ static inline void stl_phys_internal(AddressSpace *as,stl_p (ptr, val);break ; } -invalidate_and_set_dirty (addr1,4 ); +invalidate_and_set_dirty (mr, addr1,4 ); } }
@@ -2967,7 +2971,7 @@ static inline void stw_phys_internal(AddressSpace *as,stw_p (ptr, val);break ; } -invalidate_and_set_dirty (addr1,2 ); +invalidate_and_set_dirty (mr, addr1,2 ); } }
include/exec/exec-all.h +41 -0
53 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [RFC/WIP PATCH 6/6] memory: add clear_cache_to_poc
Quoted 6 lines · Andrew Jones
Add a function that flushes the cache to PoC. We need a new function because
__builtin___clear_cacheonly flushes to PoU. Call this function each time an address in a memory region that has been flagged as having an incoherent cache is written. For starters we only implement it for ARM. Most other architectures don't need it anyway.
I started looking for my missing flushes, and see I have stupidity in this patch. I'm not flushing in the right place at all... My testing was just [un]lucky, making me think it was on the right track. I'll send an update to this tomorrow after I remove my head from a dark hole near my chair.
drew
Quoted 144 lines
Signed-off-by Andrew Jones <[email protected]>---
Currently only implemented for aarch64, doesn't completely work yet.exec.c | 16 ++++++++++------ include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 6 deletions(-)exec.c +10
-6@@ -2261,7 +2261,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,#else -static void invalidate_and_set_dirty (hwaddr addr, +static void invalidate_and_set_dirty (MemoryRegion *mr,hwaddr addr,hwaddr length) {if (cpu_physical_memory_range_includes_clean (addr, length)) {@@ -2269,6 +2269,10 @@ static void invalidate_and_set_dirty(hwaddr addr,cpu_physical_memory_set_dirty_range_nocode (addr, length); }xen_modified_memory (addr, length); +if (memory_region_has_incoherent_cache (mr)) { +char *start =qemu_get_ram_ptr (addr); +clear_cache_to_poc (start, start + length); + } }static int memory_access_size (MemoryRegion *mr,unsigned l,hwaddr addr)@@ -2348,7 +2352,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,/* RAM case */ ptr =qemu_get_ram_ptr (addr1);memcpy (ptr, buf, l); -invalidate_and_set_dirty (addr1, l); +invalidate_and_set_dirty (mr, addr1, l); } }else {if (!memory_access_is_direct (mr, is_write)) {@@ -2437,7 +2441,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as,switch (type) {case WRITE_DATA :memcpy (ptr, buf, l); -invalidate_and_set_dirty (addr1, l); +invalidate_and_set_dirty (mr, addr1, l);break ;case FLUSH_CACHE :flush_icache_range ((uintptr_t )ptr, (uintptr_t )ptr + l);@@ -2622,7 +2626,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len, mr =qemu_ram_addr_from_host (buffer, &addr1);assert (mr !=NULL );if (is_write) { -invalidate_and_set_dirty (addr1, access_len); +invalidate_and_set_dirty (mr, addr1, access_len); }if (xen_enabled ()) {xen_invalidate_map_cache_entry (buffer);@@ -2904,7 +2908,7 @@ static inline void stl_phys_internal(AddressSpace *as,stl_p (ptr, val);break ; } -invalidate_and_set_dirty (addr1,4 ); +invalidate_and_set_dirty (mr, addr1,4 ); } }@@ -2967,7 +2971,7 @@ static inline void stw_phys_internal(AddressSpace *as,stw_p (ptr, val);break ; } -invalidate_and_set_dirty (addr1,2 ); +invalidate_and_set_dirty (mr, addr1,2 ); } }include/exec/exec-all.h +41
-0@@ -106,6 +106,43 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr, hwaddr paddr, int prot, int mmu_idx, target_ulong size);void tb_invalidate_phys_addr (AddressSpace *as,hwaddr addr); +#if defined(__aarch64__) +static inline void clear_cache_to_poc (char *begin,char *end) +{ +/* Unfortunately __builtin___clear_cache only flushes +* to PoU, we need to implement this for PoC. +*/ +static unsigned long line_sz =0 ; +unsigned long start, stop, addr; + +if (!line_sz) { +unsigned int ctr_el0; + asmvolatile ("mrs %0, ctr_el0" :"=&r" (ctr_el0)); + line_sz = (1UL << ((ctr_el0 >>16 ) &0xf )) *sizeof (int ); + } + + start = (unsigned long )begin & ~(line_sz -1 ); + stop = ((unsigned long )(end + line_sz) & ~(line_sz -1 )); + +for (addr = start; addr < stop; addr += line_sz) { + asmvolatile ("dc cvac, %0" : :"r" (addr)); + } + +/* FIXME: Ideally, we'd also flush the icache now, just in +* case this is for an executable region. But, AArch64 can't +* flush it to PoC from userspace. We need a syscall. +*/ +} +#elif defined(__arm__) +static inline void clear_cache_to_poc (char *begin,char *end) +{ +/* TODO */ +} +#else +static inline void clear_cache_to_poc (char *begin,char *end) +{ +} +#endif #else static inline void tlb_flush_page (CPUState *cpu,target_ulong addr) {@@ -114,6 +151,10 @@ static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)static inline void tlb_flush (CPUState *cpu,int flush_global) { } + +void clear_cache_to_poc (char *begin,char *end) +{ +}#endif #define CODE_GEN_ALIGN 16/* must be >= of the size of a icache line */ Signature · 6 lines
1.8.3.1 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 41 lines · Andrew Jones
In reply to this message I'll send two series' one for KVM and one for QEMU. The two series' are their respective component complements, and attempt to implement cache coherency for arm guests using emulated devices, where the emulator (qemu) uses cached memory for the device memory, but the guest uses uncached - as device memory is generally used. Right now I've just focused on VGA vram.
This approach starts as the "add a new memslot flag" approach, and then turns into the "make qemu do some cache maintenance" approach with the final patch of each series (6/6). It stops short of the "add syscalls..." approach. Below is a summary of all the approaches discussed so far, to my knowledge.
"MAIR manipulating"
Posted[1] by Ard. Works. No performance degradation. Potential issues with device assignment and the guest getting confused."add a new memslot flag"
This posting (not counting patches 6/6). Works. Huge performance degradation."make qemu do some cache maintenance"
This posting (patches 6/6). We can only do so much in qemu without syscalls. This series does what it can. Almost works, probably could work, after playing 'find the missing flush'. This approach still requires the new memslot flag, as userspace can't invalidate the cache, only clean, or clean+invalidate. No noticeable performance degradation."add syscalls to make qemu do all cache maintenance" Variant 1: implement as kvm ioctls - to avoid trying to get
syscalls into the general kernel
Variant 2: add real syscalls, or maybe just ARM private SWIs
like__ARM_NR_cacheflush
This approach should work, and if we add an invalidate syscall, then we shouldn't need any kvm changes at all, i.e. no need for the memslot flag. I haven't experimented with this yet, but I'm starting to like the idea of variant 2, with a private SWI, so will try to pull something together soon for that.
Quoted 7 lines
"describe the problematic memory as cached to the guest" Not an ideal solution for virt. Could maybe be workable as a quirk for a specific device though.
re: $SUBJECT; Here 'cluster' is defined by the urban dictionary.
[1] http://thread.gmane.org/gmane.comp.emulators.kvm.arm.devel/34/
I'm going to send another pair of series'. A "v2", as, IMO, the new approach supersedes the pair implemented here. After poking around in qemu, looking for the best places to do cache maintenance, I decided I didn't really like doing it there at all, and opted to try another approach, that I'd forgotten to mention in this mail. That approach is the "MADV_UNCACHED" type that Paolo suggested. This type of approach could also be described as "make userspace's memory access type match the expected access type of the guest", and Mario has suggested using a memory driver, which could have the same result.
The series I'll send is inspired by both Paolo's and Mario's suggestions, but it uses a kvm memslot flag, rather than an madvise flag, and thus for the memory driver, it's just KVM.
drew