kvm: memory slot cleanups, fix, and increase
v2 cover kvm: memory slot cleanups, fix, and increase
v2: Update 02/10 to not check userspace_addr when slot is removed.
Yoshikawa-san withdrew objection to increase slot_bitmap prior to his series to remove slot_bitmap.
This series does away with any kind of complicated resizing of the slot array and simply does a one time increase. I do compact struct kvm_memory_slot a bit to take better advantage of the space we are using. This reduces each slot from 64 bytes (x86_64) to 56 bytes. By enforcing the API around valid operations for an unused slot and fields that can be modified runtime, I found and was able to fix a bug in iommu mapping for slots. The renames enabled me to find the previously posted bug fix for catching slot overlaps.
As mentioned in the series, the primary motivation for increasing memory slots is assigned devices. With this, I've been able to assign 30 devices to a single VM and could have gone further, but ran out of SRIOV VFs. Typical devices use anywhere from 2-4 slots and max out at 8 slots. 125 user slots (3 private slots) allows us to support between 28 and 56 typical devices per VM.
Tested on x86_64, compiled on ia64, powerpc, and s390.
Thanks,
Alex
---
Alex Williamson (10):
kvm: Restrict non-existing slot state transitions
kvm: Check userspace_addr when modifying a memory slot
kvm: Fix iommu map/unmap to handle memory slot moves
kvm: Minor memory slot optimization
kvm: Rename KVM_MEMORY_SLOTS -> KVM_USER_MEM_SLOTS
kvm: Make KVM_PRIVATE_MEM_SLOTS optional
kvm: struct kvm_memory_slot.user_alloc -> bool
kvm: struct kvm_memory_slot.flags -> u32
kvm: struct kvm_memory_slot.id -> short
kvm: Increase user memory slots on x86 to 125
arch/ia64/include/asm/kvm_host.h | 4 -- arch/ia64/kvm/kvm-ia64.c | 8 ++-- arch/powerpc/include/asm/kvm_host.h | 6 +-- arch/powerpc/kvm/book3s_hv.c | 2 - arch/powerpc/kvm/powerpc.c | 4 +- arch/s390/include/asm/kvm_host.h | 4 -- arch/s390/kvm/kvm-s390.c | 4 +- arch/x86/include/asm/kvm_host.h | 8 ++-- arch/x86/include/asm/vmx.h | 6 +-- arch/x86/kvm/vmx.c | 6 +-- arch/x86/kvm/x86.c | 10 ++--- include/linux/kvm_host.h | 24 +++++++----- virt/kvm/kvm_main.c | 72 +++++++++++++++++++++++------------ 13 files changed, 90 insertions(+), 68 deletions(-)
PATCH v2 1/10 kvm: Restrict non-existing slot state transitions
The API documentation states:
When changing an existing slot, it may be moved in the guest physical memory space, or its flags may be modified.
An "existing slot" requires a non-zero npages (memory_size). The only transition we should therefore allow for a non-existing slot should be to create the slot, which includes setting a non-zero memory_size. We currently allow calls to modify non-existing slots, which is pointless, confusing, and possibly wrong.
With this we know that the invalidation path of __kvm_set_memory_region is always for a delete or move and never for adding a zero size slot.
--- virt/kvm/kvm_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
virt/kvm/kvm_main.c +7 -2
@@ -753,10 +753,15 @@ int __kvm_set_memory_region(struct kvm *kvm, new.npages = npages; new.flags = mem->flags; -/* Disallow changing a memory slot's size. */ +/* +* Disallow changing a memory slot's size or changing anything about +* zero sized slots that doesn't involve making them non-zero. +*/ r = -EINVAL ;if (npages && old.npages && npages != old.npages) goto out_free; +if (!npages && !old.npages) + goto out_free;/* Check for overlaps */ r = -EEXIST ;
@@ -775,7 +780,7 @@ int __kvm_set_memory_region(struct kvm *kvm, r = -ENOMEM ;/* Allocate if a slot is being created */ -if (npages && !old.npages) { +if (!old.npages) { new.user_alloc = user_alloc; new.userspace_addr = mem->userspace_addr;
PATCH v2 2/10 kvm: Check userspace_addr when modifying a memory slot
In reply to Alex WilliamsonThe API documents that only flags and guest physical memory space can be modified on an existing slot, but we don't enforce that the userspace address cannot be modified. Instead we just ignore it. This means that a user may think they've successfully moved both the guest and user addresses, when in fact only the guest address changed. Check and error instead.
--- virt/kvm/kvm_main.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +7 -1
@@ -779,13 +779,19 @@ int __kvm_set_memory_region(struct kvm *kvm, r = -ENOMEM ; -/* Allocate if a slot is being created */ +/* +* Allocate if a slot is being created. If modifying a slot, +* the userspace_addr cannot change. +*/ if (!old.npages) { new.user_alloc = user_alloc; new.userspace_addr = mem->userspace_addr;if (kvm_arch_create_memslot (&new, npages)) goto out_free; + }else if (npages && mem->userspace_addr != old.userspace_addr) { + r = -EINVAL ; + goto out_free; }/* Allocate page dirty bitmap if needed */
PATCH v2 3/10 kvm: Fix iommu map/unmap to handle memory slot moves
In reply to Alex WilliamsonThe iommu integration into memory slots expects memory slots to be added or removed and doesn't handle the move case. We can unmap slots from the iommu after we mark them invalid and map them before installing the final memslot array. Also re-order the kmemdup vs map so we don't leave iommu mappings if we get ENOMEM.
--- virt/kvm/kvm_main.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
virt/kvm/kvm_main.c +11 -8
@@ -817,6 +817,8 @@ int __kvm_set_memory_region(struct kvm *kvm, old_memslots = kvm->memslots;rcu_assign_pointer (kvm->memslots, slots);synchronize_srcu_expedited (&kvm->srcu); +/* slot was deleted or moved, clear iommu mapping */ +kvm_iommu_unmap_pages (kvm, &old);/* From this point no new shadow pages pointing to a deleted, * or moved, memslot will be created. *
@@ -832,20 +834,19 @@ int __kvm_set_memory_region(struct kvm *kvm,if (r) goto out_free; -/* map/unmap the pages in iommu page table */ -if (npages) { - r =kvm_iommu_map_pages (kvm, &new); -if (r) - goto out_free; - }else -kvm_iommu_unmap_pages (kvm, &old); - r = -ENOMEM ; slots =kmemdup (kvm->memslots,sizeof (struct kvm_memslots ),GFP_KERNEL );if (!slots) goto out_free; +/* map new memory slot into the iommu */ +if (npages) { + r =kvm_iommu_map_pages (kvm, &new); +if (r) + goto out_slots; + } +/* actual memory is freed via old in kvm_free_physmem_slot below */ if (!npages) { new.dirty_bitmap =NULL ;
@@ -864,6 +865,8 @@ int __kvm_set_memory_region(struct kvm *kvm,return 0 ; +out_slots: +kfree (slots); out_free:kvm_free_physmem_slot (&new, &old); out:
PATCH v2 4/10 kvm: Minor memory slot optimization
In reply to Alex WilliamsonIf a slot is removed or moved in the guest physical address space, we first allocate and install a new slot array with the invalidated entry. The old array is then freed. We then proceed to allocate yet another slot array to install the permanent replacement. Re-use the original array when this occurs and avoid the extra kfree/kmalloc.
--- virt/kvm/kvm_main.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
virt/kvm/kvm_main.c +14 -7
@@ -711,7 +711,7 @@ int __kvm_set_memory_region(struct kvm *kvm,unsigned long npages;struct kvm_memory_slot *memslot, *slot;struct kvm_memory_slot old, new; -struct kvm_memslots *slots, *old_memslots; +struct kvm_memslots *slots =NULL , *old_memslots; r =check_memory_region_flags (mem);if (r)
@@ -827,18 +827,25 @@ int __kvm_set_memory_region(struct kvm *kvm,* - kvm_is_visible_gfn (mmu_check_roots) */ kvm_arch_flush_shadow_memslot (kvm, slot); -kfree (old_memslots); + slots = old_memslots; } r =kvm_arch_prepare_memory_region (kvm, &new, old, mem, user_alloc);if (r) - goto out_free; + goto out_slots; r = -ENOMEM ; - slots =kmemdup (kvm->memslots,sizeof (struct kvm_memslots ), -GFP_KERNEL ); -if (!slots) - goto out_free; +/* +* We can re-use the old_memslots from above, the only difference +* from the currently installed memslots is the invalid flag. This +* will get overwritten by update_memslots anyway. +*/ +if (!slots) { + slots =kmemdup (kvm->memslots,sizeof (struct kvm_memslots ), +GFP_KERNEL ); +if (!slots) + goto out_free; + }/* map new memory slot into the iommu */ if (npages) {
PATCH v2 5/10 kvm: Rename KVM_MEMORY_SLOTS -> KVM_USER_MEM_SLOTS
In reply to Alex WilliamsonIt's easy to confuse KVM_MEMORY_SLOTS and KVM_MEM_SLOTS_NUM. One is the user accessible slots and the other is user + private. Make this more obvious.
--- arch/ia64/include/asm/kvm_host.h | 2 +- arch/ia64/kvm/kvm-ia64.c | 2 +- arch/powerpc/include/asm/kvm_host.h | 4 ++-- arch/powerpc/kvm/book3s_hv.c | 2 +- arch/s390/include/asm/kvm_host.h | 2 +- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/include/asm/vmx.h | 6 +++--- arch/x86/kvm/x86.c | 6 +++--- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 8 ++++---- 10 files changed, 19 insertions(+), 19 deletions(-)
arch/ia64/include/asm/kvm_host.h +1 -1
@@ -23,7 +23,7 @@#ifndef __ASM_KVM_HOST_H#define __ASM_KVM_HOST_H -#define KVM_MEMORY_SLOTS 32 +#define KVM_USER_MEM_SLOTS 32/* memory slots that does not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 4
arch/ia64/kvm/kvm-ia64.c +1 -1
@@ -1831,7 +1831,7 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,mutex_lock (&kvm->slots_lock); r = -EINVAL ; -if (log->slot >=KVM_MEMORY_SLOTS ) +if (log->slot >=KVM_USER_MEM_SLOTS ) goto out; memslot =id_to_memslot (kvm->memslots, log->slot);
arch/powerpc/include/asm/kvm_host.h +2 -2
@@ -37,10 +37,10 @@#define KVM_MAX_VCPUS NR_CPUS#define KVM_MAX_VCORES NR_CPUS -#define KVM_MEMORY_SLOTS 32 +#define KVM_USER_MEM_SLOTS 32/* memory slots that does not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 4 -#define KVM_MEM_SLOTS_NUM (KVM_MEMORY_SLOTS + KVM_PRIVATE_MEM_SLOTS) +#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)#ifdef CONFIG_KVM_MMIO #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
arch/powerpc/kvm/book3s_hv.c +1 -1
@@ -1262,7 +1262,7 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)mutex_lock (&kvm->slots_lock); r = -EINVAL ; -if (log->slot >=KVM_MEMORY_SLOTS ) +if (log->slot >=KVM_USER_MEM_SLOTS ) goto out; memslot =id_to_memslot (kvm->memslots, log->slot);
arch/s390/include/asm/kvm_host.h +1 -1
@@ -20,7 +20,7 @@#include <asm/cpu.h> #define KVM_MAX_VCPUS 64 -#define KVM_MEMORY_SLOTS 32 +#define KVM_USER_MEM_SLOTS 32/* memory slots that does not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 4
arch/x86/include/asm/kvm_host.h +2 -2
@@ -31,10 +31,10 @@#define KVM_MAX_VCPUS 254#define KVM_SOFT_MAX_VCPUS 160 -#define KVM_MEMORY_SLOTS 32 +#define KVM_USER_MEM_SLOTS 32/* memory slots that does not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 4 -#define KVM_MEM_SLOTS_NUM (KVM_MEMORY_SLOTS + KVM_PRIVATE_MEM_SLOTS) +#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)#define KVM_MMIO_SIZE 16
arch/x86/include/asm/vmx.h +3 -3
@@ -427,9 +427,9 @@ enum vmcs_field {#define AR_RESERVD_MASK 0xfffe0f00 -#define TSS_PRIVATE_MEMSLOT (KVM_MEMORY_SLOTS + 0) -#define APIC_ACCESS_PAGE_PRIVATE_MEMSLOT (KVM_MEMORY_SLOTS + 1) -#define IDENTITY_PAGETABLE_PRIVATE_MEMSLOT (KVM_MEMORY_SLOTS + 2) +#define TSS_PRIVATE_MEMSLOT (KVM_USER_MEM_SLOTS + 0) +#define APIC_ACCESS_PAGE_PRIVATE_MEMSLOT (KVM_USER_MEM_SLOTS + 1) +#define IDENTITY_PAGETABLE_PRIVATE_MEMSLOT (KVM_USER_MEM_SLOTS + 2)#define VMX_NR_VPIDS (1 << 16)#define VMX_VPID_EXTENT_SINGLE_CONTEXT 1
arch/x86/kvm/x86.c +3 -3
@@ -2204,7 +2204,7 @@ int kvm_dev_ioctl_check_extension(long ext) r =KVM_MAX_VCPUS ;break ;case KVM_CAP_NR_MEMSLOTS : - r =KVM_MEMORY_SLOTS ; + r =KVM_USER_MEM_SLOTS ;break ;case KVM_CAP_PV_MMU :/* obsolete */ r =0 ;
@@ -3135,7 +3135,7 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)mutex_lock (&kvm->slots_lock); r = -EINVAL ; -if (log->slot >=KVM_MEMORY_SLOTS ) +if (log->slot >=KVM_USER_MEM_SLOTS ) goto out; memslot =id_to_memslot (kvm->memslots, log->slot);
@@ -6449,7 +6449,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,int map_flags =MAP_PRIVATE |MAP_ANONYMOUS ;/* Prevent internal slot pages from being moved by fork()/COW. */ -if (memslot->id >=KVM_MEMORY_SLOTS ) +if (memslot->id >=KVM_USER_MEM_SLOTS ) map_flags =MAP_SHARED |MAP_ANONYMOUS ;/*To keep backward compatibility with older userspace,
include/linux/kvm_host.h +1 -1
@@ -299,7 +299,7 @@ struct kvm_irq_routing_table {};#endif #ifndef KVM_MEM_SLOTS_NUM -#define KVM_MEM_SLOTS_NUM (KVM_MEMORY_SLOTS + KVM_PRIVATE_MEM_SLOTS) +#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)#endif /*
virt/kvm/kvm_main.c +4 -4
@@ -766,7 +766,7 @@ int __kvm_set_memory_region(struct kvm *kvm,/* Check for overlaps */ r = -EEXIST ;kvm_for_each_memslot (slot, kvm->memslots) { -if (slot->id >=KVM_MEMORY_SLOTS || slot == memslot) +if (slot->id >=KVM_USER_MEM_SLOTS || slot == memslot)continue ;if (!((base_gfn + npages <= slot->base_gfn) || (base_gfn >= slot->base_gfn + slot->npages)))
@@ -900,7 +900,7 @@ int kvm_vm_ioctl_set_memory_region(struct kvm *kvm, kvm_userspace_memory_region *mem, int user_alloc) { -if (mem->slot >=KVM_MEMORY_SLOTS ) +if (mem->slot >=KVM_USER_MEM_SLOTS )return -EINVAL ;return kvm_set_memory_region (kvm, mem, user_alloc); }
@@ -914,7 +914,7 @@ int kvm_get_dirty_log(struct kvm *kvm,unsigned long any =0 ; r = -EINVAL ; -if (log->slot >=KVM_MEMORY_SLOTS ) +if (log->slot >=KVM_USER_MEM_SLOTS ) goto out; memslot =id_to_memslot (kvm->memslots, log->slot);
@@ -960,7 +960,7 @@ int kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn) {struct kvm_memory_slot *memslot =gfn_to_memslot (kvm, gfn); -if (!memslot || memslot->id >=KVM_MEMORY_SLOTS || +if (!memslot || memslot->id >=KVM_USER_MEM_SLOTS || memslot->flags &KVM_MEMSLOT_INVALID )return 0 ;
PATCH v2 6/10 kvm: Make KVM_PRIVATE_MEM_SLOTS optional
In reply to Alex WilliamsonSeems like everyone copied x86 and defined 4 private memory slots that never actually get used. Even x86 only uses 3 of the 4. These aren't exposed so there's no need to add padding.
--- arch/ia64/include/asm/kvm_host.h | 2 -- arch/powerpc/include/asm/kvm_host.h | 4 +--- arch/s390/include/asm/kvm_host.h | 2 -- arch/x86/include/asm/kvm_host.h | 4 ++-- include/linux/kvm_host.h | 4 ++++ 5 files changed, 7 insertions(+), 9 deletions(-)
arch/ia64/include/asm/kvm_host.h +0 -2
@@ -24,8 +24,6 @@#define __ASM_KVM_HOST_H#define KVM_USER_MEM_SLOTS 32 -/* memory slots that does not exposed to userspace */ -#define KVM_PRIVATE_MEM_SLOTS 4#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
arch/powerpc/include/asm/kvm_host.h +1 -3
@@ -38,9 +38,7 @@#define KVM_MAX_VCPUS NR_CPUS#define KVM_MAX_VCORES NR_CPUS#define KVM_USER_MEM_SLOTS 32 -/* memory slots that does not exposed to userspace */ -#define KVM_PRIVATE_MEM_SLOTS 4 -#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS) +#define KVM_MEM_SLOTS_NUM KVM_USER_MEM_SLOTS#ifdef CONFIG_KVM_MMIO #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
arch/s390/include/asm/kvm_host.h +0 -2
@@ -21,8 +21,6 @@#define KVM_MAX_VCPUS 64#define KVM_USER_MEM_SLOTS 32 -/* memory slots that does not exposed to userspace */ -#define KVM_PRIVATE_MEM_SLOTS 4struct sca_entry {atomic_t scn;
arch/x86/include/asm/kvm_host.h +2 -2
@@ -32,8 +32,8 @@#define KVM_MAX_VCPUS 254#define KVM_SOFT_MAX_VCPUS 160#define KVM_USER_MEM_SLOTS 32 -/* memory slots that does not exposed to userspace */ -#define KVM_PRIVATE_MEM_SLOTS 4 +/* memory slots that are not exposed to userspace */ +#define KVM_PRIVATE_MEM_SLOTS 3#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)#define KVM_MMIO_SIZE 16
include/linux/kvm_host.h +4 -0
@@ -298,6 +298,10 @@ struct kvm_irq_routing_table {};#endif +#ifndef KVM_PRIVATE_MEM_SLOTS +#define KVM_PRIVATE_MEM_SLOTS 0 +#endif +#ifndef KVM_MEM_SLOTS_NUM #define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)#endif
PATCH v2 7/10 kvm: struct kvm_memory_slot.user_alloc -> bool
In reply to Alex WilliamsonThere's no need for this to be an int, it holds a boolean. Move to the end of the struct for alignment.
--- arch/ia64/kvm/kvm-ia64.c | 6 +++--- arch/powerpc/kvm/powerpc.c | 4 ++-- arch/s390/kvm/kvm-s390.c | 4 ++-- arch/x86/kvm/vmx.c | 6 +++--- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 12 ++++++------ virt/kvm/kvm_main.c | 9 +++++---- 7 files changed, 23 insertions(+), 22 deletions(-)
arch/ia64/kvm/kvm-ia64.c +3 -3
@@ -955,7 +955,7 @@ long kvm_arch_vm_ioctl(struct file *filp, kvm_mem.guest_phys_addr; kvm_userspace_mem.memory_size = kvm_mem.memory_size; r =kvm_vm_ioctl_set_memory_region (kvm, - &kvm_userspace_mem,0 ); + &kvm_userspace_mem, false);if (r) goto out;break ;
@@ -1577,7 +1577,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,struct kvm_memory_slot *memslot, struct kvm_memory_slot old, struct kvm_userspace_memory_region *mem, - int user_alloc) + bool user_alloc) { unsigned long i;unsigned long pfn;
@@ -1608,7 +1608,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,void kvm_arch_commit_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem,struct kvm_memory_slot old, -int user_alloc) +bool user_alloc) {return ; }
arch/powerpc/kvm/powerpc.c +2 -2
@@ -321,7 +321,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,struct kvm_memory_slot *memslot, struct kvm_memory_slot old, struct kvm_userspace_memory_region *mem, - int user_alloc) + bool user_alloc) { return kvmppc_core_prepare_memory_region(kvm, mem); }
@@ -329,7 +329,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,void kvm_arch_commit_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem,struct kvm_memory_slot old, -int user_alloc) +bool user_alloc) {kvmppc_core_commit_memory_region (kvm, mem); }
arch/s390/kvm/kvm-s390.c +2 -2
@@ -927,7 +927,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,struct kvm_memory_slot *memslot, struct kvm_memory_slot old, struct kvm_userspace_memory_region *mem, - int user_alloc) + bool user_alloc) {/* A few sanity checks. We can have exactly one memory slot which has to start at guest virtual zero and which has to be located at a
@@ -957,7 +957,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,void kvm_arch_commit_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem,struct kvm_memory_slot old, -int user_alloc) +bool user_alloc) {int rc;
arch/x86/kvm/vmx.c +3 -3
@@ -3597,7 +3597,7 @@ static int alloc_apic_access_page(struct kvm *kvm) kvm_userspace_mem.flags =0 ; kvm_userspace_mem.guest_phys_addr =0xfee00000ULL ; kvm_userspace_mem.memory_size =PAGE_SIZE ; - r =__kvm_set_memory_region (kvm, &kvm_userspace_mem,0 ); + r =__kvm_set_memory_region (kvm, &kvm_userspace_mem, false);if (r) goto out;
@@ -3627,7 +3627,7 @@ static int alloc_identity_pagetable(struct kvm *kvm) kvm_userspace_mem.guest_phys_addr = kvm->arch.ept_identity_map_addr; kvm_userspace_mem.memory_size =PAGE_SIZE ; - r =__kvm_set_memory_region (kvm, &kvm_userspace_mem,0 ); + r =__kvm_set_memory_region (kvm, &kvm_userspace_mem, false);if (r) goto out;
@@ -4191,7 +4191,7 @@ static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr) .flags = 0, }; - ret =kvm_set_memory_region (kvm, &tss_mem,0 ); + ret =kvm_set_memory_region (kvm, &tss_mem, false);if (ret)return ret; kvm->arch.tss_addr = addr;
arch/x86/kvm/x86.c +2 -2
@@ -6443,7 +6443,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,struct kvm_memory_slot *memslot, struct kvm_memory_slot old, struct kvm_userspace_memory_region *mem, - int user_alloc) + bool user_alloc) { int npages = memslot->npages;int map_flags =MAP_PRIVATE |MAP_ANONYMOUS ;
@@ -6479,7 +6479,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,void kvm_arch_commit_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem,struct kvm_memory_slot old, -int user_alloc) +bool user_alloc) {int nr_mmu_pages =0 , npages = mem->memory_size >>PAGE_SHIFT ;
include/linux/kvm_host.h +6 -6
@@ -255,8 +255,8 @@ struct kvm_memory_slot {unsigned long *dirty_bitmap;struct kvm_arch_memory_slot arch;unsigned long userspace_addr; -int user_alloc;int id; +bool user_alloc; };static inline unsigned long kvm_dirty_bitmap_bytes (struct kvm_memory_slot *memslot)
@@ -436,10 +436,10 @@ id_to_memslot(struct kvm_memslots *slots, int id)int kvm_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem, -int user_alloc); +bool user_alloc);int __kvm_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem, -int user_alloc); +bool user_alloc);void kvm_arch_free_memslot (struct kvm_memory_slot *free,struct kvm_memory_slot *dont);int kvm_arch_create_memslot (struct kvm_memory_slot *slot,unsigned long npages);
@@ -447,11 +447,11 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,struct kvm_memory_slot *memslot, struct kvm_memory_slot old, struct kvm_userspace_memory_region *mem, - int user_alloc); + bool user_alloc);void kvm_arch_commit_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem,struct kvm_memory_slot old, -int user_alloc); +bool user_alloc);bool kvm_largepages_enabled (void );void kvm_disable_largepages (void );/* flush all memory translations */
@@ -537,7 +537,7 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,int kvm_vm_ioctl_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem, -int user_alloc); +bool user_alloc);int kvm_vm_ioctl_irq_line (struct kvm *kvm,struct kvm_irq_level *irq_level);long kvm_arch_vm_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg);
virt/kvm/kvm_main.c +5 -4
@@ -704,7 +704,7 @@ static int check_memory_region_flags(struct kvm_userspace_memory_region *mem)*/ int __kvm_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem, -int user_alloc) +bool user_alloc) {int r;gfn_t base_gfn;
@@ -884,7 +884,7 @@ EXPORT_SYMBOL_GPL(__kvm_set_memory_region);int kvm_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem, -int user_alloc) +bool user_alloc) {int r;
@@ -898,7 +898,7 @@ EXPORT_SYMBOL_GPL(kvm_set_memory_region);int kvm_vm_ioctl_set_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem, -int user_alloc) +bool user_alloc) {if (mem->slot >=KVM_USER_MEM_SLOTS )return -EINVAL ;
@@ -2158,7 +2158,8 @@ static long kvm_vm_ioctl(struct file *filp, sizeof kvm_userspace_mem)) goto out; - r =kvm_vm_ioctl_set_memory_region (kvm, &kvm_userspace_mem,1 ); + r =kvm_vm_ioctl_set_memory_region (kvm, + &kvm_userspace_mem, true);if (r) goto out;break ;
PATCH v2 8/10 kvm: struct kvm_memory_slot.flags -> u32
In reply to Alex Williamsonstruct kvm_userspace_memory_region.flags is a u32 with a comment that bits 0 ~ 15 are visible to userspace and the other bits are reserved for kvm internal use. KVM_MEMSLOT_INVALID is the only internal use flag and it has a comment that bits 16 ~ 31 are internally used and the other bits are visible to userspace.
Therefore, let's define this as a u32 so we don't waste bytes on LP64 systems. Move to the end of the struct for alignment.
--- include/linux/kvm_host.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/kvm_host.h +1 -1
@@ -251,10 +251,10 @@ static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu)struct kvm_memory_slot {gfn_t base_gfn;unsigned long npages; -unsigned long flags;unsigned long *dirty_bitmap;struct kvm_arch_memory_slot arch;unsigned long userspace_addr; +u32 flags;int id;bool user_alloc; };
PATCH v2 9/10 kvm: struct kvm_memory_slot.id -> short
In reply to Alex WilliamsonWe're currently offering a whopping 32 memory slots to user space, an int is a bit excessive for storing this. We would like to increase our memslots, but SHRT_MAX should be more than enough.
--- include/linux/kvm_host.h | 4 ++-- virt/kvm/kvm_main.c | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-)
include/linux/kvm_host.h +2 -2
@@ -255,7 +255,7 @@ struct kvm_memory_slot {struct kvm_arch_memory_slot arch;unsigned long userspace_addr;u32 flags; -int id; +short id;bool user_alloc; };
@@ -315,7 +315,7 @@ struct kvm_memslots {u64 generation;struct kvm_memory_slot memslots[KVM_MEM_SLOTS_NUM ];/* The mapping table from slot id to the index in memslots[]. */ -int id_to_index[KVM_MEM_SLOTS_NUM ]; +short id_to_index[KVM_MEM_SLOTS_NUM ]; };struct kvm {
virt/kvm/kvm_main.c +2 -0
@@ -469,6 +469,8 @@ static struct kvm *kvm_create_vm(unsigned long type)INIT_HLIST_HEAD (&kvm->irq_ack_notifier_list);#endif +BUILD_BUG_ON (KVM_MEM_SLOTS_NUM >SHRT_MAX ); + r = -ENOMEM ; kvm->memslots =kzalloc (sizeof (struct kvm_memslots ),GFP_KERNEL );if (!kvm->memslots)
PATCH v2 10/10 kvm: Increase user memory slots on x86 to 125
In reply to Alex WilliamsonWith the 3 private slots, this gives us a nice round 128 slots total. The primary motivation for this is to support more assigned devices. Each assigned device can theoretically use up to 8 slots (6 MMIO BARs, 1 ROM BAR, 1 spare for a split MSI-X table mapping) though it's far more typical for a device to use 3-4 slots. If we assume a typical VM uses a dozen slots for non-assigned devices purposes, we should always be able to support 14 worst case assigned devices or 28 to 37 typical devices.
--- arch/x86/include/asm/kvm_host.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -31,7 +31,7 @@#define KVM_MAX_VCPUS 254#define KVM_SOFT_MAX_VCPUS 160 -#define KVM_USER_MEM_SLOTS 32 +#define KVM_USER_MEM_SLOTS 125/* memory slots that are not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 3#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)
Re: [PATCH v2 01/10] kvm: Restrict non-existing slot state transitions
In reply to Alex WilliamsonQuoted 27 lines · Alex Williamson
The API documentation states:
When changing an existing slot, it may be moved in the guest physical memory space, or its flags may be modified.
An "existing slot" requires a non-zero npages (
memory_size). The only transition we should therefore allow for a non-existing slot should be to create the slot, which includes setting a non-zeromemory_size. We currently allow calls to modify non-existing slots, which is pointless, confusing, and possibly wrong.With this we know that the invalidation path of
__kvm_set_memory_regionis always for a delete or move and never for adding a zero size slot.Signed-off-by Alex Williamson <[email protected]>--- virt/kvm/kvm_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +0
-0@@ -753,10 +753,15 @@ int __kvm_set_memory_region(struct kvm *kvm, new.npages = npages; new.flags = mem->flags;
virt/kvm/kvm_main.c int __kvm_set_memory_region(struct kvm *kvm, · line 764
Quoted 10 lines
virt/kvm/kvm_main.c +6
-1-/* Disallow changing a memory slot's size. */ +/* +* Disallow changing a memory slot's size or changing anything about +* zero sized slots that doesn't involve making them non-zero. +*/ r = -EINVAL ;if (npages && old.npages && npages != old.npages) goto out_free; +if (!npages && !old.npages) + goto out_free;
/* General sanity checks */
if (mem->memory_size & (PAGE_SIZE - 1))
goto out;
if (mem->guest_phys_addr & (PAGE_SIZE - 1))
goto out;
if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr)
goto out;If mem->memory_size is zero, then check above fails.
If mem->memory_size > 0, it must be at least PAGE_SIZE, in which case npages == 1.
Which means the code does not allow changes to non-existing slots. Yes?
Re: [PATCH v2 01/10] kvm: Restrict non-existing slot state transitions
Quoted 48 lines · Marcelo Tosatti
Quoted 27 lines · Alex Williamson
The API documentation states:
When changing an existing slot, it may be moved in the guest physical memory space, or its flags may be modified.
An "existing slot" requires a non-zero npages (
memory_size). The only transition we should therefore allow for a non-existing slot should be to create the slot, which includes setting a non-zeromemory_size. We currently allow calls to modify non-existing slots, which is pointless, confusing, and possibly wrong.With this we know that the invalidation path of
__kvm_set_memory_regionis always for a delete or move and never for adding a zero size slot.Signed-off-by Alex Williamson <[email protected]>--- virt/kvm/kvm_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +0
-0@@ -753,10 +753,15 @@ int __kvm_set_memory_region(struct kvm *kvm, new.npages = npages; new.flags = mem->flags;virt/kvm/kvm_main.c int __kvm_set_memory_region(struct kvm *kvm, · line 764
Quoted 10 lines
virt/kvm/kvm_main.c +6
-1-/* Disallow changing a memory slot's size. */ +/* +* Disallow changing a memory slot's size or changing anything about +* zero sized slots that doesn't involve making them non-zero. +*/ r = -EINVAL ;if (npages && old.npages && npages != old.npages) goto out_free; +if (!npages && !old.npages) + goto out_free;/* General sanity checks */ if (mem->memory_size & (PAGE_SIZE - 1)) goto out; if (mem->guest_phys_addr & (PAGE_SIZE - 1)) goto out; if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr) goto out;
Quoted 1 line
If
mem->memory_sizeis zero, then check above fails.
Err, no read a "<=" while writing that.
Quoted 6 lines
If
mem->memory_size> 0, it must be at leastPAGE_SIZE, in which case npages == 1.Which means the code does not allow changes to non-existing slots. Yes?
Quoted 20 lines · Alex Williamson
v2: Update 02/10 to not check
userspace_addrwhen slot is removed.
Yoshikawa-san withdrew objection to increaseslot_bitmapprior to his series to removeslot_bitmap.This series does away with any kind of complicated resizing of the slot array and simply does a one time increase. I do compact struct
kvm_memory_slota bit to take better advantage of the space we are using. This reduces each slot from 64 bytes (x86_64) to 56 bytes. By enforcing the API around valid operations for an unused slot and fields that can be modified runtime, I found and was able to fix a bug in iommu mapping for slots. The renames enabled me to find the previously posted bug fix for catching slot overlaps.As mentioned in the series, the primary motivation for increasing memory slots is assigned devices. With this, I've been able to assign 30 devices to a single VM and could have gone further, but ran out of SRIOV VFs. Typical devices use anywhere from 2-4 slots and max out at 8 slots. 125 user slots (3 private slots) allows us to support between 28 and 56 typical devices per VM.
Quoted 4 lines
Tested on
x86_64, compiled on ia64, powerpc, and s390.Thanks,
Alex
Re: [PATCH v2 01/10] kvm: Restrict non-existing slot state transitions
In reply to Marcelo TosattiQuoted 46 lines · Marcelo Tosatti
Quoted 45 lines · Marcelo Tosatti
Quoted 27 lines · Alex Williamson
The API documentation states:
When changing an existing slot, it may be moved in the guest physical memory space, or its flags may be modified.
An "existing slot" requires a non-zero npages (
memory_size). The only transition we should therefore allow for a non-existing slot should be to create the slot, which includes setting a non-zeromemory_size. We currently allow calls to modify non-existing slots, which is pointless, confusing, and possibly wrong.With this we know that the invalidation path of
__kvm_set_memory_regionis always for a delete or move and never for adding a zero size slot.Signed-off-by Alex Williamson <[email protected]>--- virt/kvm/kvm_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +0
-0@@ -753,10 +753,15 @@ int __kvm_set_memory_region(struct kvm *kvm, new.npages = npages; new.flags = mem->flags;virt/kvm/kvm_main.c int __kvm_set_memory_region(struct kvm *kvm, · line 764
Quoted 10 lines
virt/kvm/kvm_main.c +6
-1-/* Disallow changing a memory slot's size. */ +/* +* Disallow changing a memory slot's size or changing anything about +* zero sized slots that doesn't involve making them non-zero. +*/ r = -EINVAL ;if (npages && old.npages && npages != old.npages) goto out_free; +if (!npages && !old.npages) + goto out_free;/* General sanity checks */ if (mem->memory_size & (PAGE_SIZE - 1)) goto out; if (mem->guest_phys_addr & (PAGE_SIZE - 1)) goto out;
Quoted 6 lines
Quoted 4 lines
if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr) goto out;If
mem->memory_sizeis zero, then check above fails.Err, no read a "<=" while writing that.
;) Thanks for the double check,
Alex
Quoted 6 lines
Quoted 6 lines
If
mem->memory_size> 0, it must be at leastPAGE_SIZE, in which case npages == 1.Which means the code does not allow changes to non-existing slots. Yes?
Quoted 20 lines · Alex Williamson
v2: Update 02/10 to not check
userspace_addrwhen slot is removed.
Yoshikawa-san withdrew objection to increaseslot_bitmapprior to his series to removeslot_bitmap.This series does away with any kind of complicated resizing of the slot array and simply does a one time increase. I do compact struct
kvm_memory_slota bit to take better advantage of the space we are using. This reduces each slot from 64 bytes (x86_64) to 56 bytes. By enforcing the API around valid operations for an unused slot and fields that can be modified runtime, I found and was able to fix a bug in iommu mapping for slots. The renames enabled me to find the previously posted bug fix for catching slot overlaps.As mentioned in the series, the primary motivation for increasing memory slots is assigned devices. With this, I've been able to assign 30 devices to a single VM and could have gone further, but ran out of SRIOV VFs. Typical devices use anywhere from 2-4 slots and max out at 8 slots. 125 user slots (3 private slots) allows us to support between 28 and 56 typical devices per VM.
Quoted 5 lines
Tested on
x86_64, compiled on ia64, powerpc, and s390.Thanks,
Alex
Quoted 15 lines
---
Alex Williamson (10):
kvm: Restrict non-existing slot state transitions
kvm: Checkuserspace_addrwhen modifying a memory slot
kvm: Fix iommu map/unmap to handle memory slot moves
kvm: Minor memory slot optimization
kvm: RenameKVM_MEMORY_SLOTS->KVM_USER_MEM_SLOTS
kvm: MakeKVM_PRIVATE_MEM_SLOTSoptional
kvm: struct kvm_memory_slot.user_alloc -> bool
kvm: struct kvm_memory_slot.flags -> u32
kvm: struct kvm_memory_slot.id -> short
kvm: Increase user memory slots on x86 to 125
Quoted 14 lines
arch/ia64/include/asm/kvm_host.h | 4 -- arch/ia64/kvm/kvm-ia64.c | 8 ++-- arch/powerpc/include/asm/kvm_host.h | 6 +-- arch/powerpc/kvm/book3s_hv.c | 2 - arch/powerpc/kvm/powerpc.c | 4 +- arch/s390/include/asm/kvm_host.h | 4 -- arch/s390/kvm/kvm-s390.c | 4 +- arch/x86/include/asm/kvm_host.h | 8 ++-- arch/x86/include/asm/vmx.h | 6 +-- arch/x86/kvm/vmx.c | 6 +-- arch/x86/kvm/x86.c | 10 ++--- include/linux/kvm_host.h | 24 +++++++----- virt/kvm/kvm_main.c | 72 +++++++++++++++++++++++------------ 13 files changed, 90 insertions(+), 68 deletions(-)
--
Gleb.
Marcelo Tosatti
AppliedQuoted 20 lines · Alex Williamson
v2: Update 02/10 to not check
userspace_addrwhen slot is removed.
Yoshikawa-san withdrew objection to increaseslot_bitmapprior to his series to removeslot_bitmap.This series does away with any kind of complicated resizing of the slot array and simply does a one time increase. I do compact struct
kvm_memory_slota bit to take better advantage of the space we are using. This reduces each slot from 64 bytes (x86_64) to 56 bytes. By enforcing the API around valid operations for an unused slot and fields that can be modified runtime, I found and was able to fix a bug in iommu mapping for slots. The renames enabled me to find the previously posted bug fix for catching slot overlaps.As mentioned in the series, the primary motivation for increasing memory slots is assigned devices. With this, I've been able to assign 30 devices to a single VM and could have gone further, but ran out of SRIOV VFs. Typical devices use anywhere from 2-4 slots and max out at 8 slots. 125 user slots (3 private slots) allows us to support between 28 and 56 typical devices per VM.
Quoted 4 lines
Tested on
x86_64, compiled on ia64, powerpc, and s390.Thanks,
Alex
Applied, thanks.