Skip to contentSkip to conversation

KVM: Use kvcalloc() to allocate lpage_info arrays and dirty bitmaps

PATCH In review Mushahid Hussain 2 participants 2 messages Aug 15 – Sep 28

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: Use kvcalloc() to allocate lpage_info arrays and dirty bitmaps

Use kvcalloc() instead of __vcalloc() for the lpage_info arrays and memslot dirty bitmaps, and switch their frees from vfree() back to kvfree(). kvcalloc() serves sub-page requests from the slab and falls back to vmalloc for anything larger; kvfree() handles either allocator. The vfree() pairing came from commit a952d608f0be ("KVM:
Use vfree for memory allocated by vcalloc()/__vcalloc()").

Commit 37b2a6510a48 ("KVM: use __vcalloc for very large allocations") moved these sites onto __vcalloc() to escape kvmalloc()'s INT_MAX size cap, since memslot metadata sizes scale with the memslot size and can exceed 2 GiB. Neither site can reach the cap: lpage_info holds one 4 byte entry per hugepage granule (per 2 MiB at level 2, per 1 GiB at level 3), so reaching INT_MAX would take a 1 PiB memslot, while userspace slots are capped at 8 TiB by KVM_MEM_MAX_NR_PAGES and x86's internal slots at 4 GiB by __x86_set_memory_region()'s u32 size. The dirty bitmap is two bitmaps at one bit per page, npages/4 bytes, at most 512 MiB, and internal slots never have one because kvm_set_internal_memslot() rejects any flags.

__vcalloc() makes every allocation at least a page, so a single page memslot consumes 8 KiB of vmalloc for 8 bytes of lpage_info and another 4 KiB for a 16 byte dirty bitmap when dirty logging is enabled. This overhead scales with the number of slots and VMs on a host, adding up to memory pressure when guest address spaces are fragmented into small slots.

The rmap and gfn_write_track arrays keep __vcalloc() and vfree():
the 4K rmap and gfn_write_track are per-page arrays, 8 and 2 bytes per 4 KiB page, which legitimately cross INT_MAX below the 8 TiB slot ceiling; the smaller higher-level rmaps share the 4K rmap's allocation loop; and none of them allocate under the TDP MMU, where the waste above was observed.

Fixes 37b2a6510a48 ("KVM: use __vcalloc for very large allocations")
Assisted-by Kiro:claude-fable-5
---
 arch/x86/kvm/x86.c  | 6 +++---
 virt/kvm/kvm_main.c | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/x86.c +3 -3
index afcac1042947..77567aa20d83 100644
@@ -13488,7 +13488,7 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot)
 	memslot_rmap_free(slot);
 
 	for (i = 1; i < KVM_NR_PAGE_SIZES; ++i) {
-		vfree(slot->arch.lpage_info[i - 1]);
+		kvfree(slot->arch.lpage_info[i - 1]);
 		slot->arch.lpage_info[i - 1] = NULL;
 	}
 
@@ -13544,7 +13544,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm,
 
 		lpages = __kvm_mmu_slot_lpages(slot, npages, level);
 
-		linfo = __vcalloc(lpages, sizeof(*linfo), GFP_KERNEL_ACCOUNT);
+		linfo = kvcalloc(lpages, sizeof(*linfo), GFP_KERNEL_ACCOUNT);
 		if (!linfo)
 			goto out_free;
 
@@ -13580,7 +13580,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm,
 	memslot_rmap_free(slot);
 
 	for (i = 1; i < KVM_NR_PAGE_SIZES; ++i) {
-		vfree(slot->arch.lpage_info[i - 1]);
+		kvfree(slot->arch.lpage_info[i - 1]);
 		slot->arch.lpage_info[i - 1] = NULL;
 	}
 	return -ENOMEM;
virt/kvm/kvm_main.c +2 -2
index e44c20c04961..52af1ebdd14c 100644
@@ -927,7 +927,7 @@ static void kvm_destroy_dirty_bitmap(struct kvm_memory_slot *memslot)
 	if (!memslot->dirty_bitmap)
 		return;
 
-	vfree(memslot->dirty_bitmap);
+	kvfree(memslot->dirty_bitmap);
 	memslot->dirty_bitmap = NULL;
 }
 
@@ -1422,7 +1422,7 @@ static int kvm_alloc_dirty_bitmap(struct kvm_memory_slot *memslot)
 {
 	unsigned long dirty_bytes = kvm_dirty_bitmap_bytes(memslot);
 
-	memslot->dirty_bitmap = __vcalloc(2, dirty_bytes, GFP_KERNEL_ACCOUNT);
+	memslot->dirty_bitmap = kvcalloc(2, dirty_bytes, GFP_KERNEL_ACCOUNT);
 	if (!memslot->dirty_bitmap)
 		return -ENOMEM;
 

base-commit: 8cdeaa50eae8dad34885515f62559ee83e7e8dda

Signature · 1 line
2.47.3
#1 lore raw
Quoted 6 lines · Mushahid Hussain

__vcalloc() makes every allocation at least a page, so a single page memslot consumes 8 KiB of vmalloc for 8 bytes of lpage_info and another 4 KiB for a 16 byte dirty bitmap when dirty logging is enabled. This overhead scales with the number of slots and VMs on a host, adding up to memory pressure when guest address spaces are fragmented into small slots.

If memslots are fragmented that badly, then the rmaps are also going to be extremely wasteful.

Quoted 5 lines

The rmap and gfn_write_track arrays keep __vcalloc() and vfree():
the 4K rmap and gfn_write_track are per-page arrays, 8 and 2 bytes per 4 KiB page, which legitimately cross INT_MAX below the 8 TiB slot ceiling; the smaller higher-level rmaps share the 4K rmap's allocation loop; and none of them allocate under the TDP MMU,

Until nested virtualization gets used, and then KVM pays the overhead cost for every memslot.

Rather than flip-flop because of a semi-arbitrary limit that has nothing to do with KVM, I think we should provide dedicated KVM APIs for allocating memslot metadata, and pick a pivot that makes sense for KVM. Or just pivot on INT_MAX to route to kv() vs. v() to play nice with the "not crazy" rule.

Quoted 1 line

where the waste above was observed.