Skip to content

KVM: optimize memslots searching and cache GPN to GFN

Xiao Guangrong kvm lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest6 of 7 patches 7 repliesIn review

6 of 7 patches are in this archive. The cover letter announces 7.

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7KVM: cleanup memslot_id functioninclude/linux/kvm_host.h virt/kvm/kvm_main.c+6 -180 replies
2/7KVM: introduce KVM_MEM_SLOTS_NUM macroarch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu.c include/linux/kvm_host.h +1 more+9 -50 replies
4/7KVM: sort memslots and use binary search to search the right slotinclude/linux/kvm_host.h virt/kvm/kvm_main.c+65 -2230 replies
5/7KVM: cache the last used slotinclude/linux/kvm_host.h virt/kvm/kvm_main.c+10 -21 reply
6/7KVM: cleanup traversal used slotsarch/ia64/kvm/kvm-ia64.c arch/x86/kvm/mmu.c arch/x86/kvm/x86.c +3 more+26 -230 replies
7/7KVM: MMU: cache guest page number to guest frame numberarch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu.c arch/x86/kvm/mmutrace.h +1 more+196 -63 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/7] KVM: optimize memslots searching and cache GPN to GFN

I noticed paging64/32_walk_addr_generic and gfn_to_memslot are very hot in kvm modules:

# perf report
# Events: 38K cycles
#
# Overhead          Command       Shared Object  Symbol
# ........  ...............  ..................  ......
#
     6.73%  qemu-system-x86  [kernel.kallsyms]   [k] native_read_tsc
     4.51%  qemu-system-x86  [kernel.kallsyms]   [k] delay_tsc
     3.93%  qemu-system-x86  [kvm]               [k] paging64_walk_addr_generic
     3.91%  qemu-system-x86  [kernel.kallsyms]   [k] arch_local_irq_restore
     3.28%  qemu-system-x86  [kvm_intel]         [k] vmx_vcpu_run
     3.12%  qemu-system-x86  [kernel.kallsyms]   [k] lock_acquire
     3.10%  qemu-system-x86  [kernel.kallsyms]   [k] paravirt_read_tsc
     2.70%  qemu-system-x86  [kernel.kallsyms]   [k] copy_user_generic_string
     2.30%  qemu-system-x86  [kernel.kallsyms]   [k] check_flags
     2.15%  qemu-system-x86  [kernel.kallsyms]   [k] do_raw_spin_lock
     2.03%  qemu-system-x86  [kvm]               [k] gfn_to_memslot

The aim of this patchset is to reduce the overload of these two functions, it does:
- sort memslots by ->base_gfn and use binary search to search the right slot which is implemented by patch 1 ~ patch 7
- cache guest page number to frame number to reduce *_walk_addr_generic called, and patch 7 does it.

After the patchset:
# Events: 36K cycles
#
# Overhead Command Shared Object Symbol

# ........  ...............  ..................  ......
#
     5.64%  qemu-system-x86  [kernel.kallsyms]   [k] native_read_tsc
     4.27%  qemu-system-x86  [kvm_intel]         [k] vmx_vcpu_run
     3.85%  qemu-system-x86  [kernel.kallsyms]   [k] delay_tsc
     3.32%  qemu-system-x86  [kernel.kallsyms]   [k] arch_local_irq_restore
     2.68%  qemu-system-x86  [kernel.kallsyms]   [k] paravirt_read_tsc
     2.41%  qemu-system-x86  [kernel.kallsyms]   [k] lock_acquire
     2.25%  qemu-system-x86  [kernel.kallsyms]   [k] copy_user_generic_string
     2.14%  qemu-system-x86  [kvm]               [k] kvm_mmu_pte_write
     1.82%  qemu-system-x86  [kernel.kallsyms]   [k] check_flags
     1.80%  qemu-system-x86  [kvm]               [k] paging64_walk_addr_generic
[......]
[......] 
     0.81%  qemu-system-x86  [kvm]               [k] gfn_to_memslot

There is the Unixbecn(the parameter is -c 10 fsdisk) test report:
Before patchset:

	2090.5
	1882.5
	1845.2

After vtlb patch:

	2237.4
	2151.8
	2118.0

After memslots sorted patches:

	2392.5
	2485.9
	2416.1

Read the cover letter in its thread