KVM monolithic v3
Subsystems (7)
cover KVM monolithic v3
Hello,
Here's the rebase of KVM monolithic on 5.4-rc6 after improving the Kbuild as suggested in the v2 review.
This should have corrected also the non-x86 KVM builds (not verified so it may not have full coverage yet, but I should have fixed all issues reported by kbot so far).
The effect of the patchset is visible at the end of the below pdf. I dropped CPUID so the results are still an exact match only for the last two bench slides which are the only important ones in real life.
https://people.redhat.com/~aarcange/slides/2019-KVM-monolithic.pdf
https://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git/log/?h=kvm-mono3
Andrea Arcangeli (13):
KVM: monolithic: x86: remove kvm.ko
KVM: monolithic: x86: convert the kvm_x86_ops and kvm_pmu_ops methods
to external functions
kvm: monolithic: fixup x86-32 build
KVM: monolithic: x86: handle the request_immediate_exit variation
KVM: monolithic: add more section prefixes
KVM: monolithic: x86: remove __exit section prefix from
machine_unsetup
KVM: monolithic: x86: remove __init section prefix from
kvm_x86_cpu_has_kvm_support
KVM: monolithic: remove exports
KVM: monolithic: x86: drop the kvm_pmu_ops structure
KVM: x86: optimize more exit handlers in vmx.c
KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers
KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers
x86: retpolines: eliminate retpoline from msr event handlers
arch/powerpc/kvm/book3s.c | 2 +- arch/x86/events/intel/core.c | 11 + arch/x86/include/asm/kvm_host.h | 210 ++++++++- arch/x86/kvm/Kconfig | 30 +- arch/x86/kvm/Makefile | 5 +- arch/x86/kvm/cpuid.c | 27 +- arch/x86/kvm/hyperv.c | 8 +- arch/x86/kvm/irq.c | 4 - arch/x86/kvm/irq_comm.c | 2 - arch/x86/kvm/kvm_cache_regs.h | 10 +- arch/x86/kvm/lapic.c | 46 +- arch/x86/kvm/mmu.c | 50 +- arch/x86/kvm/mmu.h | 4 +- arch/x86/kvm/mtrr.c | 2 - arch/x86/kvm/pmu.c | 27 +- arch/x86/kvm/pmu.h | 37 +- arch/x86/kvm/pmu_amd.c | 43 +- arch/x86/kvm/svm.c | 683 ++++++++++++++++----------- arch/x86/kvm/trace.h | 4 +- arch/x86/kvm/vmx/nested.c | 84 ++-- arch/x86/kvm/vmx/pmu_intel.c | 46 +- arch/x86/kvm/vmx/vmx.c | 807 ++++++++++++++++++-------------- arch/x86/kvm/vmx/vmx.h | 39 +- arch/x86/kvm/x86.c | 418 ++++++----------- arch/x86/kvm/x86.h | 2 +- include/linux/kvm_host.h | 8 +- virt/kvm/arm/arm.c | 2 +- virt/kvm/eventfd.c | 1 - virt/kvm/kvm_main.c | 70 +-- 29 files changed, 1440 insertions(+), 1242 deletions(-)
PATCH 1/13 KVM: monolithic: x86: remove kvm.ko
This is the first commit of a patch series that aims to replace the modular kvm.ko kernel module with a monolithic kvm-intel/kvm-amd model. This change has the only possible cons of wasting some disk space in /lib/modules/. The pros are that it saves CPUS and some minor iTLB and RAM which are more scarse resources than disk space.
The pointer to function virtual template model cannot provide any runtime benefit because kvm-intel and kvm-amd can't be loaded at the same time.
This removes kvm.ko and it links and duplicates all kvm.ko objects to both kvm-amd and kvm-intel.
Linking both vmx and svm into the kernel at the same time isn't possible anymore or the kvm_x86/kvm_x86_pmu external function names would collide.
Explanation of Kbuild from Paolo Bonzini follows:
===
The left side of the "||" ensures that, if KVM=m, you can only choose module build for both KVM_INTEL and KVM_AMD. Having just "depends on KVM" would allow a pre-existing .config to choose the now-invalid combination
CONFIG_KVM=y
CONFIG_KVM_INTEL=y
CONFIG_KVM_AMD=y
The right side of the "||" part is just for documentation, to avoid that a selected symbol does not satisfy its dependencies.
====
--- arch/x86/kvm/Kconfig | 30 ++++++++++++++++++++++++++---- arch/x86/kvm/Makefile | 5 ++--- 2 files changed, 28 insertions(+), 7 deletions(-)
arch/x86/kvm/Kconfig +26 -4
@@ -59,9 +59,30 @@ config KVM If unsure, say N. +if KVM=y + +choice + prompt "KVM built-in support" + help + In order to build a kernel with support for both AMD and Intel + CPUs, you need to set CONFIG_KVM=m instead. + +config KVM_AMD_STATIC + select KVM_AMD + bool "AMD" + +config KVM_INTEL_STATIC + select KVM_INTEL + bool "Intel" + +endchoice + +endif + config KVM_INTEL - tristate "KVM for Intel processors support" - depends on KVM + tristate + prompt "KVM for Intel processors support" if KVM=m + depends on (KVM=m && m) || KVM_INTEL_STATIC # for perf_guest_get_msrs(): depends on CPU_SUP_INTEL ---help---
@@ -72,8 +93,9 @@ config KVM_INTEL will be called kvm-intel. config KVM_AMD - tristate "KVM for AMD processors support" - depends on KVM + tristate + prompt "KVM for AMD processors support" if KVM=m + depends on (KVM=m && m) || KVM_AMD_STATIC ---help--- Provides support for KVM on AMD processors equipped with the AMD-V (SVM) extensions.
arch/x86/kvm/Makefile +2 -3
@@ -12,9 +12,8 @@ kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \ hyperv.o page_track.o debugfs.o -kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o -kvm-amd-y += svm.o pmu_amd.o +kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o $(kvm-y) +kvm-amd-y += svm.o pmu_amd.o $(kvm-y) -obj-$(CONFIG_KVM) += kvm.o obj-$(CONFIG_KVM_INTEL) += kvm-intel.o obj-$(CONFIG_KVM_AMD) += kvm-amd.o
PATCH 2/13 KVM: monolithic: x86: convert the kvm_x86_ops and kvm_pmu_ops methods to external functions
In reply to Andrea ArcangeliThis replaces all kvm_x86_ops and kvm_pmu_ops pointer to functions with regular external functions that don't require indirect calls.
In practice this optimization results in a double digit percent reduction in the vmexit latency with the default retpoline spectre v2 mitigation enabled in the host.
When the host is booted with spectre_v2=off this still results in a best case measurable improvement up to 2% that varies wildly depending on the CPU vendor implementation under high frequency timer guest workloads. Possibly guest userland workloads stressing the BTB might see an improvement even on CPUs where the vmexit latency isn't reduced with all mitigations disabled.
To reduce the rejecting parts while tracking upstream, this doesn't attempt to entirely remove the kvm_x86_ops structure yet, that is meant for a later cleanup. The pmu ops have been already cleaned up in this patchset because it was left completely unused right after the conversion from pointer to functions to external functions.
Further incremental minor optimizations that weren't possible before are now enabled by the monolithic model. For example it is now possible to convert some of the small kvm_x86_* external methods to inline functions. However that will require more Makefile tweaks and so it is left for later.
This is a list of the most common retpolines executed in KVM on VMX under a guest workload triggering a high resolution timer SIGALRM flood before the monolithic KVM patchset is applied.
[..]
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
cancel_hv_timer.isra.46+44
restart_apic_timer+295
kvm_set_msr_common+1435
vmx_set_msr+478
handle_wrmsr+85
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 65382
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+1646
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 66164
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_read_l1_tsc+41
__kvm_wait_lapic_expire+60
vmx_vcpu_run.part.88+1091
vcpu_enter_guest+423
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 66199
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+4958
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 66227
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
restart_apic_timer+99
kvm_set_msr_common+1435
vmx_set_msr+478
handle_wrmsr+85
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 130619
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_read_l1_tsc+41
vmx_set_hv_timer+81
restart_apic_timer+99
kvm_set_msr_common+1435
vmx_set_msr+478
handle_wrmsr+85
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 130665
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_skip_emulated_instruction+49
handle_wrmsr+102
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 131020
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_skip_emulated_instruction+82
handle_wrmsr+102
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 131025
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
handle_wrmsr+85
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 131043
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
skip_emulated_instruction+48
kvm_skip_emulated_instruction+82
handle_wrmsr+102
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 131046
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+4009
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 132405
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rcx+33
vcpu_enter_guest+1689
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 197697
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vmx_vcpu_run.part.88+358
vcpu_enter_guest+423
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 198736
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+575
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 198771
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+423
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 198793
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+486
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 198801
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+168
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 198848
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68]: 397680
@total: 3816655
Here the same but on SVM:
[..]
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
clockevents_program_event+148
hrtimer_start_range_ns+528
start_sw_timer+356
restart_apic_timer+111
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 36031
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
lapic_next_event+28
clockevents_program_event+148
hrtimer_start_range_ns+528
start_sw_timer+356
restart_apic_timer+111
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 36063
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get+58
clockevents_program_event+84
hrtimer_try_to_cancel+168
hrtimer_cancel+21
kvm_set_lapic_tscdeadline_msr+43
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 36134
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
lapic_next_event+28
clockevents_program_event+148
hrtimer_try_to_cancel+168
hrtimer_cancel+21
kvm_set_lapic_tscdeadline_msr+43
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 36146
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
clockevents_program_event+148
hrtimer_try_to_cancel+168
hrtimer_cancel+21
kvm_set_lapic_tscdeadline_msr+43
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 36190
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get+58
clockevents_program_event+84
hrtimer_start_range_ns+528
start_sw_timer+356
restart_apic_timer+111
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 36281
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+1646
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 37752
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_read_l1_tsc+41
__kvm_wait_lapic_expire+60
svm_vcpu_run+1276
]: 37886
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+4958
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 37957
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_read_l1_tsc+41
start_sw_timer+302
restart_apic_timer+111
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 74358
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get+58
start_sw_timer+279
restart_apic_timer+111
kvm_set_msr_common+1435
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 74558
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_skip_emulated_instruction+82
msr_interception+356
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 74713
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_skip_emulated_instruction+49
msr_interception+356
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 74757
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
msr_interception+138
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 74795
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
kvm_get_rflags+28
svm_interrupt_allowed+50
vcpu_enter_guest+4009
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 75647
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+4009
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 75812
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rcx+33
vcpu_enter_guest+1689
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 112579
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+575
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 113371
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+423
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 113386
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+486
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 113414
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+168
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 113601
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
vcpu_enter_guest+772
kvm_arch_vcpu_ioctl_run+263
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68]: 227076
@total: 3829460
--- arch/x86/include/asm/kvm_host.h | 203 ++++++++- arch/x86/kvm/cpuid.c | 22 +- arch/x86/kvm/hyperv.c | 6 +- arch/x86/kvm/kvm_cache_regs.h | 10 +- arch/x86/kvm/lapic.c | 30 +- arch/x86/kvm/mmu.c | 26 +- arch/x86/kvm/mmu.h | 4 +- arch/x86/kvm/pmu.c | 24 +- arch/x86/kvm/pmu.h | 19 +- arch/x86/kvm/pmu_amd.c | 52 +-- arch/x86/kvm/svm.c | 664 ++++++++++++++++------------ arch/x86/kvm/trace.h | 4 +- arch/x86/kvm/vmx/nested.c | 84 ++-- arch/x86/kvm/vmx/pmu_intel.c | 55 ++- arch/x86/kvm/vmx/vmx.c | 739 ++++++++++++++++++-------------- arch/x86/kvm/vmx/vmx.h | 39 +- arch/x86/kvm/x86.c | 308 ++++++------- arch/x86/kvm/x86.h | 2 +- 18 files changed, 1361 insertions(+), 930 deletions(-)
arch/x86/include/asm/kvm_host.h +197 -6
@@ -998,6 +998,197 @@ struct kvm_lapic_irq {bool msi_redir_hint; }; +extern __init int kvm_x86_cpu_has_kvm_support (void ); +extern __init int kvm_x86_disabled_by_bios (void ); +extern int kvm_x86_hardware_enable (void ); +extern void kvm_x86_hardware_disable (void ); +extern __init int kvm_x86_check_processor_compatibility (void ); +extern __init int kvm_x86_hardware_setup (void ); +extern __exit void kvm_x86_hardware_unsetup(void); +extern bool kvm_x86_cpu_has_accelerated_tpr (void ); +extern bool kvm_x86_has_emulated_msr (int index); +extern void kvm_x86_cpuid_update (struct kvm_vcpu *vcpu); +extern struct kvm *kvm_x86_vm_alloc (void ); +extern void kvm_x86_vm_free (struct kvm *kvm); +extern int kvm_x86_vm_init (struct kvm *kvm); +extern void kvm_x86_vm_destroy (struct kvm *kvm); +/* Create, but do not attach this VCPU */ +extern struct kvm_vcpu *kvm_x86_vcpu_create (struct kvm *kvm,unsigned id); +extern void kvm_x86_vcpu_free (struct kvm_vcpu *vcpu); +extern void kvm_x86_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); +extern void kvm_x86_prepare_guest_switch (struct kvm_vcpu *vcpu); +extern void kvm_x86_vcpu_load (struct kvm_vcpu *vcpu,int cpu); +extern void kvm_x86_vcpu_put (struct kvm_vcpu *vcpu); +extern void kvm_x86_update_bp_intercept (struct kvm_vcpu *vcpu); +extern int kvm_x86_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr); +extern int kvm_x86_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr); +extern u64 kvm_x86_get_segment_base (struct kvm_vcpu *vcpu,int seg); +extern void kvm_x86_get_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var, +int seg); +extern int kvm_x86_get_cpl (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var, +int seg); +extern void kvm_x86_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l); +extern void kvm_x86_decache_cr0_guest_bits (struct kvm_vcpu *vcpu); +extern void kvm_x86_decache_cr3 (struct kvm_vcpu *vcpu); +extern void kvm_x86_decache_cr4_guest_bits (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0); +extern void kvm_x86_set_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3); +extern int kvm_x86_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); +extern void kvm_x86_set_efer (struct kvm_vcpu *vcpu,u64 efer); +extern void kvm_x86_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +extern void kvm_x86_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +extern void kvm_x86_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +extern void kvm_x86_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +extern u64 kvm_x86_get_dr6 (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_dr6 (struct kvm_vcpu *vcpu,unsigned long value); +extern void kvm_x86_sync_dirty_debug_regs (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_dr7 (struct kvm_vcpu *vcpu,unsigned long value); +extern void kvm_x86_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg); +extern unsigned long kvm_x86_get_rflags (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_rflags (struct kvm_vcpu *vcpu,unsigned long rflags); +extern void kvm_x86_tlb_flush (struct kvm_vcpu *vcpu,bool invalidate_gpa); +extern int kvm_x86_tlb_remote_flush (struct kvm *kvm); +extern int kvm_x86_tlb_remote_flush_with_range (struct kvm *kvm, +struct kvm_tlb_range *range); +/* +* Flush any TLB entries associated with the given GVA. +* Does not need to flush GPA->HPA mappings. +* Can potentially get non-canonical addresses through INVLPGs, which +* the implementation may choose to ignore if appropriate. +*/ +extern void kvm_x86_tlb_flush_gva (struct kvm_vcpu *vcpu,gva_t addr); +extern void kvm_x86_run (struct kvm_vcpu *vcpu); +extern int kvm_x86_handle_exit (struct kvm_vcpu *vcpu); +extern int kvm_x86_skip_emulated_instruction (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask); +extern u32 kvm_x86_get_interrupt_shadow (struct kvm_vcpu *vcpu); +extern void kvm_x86_patch_hypercall (struct kvm_vcpu *vcpu, +unsigned char *hypercall_addr); +extern void kvm_x86_set_irq (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_nmi (struct kvm_vcpu *vcpu); +extern void kvm_x86_queue_exception (struct kvm_vcpu *vcpu); +extern void kvm_x86_cancel_injection (struct kvm_vcpu *vcpu); +extern int kvm_x86_interrupt_allowed (struct kvm_vcpu *vcpu); +extern int kvm_x86_nmi_allowed (struct kvm_vcpu *vcpu); +extern bool kvm_x86_get_nmi_mask (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_nmi_mask (struct kvm_vcpu *vcpu,bool masked); +extern void kvm_x86_enable_nmi_window (struct kvm_vcpu *vcpu); +extern void kvm_x86_enable_irq_window (struct kvm_vcpu *vcpu); +extern void kvm_x86_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr, +int irr); +extern bool kvm_x86_get_enable_apicv (struct kvm_vcpu *vcpu); +extern void kvm_x86_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu); +extern void kvm_x86_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr); +extern void kvm_x86_hwapic_isr_update (struct kvm_vcpu *vcpu,int isr); +extern bool kvm_x86_guest_apic_has_interrupt (struct kvm_vcpu *vcpu); +extern void kvm_x86_load_eoi_exitmap (struct kvm_vcpu *vcpu, +u64 *eoi_exit_bitmap); +extern void kvm_x86_set_virtual_apic_mode (struct kvm_vcpu *vcpu); +extern void kvm_x86_set_apic_access_page_addr (struct kvm_vcpu *vcpu, +hpa_t hpa); +extern void kvm_x86_deliver_posted_interrupt (struct kvm_vcpu *vcpu, +int vector); +extern int kvm_x86_sync_pir_to_irr (struct kvm_vcpu *vcpu); +extern int kvm_x86_set_tss_addr (struct kvm *kvm,unsigned int addr); +extern int kvm_x86_set_identity_map_addr (struct kvm *kvm,u64 ident_addr); +extern int kvm_x86_get_tdp_level (struct kvm_vcpu *vcpu); +extern u64 kvm_x86_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio); +extern int kvm_x86_get_lpage_level (void ); +extern bool kvm_x86_rdtscp_supported (void ); +extern bool kvm_x86_invpcid_supported (void ); +extern void kvm_x86_set_tdp_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3); +extern void kvm_x86_set_supported_cpuid (u32 func, +struct kvm_cpuid_entry2 *entry); +extern bool kvm_x86_has_wbinvd_exit (void ); +extern u64 kvm_x86_read_l1_tsc_offset (struct kvm_vcpu *vcpu); +/* Returns actual tsc_offset set in active VMCS */ +extern u64 kvm_x86_write_l1_tsc_offset (struct kvm_vcpu *vcpu,u64 offset); +extern void kvm_x86_get_exit_info (struct kvm_vcpu *vcpu,u64 *info1, +u64 *info2); +extern int kvm_x86_check_intercept (struct kvm_vcpu *vcpu, +struct x86_instruction_info *info, +enum x86_intercept_stage stage); +extern void kvm_x86_handle_exit_irqoff (struct kvm_vcpu *vcpu); +extern bool kvm_x86_mpx_supported (void ); +extern bool kvm_x86_xsaves_supported (void ); +extern bool kvm_x86_umip_emulated (void ); +extern bool kvm_x86_pt_supported (void ); +extern int kvm_x86_check_nested_events (struct kvm_vcpu *vcpu, +bool external_intr); +extern void kvm_x86_request_immediate_exit (struct kvm_vcpu *vcpu); +extern void kvm_x86_sched_in (struct kvm_vcpu *kvm,int cpu); +/* +* Arch-specific dirty logging hooks. These hooks are only supposed to +* be valid if the specific arch has hardware-accelerated dirty logging +* mechanism. Currently only for PML on VMX. +* +* - slot_enable_log_dirty: +* called when enabling log dirty mode for the slot. +* - slot_disable_log_dirty: +* called when disabling log dirty mode for the slot. +* also called when slot is created with log dirty disabled. +* - flush_log_dirty: +* called before reporting dirty_bitmap to userspace. +* - enable_log_dirty_pt_masked: +* called when reenabling log dirty for the GFNs in the mask after +* corresponding bits are cleared in slot->dirty_bitmap. +*/ +extern void kvm_x86_slot_enable_log_dirty (struct kvm *kvm, +struct kvm_memory_slot *slot); +extern void kvm_x86_slot_disable_log_dirty (struct kvm *kvm, +struct kvm_memory_slot *slot); +extern void kvm_x86_flush_log_dirty (struct kvm *kvm); +extern void kvm_x86_enable_log_dirty_pt_masked (struct kvm *kvm, +struct kvm_memory_slot *slot, +gfn_t offset, +unsigned long mask); +extern int kvm_x86_write_log_dirty (struct kvm_vcpu *vcpu); +/* +* Architecture specific hooks for vCPU blocking due to +* HLT instruction. +* Returns for .pre_block(): +* - 0 means continue to block the vCPU. +* - 1 means we cannot block the vCPU since some event +* happens during this period, such as, 'ON' bit in +* posted-interrupts descriptor is set. +*/ +extern int kvm_x86_pre_block (struct kvm_vcpu *vcpu); +extern void kvm_x86_post_block (struct kvm_vcpu *vcpu); +extern void kvm_x86_vcpu_blocking (struct kvm_vcpu *vcpu); +extern void kvm_x86_vcpu_unblocking (struct kvm_vcpu *vcpu); +extern int kvm_x86_update_pi_irte (struct kvm *kvm,unsigned int host_irq, +uint32_t guest_irq,bool set); +extern void kvm_x86_apicv_post_state_restore (struct kvm_vcpu *vcpu); +extern bool kvm_x86_dy_apicv_has_pending_interrupt (struct kvm_vcpu *vcpu); +extern int kvm_x86_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired); +extern void kvm_x86_cancel_hv_timer (struct kvm_vcpu *vcpu); +extern void kvm_x86_setup_mce (struct kvm_vcpu *vcpu); +extern int kvm_x86_get_nested_state (struct kvm_vcpu *vcpu, +struct kvm_nested_state __user *user_kvm_nested_state, +unsigned user_data_size); +extern int kvm_x86_set_nested_state (struct kvm_vcpu *vcpu, +struct kvm_nested_state __user *user_kvm_nested_state, +struct kvm_nested_state *kvm_state); +extern bool kvm_x86_get_vmcs12_pages (struct kvm_vcpu *vcpu); +extern int kvm_x86_smi_allowed (struct kvm_vcpu *vcpu); +extern int kvm_x86_pre_enter_smm (struct kvm_vcpu *vcpu,char *smstate); +extern int kvm_x86_pre_leave_smm (struct kvm_vcpu *vcpu,const char *smstate); +extern int kvm_x86_enable_smi_window (struct kvm_vcpu *vcpu); +extern int kvm_x86_mem_enc_op (struct kvm *kvm,void __user *argp); +extern int kvm_x86_mem_enc_reg_region (struct kvm *kvm, +struct kvm_enc_region *argp); +extern int kvm_x86_mem_enc_unreg_region (struct kvm *kvm, +struct kvm_enc_region *argp); +extern int kvm_x86_get_msr_feature (struct kvm_msr_entry *entry); +extern int kvm_x86_nested_enable_evmcs (struct kvm_vcpu *vcpu, +uint16_t *vmcs_version); +extern uint16_t kvm_x86_nested_get_evmcs_version (struct kvm_vcpu *vcpu); +extern bool kvm_x86_need_emulation_on_page_fault (struct kvm_vcpu *vcpu); +extern bool kvm_x86_apic_init_signal_blocked (struct kvm_vcpu *vcpu); +extern int kvm_x86_enable_direct_tlbflush (struct kvm_vcpu *vcpu); +struct kvm_x86_ops {int (*cpu_has_kvm_support)(void );/* __init */ int (*disabled_by_bios)(void );/* __init */
@@ -1225,19 +1416,19 @@ extern struct kmem_cache *x86_fpu_cache;#define __KVM_HAVE_ARCH_VM_ALLOCstatic inline struct kvm *kvm_arch_alloc_vm (void ) { -return kvm_x86_ops->vm_alloc (); +return kvm_x86_vm_alloc (); }static inline void kvm_arch_free_vm (struct kvm *kvm) { -return kvm_x86_ops->vm_free (kvm); +return kvm_x86_vm_free (kvm); }#define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBstatic inline int kvm_arch_flush_remote_tlb (struct kvm *kvm) {if (kvm_x86_ops->tlb_remote_flush && - !kvm_x86_ops->tlb_remote_flush (kvm)) + !kvm_x86_tlb_remote_flush (kvm))return 0 ;else return -ENOTSUPP ;
@@ -1322,7 +1513,7 @@ extern u64 kvm_mce_cap_supported;* * EMULTYPE_SKIP - Set when emulating solely to skip an instruction, i.e. to * decode the instruction length. For use *only* by -* kvm_x86_ops->skip_emulated_instruction() implementations. +* kvm_x86_skip_emulated_instruction() implementations. * * EMULTYPE_ALLOW_RETRY - Set when the emulator should resume the guest to * retry native execution under certain conditions.
@@ -1608,13 +1799,13 @@ static inline bool kvm_irq_is_postable(struct kvm_lapic_irq *irq)static inline void kvm_arch_vcpu_blocking (struct kvm_vcpu *vcpu) {if (kvm_x86_ops->vcpu_blocking) - kvm_x86_ops->vcpu_blocking (vcpu); +kvm_x86_vcpu_blocking (vcpu); }static inline void kvm_arch_vcpu_unblocking (struct kvm_vcpu *vcpu) {if (kvm_x86_ops->vcpu_unblocking) - kvm_x86_ops->vcpu_unblocking (vcpu); +kvm_x86_vcpu_unblocking (vcpu); }static inline void kvm_arch_vcpu_block_finish (struct kvm_vcpu *vcpu) {}
arch/x86/kvm/cpuid.c +11 -11
@@ -48,7 +48,7 @@ static u32 xstate_required_size(u64 xstate_bv, bool compacted)bool kvm_mpx_supported (void ) {return ((host_xcr0 & (XFEATURE_MASK_BNDREGS |XFEATURE_MASK_BNDCSR )) - && kvm_x86_ops->mpx_supported ()); + &&kvm_x86_mpx_supported ()); }EXPORT_SYMBOL_GPL (kvm_mpx_supported);
@@ -232,7 +232,7 @@ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu, vcpu->arch.cpuid_nent = cpuid->nent;cpuid_fix_nx_cap (vcpu);kvm_apic_set_version (vcpu); - kvm_x86_ops->cpuid_update (vcpu); +kvm_x86_cpuid_update (vcpu); r =kvm_update_cpuid (vcpu); out:
@@ -255,7 +255,7 @@ int kvm_vcpu_ioctl_set_cpuid2(struct kvm_vcpu *vcpu, goto out; vcpu->arch.cpuid_nent = cpuid->nent;kvm_apic_set_version (vcpu); - kvm_x86_ops->cpuid_update (vcpu); +kvm_x86_cpuid_update (vcpu); r =kvm_update_cpuid (vcpu); out:return r;
@@ -347,10 +347,10 @@ static int __do_cpuid_func_emulated(struct kvm_cpuid_entry2 *entry,static inline void do_cpuid_7_mask (struct kvm_cpuid_entry2 *entry,int index) { -unsigned f_invpcid = kvm_x86_ops->invpcid_supported () ?F (INVPCID ) :0 ; +unsigned f_invpcid =kvm_x86_invpcid_supported () ?F (INVPCID ) :0 ;unsigned f_mpx =kvm_mpx_supported () ?F (MPX ) :0 ; -unsigned f_umip = kvm_x86_ops->umip_emulated () ?F (UMIP ) :0 ; -unsigned f_intel_pt = kvm_x86_ops->pt_supported () ?F (INTEL_PT ) :0 ; +unsigned f_umip =kvm_x86_umip_emulated () ?F (UMIP ) :0 ; +unsigned f_intel_pt =kvm_x86_pt_supported () ?F (INTEL_PT ) :0 ;unsigned f_la57;/* cpuid 7.0.ebx */
@@ -432,16 +432,16 @@ static inline int __do_cpuid_func(struct kvm_cpuid_entry2 *entry, u32 function,int r;unsigned f_nx =is_efer_nx () ?F (NX ) :0 ;#ifdef CONFIG_X86_64 -unsigned f_gbpages = (kvm_x86_ops->get_lpage_level () ==PT_PDPE_LEVEL ) +unsigned f_gbpages = (kvm_x86_get_lpage_level () ==PT_PDPE_LEVEL ) ?F (GBPAGES ) :0 ;unsigned f_lm =F (LM );#else unsigned f_gbpages =0 ;unsigned f_lm =0 ;#endif -unsigned f_rdtscp = kvm_x86_ops->rdtscp_supported () ?F (RDTSCP ) :0 ; -unsigned f_xsaves = kvm_x86_ops->xsaves_supported () ?F (XSAVES ) :0 ; -unsigned f_intel_pt = kvm_x86_ops->pt_supported () ?F (INTEL_PT ) :0 ; +unsigned f_rdtscp =kvm_x86_rdtscp_supported () ?F (RDTSCP ) :0 ; +unsigned f_xsaves =kvm_x86_xsaves_supported () ?F (XSAVES ) :0 ; +unsigned f_intel_pt =kvm_x86_pt_supported () ?F (INTEL_PT ) :0 ;/* cpuid 1.edx */ const u32 kvm_cpuid_1_edx_x86_features =
@@ -797,7 +797,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_entry2 *entry, u32 function,break ; } - kvm_x86_ops->set_supported_cpuid (function, entry); +kvm_x86_set_supported_cpuid (function, entry); r =0 ;
arch/x86/kvm/hyperv.c +3 -3
@@ -1018,7 +1018,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data, addr =gfn_to_hva (kvm, gfn);if (kvm_is_error_hva (addr))return 1 ; - kvm_x86_ops->patch_hypercall (vcpu, instructions); +kvm_x86_patch_hypercall (vcpu, instructions); ((unsigned char *)instructions)[3 ] =0xc3 ;/* ret */ if (__copy_to_user ((void __user *)addr, instructions,4 ))return 1 ;
@@ -1603,7 +1603,7 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)* hypercall generates UD from non zero cpl and real mode * per HYPER-V spec */ -if (kvm_x86_ops->get_cpl (vcpu) !=0 || !is_protmode (vcpu)) { +if (kvm_x86_get_cpl (vcpu) !=0 || !is_protmode (vcpu)) {kvm_queue_exception (vcpu,UD_VECTOR );return 1 ; }
@@ -1797,7 +1797,7 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,int i, nent =ARRAY_SIZE (cpuid_entries);if (kvm_x86_ops->nested_get_evmcs_version) - evmcs_ver = kvm_x86_ops->nested_get_evmcs_version (vcpu); + evmcs_ver =kvm_x86_nested_get_evmcs_version (vcpu);/* Skip NESTED_FEATURES if eVMCS is not supported */ if (!evmcs_ver)
arch/x86/kvm/kvm_cache_regs.h +5 -5
@@ -41,7 +41,7 @@ static inline unsigned long kvm_register_read(struct kvm_vcpu *vcpu, enum kvm_reg reg) {if (!test_bit (reg, (unsigned long *)&vcpu->arch.regs_avail)) - kvm_x86_ops->cache_reg (vcpu, reg); +kvm_x86_cache_reg (vcpu, reg);return vcpu->arch.regs[reg]; }
@@ -81,7 +81,7 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index)if (!test_bit (VCPU_EXREG_PDPTR , (unsigned long *)&vcpu->arch.regs_avail)) - kvm_x86_ops->cache_reg (vcpu, (enum kvm_reg )VCPU_EXREG_PDPTR ); +kvm_x86_cache_reg (vcpu, (enum kvm_reg )VCPU_EXREG_PDPTR );return vcpu->arch.walk_mmu->pdptrs[index]; }
@@ -90,7 +90,7 @@ static inline ulong kvm_read_cr0_bits(struct kvm_vcpu *vcpu, ulong mask) {ulong tmask = mask &KVM_POSSIBLE_CR0_GUEST_BITS ;if (tmask & vcpu->arch.cr0_guest_owned_bits) - kvm_x86_ops->decache_cr0_guest_bits (vcpu); +kvm_x86_decache_cr0_guest_bits (vcpu);return vcpu->arch.cr0 & mask; }
@@ -103,14 +103,14 @@ static inline ulong kvm_read_cr4_bits(struct kvm_vcpu *vcpu, ulong mask) {ulong tmask = mask &KVM_POSSIBLE_CR4_GUEST_BITS ;if (tmask & vcpu->arch.cr4_guest_owned_bits) - kvm_x86_ops->decache_cr4_guest_bits (vcpu); +kvm_x86_decache_cr4_guest_bits (vcpu);return vcpu->arch.cr4 & mask; }static inline ulong kvm_read_cr3 (struct kvm_vcpu *vcpu) {if (!test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) - kvm_x86_ops->decache_cr3 (vcpu); +kvm_x86_decache_cr3 (vcpu);return vcpu->arch.cr3; }
arch/x86/kvm/lapic.c +15 -15
@@ -451,7 +451,7 @@ static inline void apic_clear_irr(int vec, struct kvm_lapic *apic)if (unlikely (vcpu->arch.apicv_active)) {/* need to update RVI */ kvm_lapic_clear_vector (vec, apic->regs +APIC_IRR ); - kvm_x86_ops->hwapic_irr_update (vcpu, +kvm_x86_hwapic_irr_update (vcpu,apic_find_highest_irr (apic)); }else { apic->irr_pending = false;
@@ -476,7 +476,7 @@ static inline void apic_set_isr(int vec, struct kvm_lapic *apic)* just set SVI. */ if (unlikely (vcpu->arch.apicv_active)) - kvm_x86_ops->hwapic_isr_update (vcpu, vec); +kvm_x86_hwapic_isr_update (vcpu, vec);else { ++apic->isr_count;BUG_ON (apic->isr_count >MAX_APIC_VECTOR );
@@ -524,7 +524,7 @@ static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)* and must be left alone. */ if (unlikely (vcpu->arch.apicv_active)) - kvm_x86_ops->hwapic_isr_update (vcpu, +kvm_x86_hwapic_isr_update (vcpu,apic_find_highest_isr (apic));else { --apic->isr_count;
@@ -667,7 +667,7 @@ static int apic_has_interrupt_for_ppr(struct kvm_lapic *apic, u32 ppr) {int highest_irr;if (apic->vcpu->arch.apicv_active) - highest_irr = kvm_x86_ops->sync_pir_to_irr (apic->vcpu); + highest_irr =kvm_x86_sync_pir_to_irr (apic->vcpu);else highest_irr =apic_find_highest_irr (apic);if (highest_irr ==-1 || (highest_irr &0xF0 ) <= ppr)
@@ -1057,7 +1057,7 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode, }if (vcpu->arch.apicv_active) - kvm_x86_ops->deliver_posted_interrupt (vcpu, vector); +kvm_x86_deliver_posted_interrupt (vcpu, vector);else {kvm_lapic_set_irr (vector, apic);
@@ -1701,7 +1701,7 @@ static void cancel_hv_timer(struct kvm_lapic *apic) {WARN_ON (preemptible ());WARN_ON (!apic->lapic_timer.hv_timer_in_use); - kvm_x86_ops->cancel_hv_timer (apic->vcpu); +kvm_x86_cancel_hv_timer (apic->vcpu); apic->lapic_timer.hv_timer_in_use = false; }
@@ -1718,7 +1718,7 @@ static bool start_hv_timer(struct kvm_lapic *apic)if (!ktimer->tscdeadline)return false; -if (kvm_x86_ops->set_hv_timer (vcpu, ktimer->tscdeadline, &expired)) +if (kvm_x86_set_hv_timer (vcpu, ktimer->tscdeadline, &expired))return false; ktimer->hv_timer_in_use = true;
@@ -2138,7 +2138,7 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)kvm_apic_set_x2apic_id (apic, vcpu->vcpu_id);if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE |X2APIC_ENABLE )) - kvm_x86_ops->set_virtual_apic_mode (vcpu); +kvm_x86_set_virtual_apic_mode (vcpu); apic->base_address = apic->vcpu->arch.apic_base &MSR_IA32_APICBASE_BASE ;
@@ -2201,9 +2201,9 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) vcpu->arch.pv_eoi.msr_val =0 ;apic_update_ppr (apic);if (vcpu->arch.apicv_active) { - kvm_x86_ops->apicv_post_state_restore (vcpu); - kvm_x86_ops->hwapic_irr_update (vcpu,-1 ); - kvm_x86_ops->hwapic_isr_update (vcpu,-1 ); +kvm_x86_apicv_post_state_restore (vcpu); +kvm_x86_hwapic_irr_update (vcpu,-1 ); +kvm_x86_hwapic_isr_update (vcpu,-1 ); } vcpu->arch.apic_arb_prio =0 ;
@@ -2454,10 +2454,10 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s) 1 : count_vectors(apic->regs + APIC_ISR); apic->highest_isr_cache =-1 ;if (vcpu->arch.apicv_active) { - kvm_x86_ops->apicv_post_state_restore (vcpu); - kvm_x86_ops->hwapic_irr_update (vcpu, +kvm_x86_apicv_post_state_restore (vcpu); +kvm_x86_hwapic_irr_update (vcpu,apic_find_highest_irr (apic)); - kvm_x86_ops->hwapic_isr_update (vcpu, +kvm_x86_hwapic_isr_update (vcpu,apic_find_highest_isr (apic)); }kvm_make_request (KVM_REQ_EVENT , vcpu);
@@ -2709,7 +2709,7 @@ void kvm_apic_accept_events(struct kvm_vcpu *vcpu)* KVM_MP_STATE_INIT_RECEIVED state), just eat SIPIs * and leave the INIT pending. */ -if (is_smm (vcpu) || kvm_x86_ops->apic_init_signal_blocked (vcpu)) { +if (is_smm (vcpu) ||kvm_x86_apic_init_signal_blocked (vcpu)) {WARN_ON_ONCE (vcpu->arch.mp_state ==KVM_MP_STATE_INIT_RECEIVED );if (test_bit (KVM_APIC_SIPI , &apic->pending_events))clear_bit (KVM_APIC_SIPI , &apic->pending_events);
arch/x86/kvm/mmu.c +13 -13
@@ -292,7 +292,7 @@ static void kvm_flush_remote_tlbs_with_range(struct kvm *kvm,int ret = -ENOTSUPP ;if (range && kvm_x86_ops->tlb_remote_flush_with_range) - ret = kvm_x86_ops->tlb_remote_flush_with_range (kvm, range); + ret =kvm_x86_tlb_remote_flush_with_range (kvm, range);if (ret)kvm_flush_remote_tlbs (kvm);
@@ -1289,7 +1289,7 @@ static int mapping_level(struct kvm_vcpu *vcpu, gfn_t large_gfn,if (host_level ==PT_PAGE_TABLE_LEVEL )return host_level; - max_level =min (kvm_x86_ops->get_lpage_level (), host_level); + max_level =min (kvm_x86_get_lpage_level (), host_level);for (level =PT_DIRECTORY_LEVEL ; level <= max_level; ++level)if (__mmu_gfn_lpage_is_disallowed (large_gfn, level, slot))
@@ -1748,7 +1748,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, gfn_t gfn_offset, unsigned long mask) {if (kvm_x86_ops->enable_log_dirty_pt_masked) - kvm_x86_ops->enable_log_dirty_pt_masked (kvm, slot, gfn_offset, +kvm_x86_enable_log_dirty_pt_masked (kvm, slot, gfn_offset, mask);else kvm_mmu_write_protect_pt_masked (kvm, slot, gfn_offset, mask);
@@ -1764,7 +1764,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,int kvm_arch_write_log_dirty (struct kvm_vcpu *vcpu) {if (kvm_x86_ops->write_log_dirty) -return kvm_x86_ops->write_log_dirty (vcpu); +return kvm_x86_write_log_dirty (vcpu);return 0 ; }
@@ -3024,7 +3024,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,if (level >PT_PAGE_TABLE_LEVEL ) spte |=PT_PAGE_SIZE_MASK ;if (tdp_enabled) - spte |= kvm_x86_ops->get_mt_mask (vcpu, gfn, + spte |=kvm_x86_get_mt_mask (vcpu, gfn,kvm_is_mmio_pfn (pfn));if (host_writable)
@@ -4295,7 +4295,7 @@ static bool fast_cr3_switch(struct kvm_vcpu *vcpu, gpa_t new_cr3,kvm_make_request (KVM_REQ_LOAD_CR3 , vcpu);if (!skip_tlb_flush) {kvm_make_request (KVM_REQ_MMU_SYNC , vcpu); - kvm_x86_ops->tlb_flush (vcpu, true); +kvm_x86_tlb_flush (vcpu, true); }/*
@@ -4890,7 +4890,7 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, bool base_only)union kvm_mmu_role role =kvm_calc_mmu_role_common (vcpu, base_only); role.base.ad_disabled = (shadow_accessed_mask ==0 ); - role.base.level = kvm_x86_ops->get_tdp_level (vcpu); + role.base.level =kvm_x86_get_tdp_level (vcpu); role.base.direct = true; role.base.gpte_is_8_bytes = true;
@@ -4912,7 +4912,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu) context->sync_page = nonpaging_sync_page; context->invlpg = nonpaging_invlpg; context->update_pte = nonpaging_update_pte; - context->shadow_root_level = kvm_x86_ops->get_tdp_level (vcpu); + context->shadow_root_level =kvm_x86_get_tdp_level (vcpu); context->direct_map = true; context->set_cr3 = kvm_x86_ops->set_tdp_cr3; context->get_cr3 = get_cr3;
@@ -5169,7 +5169,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)if (r) goto out;kvm_mmu_load_cr3 (vcpu); - kvm_x86_ops->tlb_flush (vcpu, true); +kvm_x86_tlb_flush (vcpu, true); out:return r; }
@@ -5482,7 +5482,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code,* guest, with the exception of AMD Erratum 1096 which is unrecoverable. */ if (unlikely (insn && !insn_len)) { -if (!kvm_x86_ops->need_emulation_on_page_fault (vcpu)) +if (!kvm_x86_need_emulation_on_page_fault (vcpu))return 1 ; }
@@ -5517,7 +5517,7 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)if (VALID_PAGE (mmu->prev_roots[i].hpa)) mmu->invlpg (vcpu, gva, mmu->prev_roots[i].hpa); - kvm_x86_ops->tlb_flush_gva (vcpu, gva); +kvm_x86_tlb_flush_gva (vcpu, gva); ++vcpu->stat.invlpg; }EXPORT_SYMBOL_GPL (kvm_mmu_invlpg);
@@ -5542,7 +5542,7 @@ void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid) }if (tlb_flush) - kvm_x86_ops->tlb_flush_gva (vcpu, gva); +kvm_x86_tlb_flush_gva (vcpu, gva); ++vcpu->stat.invlpg;
@@ -5659,7 +5659,7 @@ static int alloc_mmu_pages(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu)* SVM's 32-bit NPT support, TDP paging doesn't use PAE paging and can * skip allocating the PDP table. */ -if (tdp_enabled && kvm_x86_ops->get_tdp_level (vcpu) >PT32E_ROOT_LEVEL ) +if (tdp_enabled &&kvm_x86_get_tdp_level (vcpu) >PT32E_ROOT_LEVEL )return 0 ; page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_DMA32);
arch/x86/kvm/mmu.h +2 -2
@@ -157,8 +157,8 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned pte_access, unsigned pte_pkey, unsigned pfec) { - int cpl = kvm_x86_ops->get_cpl(vcpu); -unsigned long rflags = kvm_x86_ops->get_rflags (vcpu); + int cpl = kvm_x86_get_cpl(vcpu); +unsigned long rflags =kvm_x86_get_rflags (vcpu);/* * If CPL < 3, SMAP prevention are disabled if EFLAGS.AC = 1.
arch/x86/kvm/pmu.c +12 -12
@@ -183,7 +183,7 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) ARCH_PERFMON_EVENTSEL_CMASK | HSW_IN_TX | HSW_IN_TX_CHECKPOINTED))) { - config = kvm_x86_ops->pmu_ops->find_arch_event (pmc_to_pmu (pmc), + config =kvm_x86_pmu_find_arch_event (pmc_to_pmu (pmc), event_select, unit_mask);if (config !=PERF_COUNT_HW_MAX )
@@ -225,7 +225,7 @@ void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx) }pmc_reprogram_counter (pmc,PERF_TYPE_HARDWARE , - kvm_x86_ops->pmu_ops->find_fixed_event (idx), +kvm_x86_pmu_find_fixed_event (idx), !(en_field &0x2 ),/* exclude user */ !(en_field &0x1 ),/* exclude kernel */ pmi, false, false);
@@ -234,7 +234,7 @@ EXPORT_SYMBOL_GPL(reprogram_fixed_counter);void reprogram_counter (struct kvm_pmu *pmu,int pmc_idx) { -struct kvm_pmc *pmc = kvm_x86_ops->pmu_ops->pmc_idx_to_pmc (pmu, pmc_idx); +struct kvm_pmc *pmc =kvm_x86_pmu_pmc_idx_to_pmc (pmu, pmc_idx);if (!pmc)return ;
@@ -259,7 +259,7 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) bitmask = pmu->reprogram_pmi;for_each_set_bit (bit, (unsigned long *)&bitmask,X86_PMC_IDX_MAX ) { -struct kvm_pmc *pmc = kvm_x86_ops->pmu_ops->pmc_idx_to_pmc (pmu, bit); +struct kvm_pmc *pmc =kvm_x86_pmu_pmc_idx_to_pmc (pmu, bit);if (unlikely (!pmc || !pmc->perf_event)) {clear_bit (bit, (unsigned long *)&pmu->reprogram_pmi);
@@ -273,7 +273,7 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu)/* check if idx is a valid index to access PMU */ int kvm_pmu_is_valid_msr_idx (struct kvm_vcpu *vcpu,unsigned idx) { -return kvm_x86_ops->pmu_ops->is_valid_msr_idx (vcpu, idx); +return kvm_x86_pmu_is_valid_msr_idx (vcpu, idx); }bool is_vmware_backdoor_pmc (u32 pmc_idx)
@@ -323,7 +323,7 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)if (is_vmware_backdoor_pmc (idx))return kvm_pmu_rdpmc_vmware (vcpu, idx, data); - pmc = kvm_x86_ops->pmu_ops->msr_idx_to_pmc (vcpu, idx, &mask); + pmc =kvm_x86_pmu_msr_idx_to_pmc (vcpu, idx, &mask);if (!pmc)return 1 ;
@@ -339,17 +339,17 @@ void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu)bool kvm_pmu_is_valid_msr (struct kvm_vcpu *vcpu,u32 msr) { -return kvm_x86_ops->pmu_ops->is_valid_msr (vcpu, msr); +return kvm_x86_pmu_is_valid_msr (vcpu, msr); }int kvm_pmu_get_msr (struct kvm_vcpu *vcpu,u32 msr,u64 *data) { -return kvm_x86_ops->pmu_ops->get_msr (vcpu, msr, data); +return kvm_x86_pmu_get_msr (vcpu, msr, data); }int kvm_pmu_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) { -return kvm_x86_ops->pmu_ops->set_msr (vcpu, msr_info); +return kvm_x86_pmu_set_msr (vcpu, msr_info); }/* refresh PMU settings. This function generally is called when underlying
@@ -358,7 +358,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)*/ void kvm_pmu_refresh (struct kvm_vcpu *vcpu) { - kvm_x86_ops->pmu_ops->refresh (vcpu); +kvm_x86_pmu_refresh (vcpu); }void kvm_pmu_reset (struct kvm_vcpu *vcpu)
@@ -366,7 +366,7 @@ void kvm_pmu_reset(struct kvm_vcpu *vcpu)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);irq_work_sync (&pmu->irq_work); - kvm_x86_ops->pmu_ops->reset (vcpu); +kvm_x86_pmu_reset (vcpu); }void kvm_pmu_init (struct kvm_vcpu *vcpu)
@@ -374,7 +374,7 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);memset (pmu,0 ,sizeof (*pmu)); - kvm_x86_ops->pmu_ops->init (vcpu); +kvm_x86_pmu_init (vcpu);init_irq_work (&pmu->irq_work, kvm_pmi_trigger_fn);kvm_pmu_refresh (vcpu); }
arch/x86/kvm/pmu.h +18 -1
@@ -19,6 +19,23 @@ struct kvm_event_hw_type_mapping {unsigned event_type; }; +extern unsigned kvm_x86_pmu_find_arch_event (struct kvm_pmu *pmu, +u8 event_select,u8 unit_mask); +extern unsigned kvm_x86_pmu_find_fixed_event (int idx); +extern bool kvm_x86_pmu_pmc_is_enabled (struct kvm_pmc *pmc); +extern struct kvm_pmc *kvm_x86_pmu_pmc_idx_to_pmc (struct kvm_pmu *pmu, +int pmc_idx); +extern struct kvm_pmc *kvm_x86_pmu_msr_idx_to_pmc (struct kvm_vcpu *vcpu, +unsigned idx,u64 *mask); +extern int kvm_x86_pmu_is_valid_msr_idx (struct kvm_vcpu *vcpu,unsigned idx); +extern bool kvm_x86_pmu_is_valid_msr (struct kvm_vcpu *vcpu,u32 msr); +extern int kvm_x86_pmu_get_msr (struct kvm_vcpu *vcpu,u32 msr,u64 *data); +extern int kvm_x86_pmu_set_msr (struct kvm_vcpu *vcpu, +struct msr_data *msr_info); +extern void kvm_x86_pmu_refresh (struct kvm_vcpu *vcpu); +extern void kvm_x86_pmu_init (struct kvm_vcpu *vcpu); +extern void kvm_x86_pmu_reset (struct kvm_vcpu *vcpu); +struct kvm_pmu_ops {unsigned (*find_arch_event)(struct kvm_pmu *pmu,u8 event_select,u8 unit_mask);
@@ -76,7 +93,7 @@ static inline bool pmc_is_fixed(struct kvm_pmc *pmc)static inline bool pmc_is_enabled (struct kvm_pmc *pmc) { -return kvm_x86_ops->pmu_ops->pmc_is_enabled (pmc); +return kvm_x86_pmu_pmc_is_enabled (pmc); }/* returns general purpose PMC with the specified MSR. Note that it can be
arch/x86/kvm/pmu_amd.c +26 -26
126 lines not on this page · raw message
arch/x86/kvm/svm.c +393 -271
1,392 lines not on this page · raw message
arch/x86/kvm/trace.h +2 -2
@@ -240,7 +240,7 @@ TRACE_EVENT(kvm_exit, __entry->guest_rip =kvm_rip_read (vcpu); __entry->isa = isa; __entry->vcpu_id = vcpu->vcpu_id; - kvm_x86_ops->get_exit_info (vcpu, &__entry->info1, +kvm_x86_get_exit_info (vcpu, &__entry->info1, &__entry->info2); ),
@@ -744,7 +744,7 @@ TRACE_EVENT(kvm_emulate_insn, ),TP_fast_assign ( - __entry->csbase = kvm_x86_ops->get_segment_base(vcpu, VCPU_SREG_CS); + __entry->csbase = kvm_x86_get_segment_base(vcpu, VCPU_SREG_CS); __entry->len = vcpu->arch.emulate_ctxt.fetch.ptr - vcpu->arch.emulate_ctxt.fetch.data; __entry->rip = vcpu->arch.emulate_ctxt._eip - __entry->len;
arch/x86/kvm/vmx/nested.c +42 -42
277 lines not on this page · raw message
arch/x86/kvm/vmx/pmu_intel.c +27 -28
137 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +423 -316
347 lines not on this page · raw message
arch/x86/kvm/x86.c +154 -154
308 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
PATCH 3/13 kvm: monolithic: fixup x86-32 build
In reply to Andrea Arcangelikvm_x86_set_hv_timer and kvm_x86_cancel_hv_timer needs to be defined to succeed the 32bit kernel build, but they can't be called.
--- arch/x86/kvm/vmx/vmx.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/x86/kvm/vmx/vmx.c +11 -0
@@ -7195,6 +7195,17 @@ void kvm_x86_cancel_hv_timer(struct kvm_vcpu *vcpu) {to_vmx (vcpu)->hv_deadline_tsc =-1 ; } +#else +int kvm_x86_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired) +{ +BUG (); +} + +void kvm_x86_cancel_hv_timer (struct kvm_vcpu *vcpu) +{ +BUG (); +}#endif void kvm_x86_sched_in (struct kvm_vcpu *vcpu,int cpu)
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 21 lines · Andrea Arcangeli
kvm_x86_set_hv_timerandkvm_x86_cancel_hv_timerneeds to be defined to succeed the 32bit kernel build, but they can't be called.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/x86/kvm/vmx/vmx.c +4
-0@@ -7195,6 +7195,17 @@ void kvm_x86_cancel_hv_timer(struct kvm_vcpu *vcpu) {to_vmx (vcpu)->hv_deadline_tsc =-1 ; } +#else +int kvm_x86_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired) +{
arch/x86/kvm/vmx/vmx.c void kvm_x86_cancel_hv_timer(struct kvm_vcpu *vcpu) · line 7211
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +7
-0+ BUG(); +} + +void kvm_x86_cancel_hv_timer (struct kvm_vcpu *vcpu) +{ +BUG (); +}#endif void kvm_x86_sched_in (struct kvm_vcpu *vcpu,int cpu)
I'll check for how long this has been broken. It may be the proof that we can actually drop 32-bit KVM support.
Paolo
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 30 lines · Paolo Bonzini
Quoted 29 lines · Andrea Arcangeli
kvm_x86_set_hv_timerandkvm_x86_cancel_hv_timerneeds to be defined to succeed the 32bit kernel build, but they can't be called.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/x86/kvm/vmx/vmx.c +11
-0@@ -7195,6 +7195,17 @@ void kvm_x86_cancel_hv_timer(struct kvm_vcpu *vcpu) {to_vmx (vcpu)->hv_deadline_tsc =-1 ; } +#else +int kvm_x86_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired) +{ +BUG (); +} + +void kvm_x86_cancel_hv_timer (struct kvm_vcpu *vcpu) +{ +BUG (); +}#endif
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c void kvm_x86_cancel_hv_timer(struct kvm_vcpu *vcpu) · line 7211
Quoted 2 lines
arch/x86/kvm/vmx/vmx.c +0
-0void kvm_x86_sched_in (struct kvm_vcpu *vcpu,int cpu)I'll check for how long this has been broken. It may be the proof that we can actually drop 32-bit KVM support.
Ah no, I was confused because this series is not bisectable (in addition to doing two things at a same time, namely the monolithic kvm.ko and the retpoline eliminations).
I have picked up the patches that are independent of the monolithic kvm.ko work or can be considered bugfixes.
For the rest, please do this before posting again:
- ensure that everything is bisectable
- look into how to remove the modpost warnings. A simple (though somewhat ugly) way is to keep a kvm.ko module that includes common virt/kvm/ code as well as, for x86 only, page_track.o. A few functions, such as kvm_mmu_gfn_disallow_lpage and kvm_mmu_gfn_allow_lpage, would have to be moved into mmu.h, but that's not a big deal.
- provide at least some examples of replacing the NULL kvm_x86_ops checks with error codes in the function (or just early "return"s). I can help with the others, but remember that for the patch to be merged, kvm_x86_ops must be removed completely.
Thanks,
Paolo
Re: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 3 lines · Paolo Bonzini
For the rest, please do this before posting again:
- ensure that everything is bisectable
x86-64 is already bisectable.
All other archs bisectable I didn't check them all anyway.
Even 4/13 is suboptimal and needs to be re-done later in more optimal way. I prefer all logic changes to happen at later steps so one can at least bisect to something that functionally works like before. And 4/13 also would need to be merged in the huge patch if one wants to guarantee bisectability on all CPUs, but it'll just be hidden there in the huge patch.
Obviously I can squash both 3/13 and 4/13 into 2/13 but I don't feel like doing the right thing by squashing them just to increase bisectability.
Quoted 5 lines
- look into how to remove the modpost warnings. A simple (though somewhat ugly) way is to keep a kvm.ko module that includes common
virt/kvm/code as well as, for x86 only, page_track.o. A few functions, such askvm_mmu_gfn_disallow_lpageandkvm_mmu_gfn_allow_lpage, would have to be moved intommu.h, but that's not a big deal.
I think we should:
1) whitelist to shut off the warnings on demand
2) verify that if two modules are registering the same export symbol the second one fails to load and the module code is robust about that, this hopefully should already be the case
Provided verification of 2), the whitelist is more efficient than losing 4k of ram in all KVM hypervisors out there.
Quoted 4 lines
- provide at least some examples of replacing the NULL
kvm_x86_opschecks with error codes in the function (or just early "return"s). I can help with the others, but remember that for the patch to be merged,kvm_x86_opsmust be removed completely.
Even if kvm_x86_ops wouldn't be guaranteed to go away, this would already provide all the performance benefit to the KVM users, so I wouldn't see a reason not to apply it even if kvm_x86_ops cannot go away. Said that it will go away and there's no concern about it. It's just that the patchset seems large enough already and it rejects heavily already at every port. I simply stopped at the first self contained step that provides all performance benefits.
If I go ahead and remove kvm_x86_ops how do I know it won't reject heavily the next day I rebase and I've to redo it all from scratch? If you explain me how you're going to guarantee that I won't have to do that work more than once I'd be happy to go ahead.
Thanks,
Andrea
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 11 lines · Andrea Arcangeli
x86-64 is already bisectable.
All other archs bisectable I didn't check them all anyway.
Even 4/13 is suboptimal and needs to be re-done later in more optimal way. I prefer all logic changes to happen at later steps so one can at least bisect to something that functionally works like before. And 4/13 also would need to be merged in the huge patch if one wants to guarantee bisectability on all CPUs, but it'll just be hidden there in the huge patch.
Quoted 3 lines
Obviously I can squash both 3/13 and 4/13 into 2/13 but I don't feel like doing the right thing by squashing them just to increase bisectability.
You can reorder patches so that kvm_x86_ops assignments never happen. That way, 4/13 for example would be moved to the very beginning.
Quoted 9 lines
Quoted 5 lines
- look into how to remove the modpost warnings. A simple (though somewhat ugly) way is to keep a kvm.ko module that includes common
virt/kvm/code as well as, for x86 only, page_track.o. A few functions, such askvm_mmu_gfn_disallow_lpageandkvm_mmu_gfn_allow_lpage, would have to be moved intommu.h, but that's not a big deal.I think we should:
1) whitelist to shut off the warnings on demand
Do you mean adding a whitelist to modpost? That would work, though I am not sure if the module maintainer (Jessica Yu) would accept that.
Quoted 6 lines
2) verify that if two modules are registering the same export symbol the second one fails to load and the module code is robust about that, this hopefully should already be the case
Provided verification of 2), the whitelist is more efficient than losing 4k of ram in all KVM hypervisors out there.
I agree.
Quoted 9 lines
Quoted 4 lines
- provide at least some examples of replacing the NULL
kvm_x86_opschecks with error codes in the function (or just early "return"s). I can help with the others, but remember that for the patch to be merged,kvm_x86_opsmust be removed completely.Even if
kvm_x86_opswouldn't be guaranteed to go away, this would already provide all the performance benefit to the KVM users, so I wouldn't see a reason not to apply it even ifkvm_x86_opscannot go away.
The answer is maintainability. My suggestion is that we start looking into removing all assignments and tests of kvm_x86_ops, one step at a time. Until this is done, unfortunately we won't be able to reap the performance benefit. But the advantage is that this can be done in many separate submissions; it doesn't have to be one huge patch.
Once this is done, removing kvm_x86_ops is trivial in the end. It's okay if the intermediate step has minimal performance regressions, we know what it will look like. I have to order patches with maintenance first and performance second, if possible.
By the way, we are already planning to make some module parameters per-VM instead of global, so this refactoring would also help that effort.
Quoted 4 lines
Said that it will go away and there's no concern about it. It's just that the patchset seems large enough already and it rejects heavily already at every port. I simply stopped at the first self contained step that provides all performance benefits.
That is good enough to prove the feasibility of the idea, so I agree that was a good plan.
Paolo
Quoted 4 lines
If I go ahead and remove
kvm_x86_opshow do I know it won't reject heavily the next day I rebase and I've to redo it all from scratch? If you explain me how you're going to guarantee that I won't have to do that work more than once I'd be happy to go ahead.
Re: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 2 lines · Paolo Bonzini
You can reorder patches so that
kvm_x86_opsassignments never happen. That way, 4/13 for example would be moved to the very beginning.
Ok 3/13 and 4/14 can both go before 2/13, I reordered them fine, the end result at least is the same and the intermediate result should improve. Hopefully this is the best solution for those two outliers.
Once this is committed I expect Sean to take over 4/14 with a more optimal version to get rid of that branch like he proposed initially.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
- look into how to remove the modpost warnings. A simple (though somewhat ugly) way is to keep a kvm.ko module that includes common
virt/kvm/code as well as, for x86 only, page_track.o. A few functions, such askvm_mmu_gfn_disallow_lpageandkvm_mmu_gfn_allow_lpage, would have to be moved intommu.h, but that's not a big deal.
Quoted 6 lines
Quoted 3 lines
I think we should:
1) whitelist to shut off the warnings on demand
Do you mean adding a whitelist to modpost? That would work, though I am not sure if the module maintainer (Jessica Yu) would accept that.
Yes that's exactly what I meant.
Quoted 8 lines
Quoted 6 lines
2) verify that if two modules are registering the same export symbol the second one fails to load and the module code is robust about that, this hopefully should already be the case
Provided verification of 2), the whitelist is more efficient than losing 4k of ram in all KVM hypervisors out there.
I agree.
Ok, so I enlarged the CC list accordingly to check how the whitelist can be done in modpost.
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
- provide at least some examples of replacing the NULL
kvm_x86_opschecks with error codes in the function (or just early "return"s). I can help with the others, but remember that for the patch to be merged,kvm_x86_opsmust be removed completely.Even if
kvm_x86_opswouldn't be guaranteed to go away, this would already provide all the performance benefit to the KVM users, so I wouldn't see a reason not to apply it even ifkvm_x86_opscannot go away.
Quoted 4 lines
The answer is maintainability. My suggestion is that we start looking into removing all assignments and tests of
kvm_x86_ops, one step at a time. Until this is done, unfortunately we won't be able to reap the performance benefit. But the advantage is that this can be done in many
There's not much performance benefit left from the removal kvm_x86_ops. It'll only remove a few branches at best (and only if we don't have to replace the branches on the pointer check with other branches on a static variable to disambiguate the different cases).
Quoted 6 lines
separate submissions; it doesn't have to be one huge patch.
Once this is done, removing
kvm_x86_opsis trivial in the end. It's okay if the intermediate step has minimal performance regressions, we know what it will look like. I have to order patches with maintenance first and performance second, if possible.
The removal of kvm_x86_ops is just a badly needed code cleanup and of course I agree it must happen sooner than later. I'm just trying to avoid running into rejects on those further commit cleanups too.
Quoted 10 lines
By the way, we are already planning to make some module parameters per-VM instead of global, so this refactoring would also help that effort.
Quoted 4 lines
Said that it will go away and there's no concern about it. It's just that the patchset seems large enough already and it rejects heavily already at every port. I simply stopped at the first self contained step that provides all performance benefits.
That is good enough to prove the feasibility of the idea, so I agree that was a good plan.
All right, so I'm not exactly sure what's the plan and if it's ok to do it over time or if I should go ahead doing all logic changes while the big patch remains out of tree.
If you apply it and reorder 4/13 and 3/13 before 2/13 in a rebase like I did locally, it should already be good starting point in my view and the modpost also can be fixed over time too, the warnings appears harmless so far.
Thanks,
Andrea
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 8 lines · Andrea Arcangeli
Quoted 6 lines
Quoted 3 lines
I think we should:
1) whitelist to shut off the warnings on demand
Do you mean adding a whitelist to modpost? That would work, though I am not sure if the module maintainer (Jessica Yu) would accept that.
Yes that's exactly what I meant.
Ok, thanks. Jessica, the issue here is that we have two (mutually exclusive) modules providing the same interface to a third module.
Andrea will check that, when the same symbol is exported by two modules, the second-loaded module correctly fails insmod. If that is okay, we will also need modpost not to warn for these symbols in sym_add_exported.
Quoted 7 lines
Quoted 4 lines
The answer is maintainability. My suggestion is that we start looking into removing all assignments and tests of
kvm_x86_ops, one step at a time. Until this is done, unfortunately we won't be able to reap the performance benefit. But the advantage is that this can be done in manyThere's not much performance benefit left from the removal
kvm_x86_ops.
Indeed; what I mean is that until then we will have to keep the retpolines. Not removing kvm_x86_ops leaves an unsustainable mess in terms of maintainability, therefore we will need to first refactor the code. Once the refactoring is over, kvm_x86_ops can be dropped easily, just like kvm_pmu_ops in this version of the series.
The good thing is that the modpost discussion can proceed in parallel.
Quoted 10 lines
The removal of
kvm_x86_opsis just a badly needed code cleanup and of course I agree it must happen sooner than later. I'm just trying to avoid running into rejects on those further commit cleanups too.Quoted 2 lines
That is good enough to prove the feasibility of the idea, so I agree that was a good plan.
All right, so I'm not exactly sure what's the plan and if it's ok to do it over time or if I should go ahead doing all logic changes while the big patch remains out of tree.
Yes, the changes to remove tests and assignments to kvm_x86_ops must happen first. I understand that the big patch is a conflict magnet, but once all the refactoring is done it will be very easy to review and it will get in quickly.
Paolo
Re: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
+++ Paolo Bonzini [05/11/19 16:10 +0100]:
Quoted 8 lines · Andrea Arcangeli
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
I think we should:
1) whitelist to shut off the warnings on demand
Do you mean adding a whitelist to modpost? That would work, though I am not sure if the module maintainer (Jessica Yu) would accept that.
Quoted 7 lines
Quoted 1 line
Yes that's exactly what I meant.
Ok, thanks. Jessica, the issue here is that we have two (mutually exclusive) modules providing the same interface to a third module.
Andrea will check that, when the same symbol is exported by two modules, the second-loaded module correctly fails insmod.
Hi Paolo, thanks for getting me up to speed.
The module loader already rejects loading a module with duplicate exported symbols.
Quoted 2 lines
If that is okay, we will also need modpost not to warn for these symbols in
sym_add_exported.
I think it's certainly doable in modpost, for example we could pass a list of whitelisted symbols and have modpost read them in and not warn if it encounters the whitelisted symbols more than once. Modpost will also have to be modified to accomodate duplicate symbols. I'm not sure how ugly this would be without seeing the actual patch. And I am not sure what Masahiro (who takes care of all things kbuild-related) thinks of this idea. But before implementing all this, is there absolutely no way around having the duplicated exported symbols? (e.g., could the modules be configured/built in a mutally exclusive way? I'm lacking the context from the rest of the thread, so not sure which are the problematic modules.)
Thanks,
Jessica
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 7 lines · Jessica Yu
And I am
not sure what Masahiro (who takes care of all things kbuild-related) thinks of this idea. But before implementing all this, is there absolutely no way around having the duplicated exported symbols? (e.g., could the modules be configured/built in a mutally exclusive way? I'm lacking the context from the rest of the thread, so not sure which are the problematic modules.)
The problematic modules are kvm_intel and kvm_amd, so we cannot build them in a mutually exclusive way (but we know it won't make sense to load both). We will have to build only one of them when built into vmlinux, but the module case must support building both.
Currently we put the common symbols in kvm.ko, and kvm.ko acts as a kind of "library" for kvm_intel.ko and kvm_amd.ko. The problem is that kvm_intel.ko and kvm_amd.ko currently pass a large array of function pointers to kvm.ko, and Andrea measured a substantial performance penalty from retpolines when kvm.ko calls back through those pointers.
Therefore he would like to remove kvm.ko, and that would result in symbols exported from two modules.
I suppose we could use code patching mechanism to avoid the retpolines.
Andrea, what do you think about that? That would have the advantage
that we won't have to remove kvm_x86_ops. :)
Thanks,
Paolo
Re: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 3 lines · Paolo Bonzini
I suppose we could use code patching mechanism to avoid the retpolines.
Andrea, what do you think about that? That would have the advantage
that we won't have to removekvm_x86_ops. :)
page 17 covers pvops:
https://people.redhat.com/~aarcange/slides/2019-KVM-monolithic.pdf
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 8 lines · Andrea Arcangeli
Quoted 3 lines · Paolo Bonzini
I suppose we could use code patching mechanism to avoid the retpolines.
Andrea, what do you think about that? That would have the advantage
that we won't have to removekvm_x86_ops. :)page 17 covers pvops:
https://people.redhat.com/~aarcange/slides/2019-KVM-monolithic.pdf
You can patch call instructions directly using text_poke when kvm_intel.ko or kvm_amd.ko, I'm not sure why that would be worse for TLB or RAM usage. The hard part is recording the location of the call sites using some pushsection/popsection magic.
Paolo
Re: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 2 lines · Paolo Bonzini
kvm_intel.ko or kvm_amd.ko, I'm not sure why that would be worse for TLB or RAM usage. The hard part is recording the location of the call sites
Let's ignore the different code complexity of supporting self modifying code: kvm.ko and kvm-*.ko will be located in different pages, hence it'll waste 1 iTLB for every vmexit and 2k of RAM in average. The L1 icache also will be wasted. It'll simply run slower.
Now about the code complexity, it is even higher than pvops:
KVM pvops
========= =============
1) Changes daily Never change
2) Patched at runtime Patched only at boot time early on
during module load
and multiple times
at every load of kvm-*.ko
3) The patching points to All patch destinations are linked into
code in kernel modules the kernel
Why exactly should we go through such a complication when it runs slower in the end and it's much more complex to implement and maintain and in fact even more complex than pvops already is?
Runtime patching the indirect call like pvops do is strictly required when you are forced to resolve the linking at runtime. The alternative would be to ship two different Linux kernels for PV and bare metal. Maintaining a whole new kernel rpm and having to install a different rpm depending on the hypervisor/bare metal is troublesome so pvops is worth it.
With kvm-amd and kvm-intel we can avoid the whole runtime patching of the call sites as already proven by KVM monolithic patchset, and it'll run faster in the CPU and it'll save RAM, so I'm not exactly sure how anybody could prefer runtime patching here when the only benefit is a few mbytes of disk space saved on disk.
Furthermore by linking the thing statically we'll also enable LTO and other gcc features which would never be possible with those indirect calls.
Thanks,
Andrea
Paolo Bonzini
maintainerRe: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
Quoted 8 lines · Andrea Arcangeli
Quoted 2 lines · Paolo Bonzini
kvm_intel.ko or kvm_amd.ko, I'm not sure why that would be worse for TLB or RAM usage. The hard part is recording the location of the call sites
Let's ignore the different code complexity of supporting self modifying code: kvm.ko and kvm-*.ko will be located in different pages, hence it'll waste 1 iTLB for every vmexit and 2k of RAM in average.
This is unlikely to make a difference, since kvm.o and kvm-intel.o are overall about 700 KiB in size. You do lose some inlining opportunities
with LTO, but without LTO the L1 cache benefits are debatable too. The
real loss is in the complexity, I agree with you about that.
Quoted 11 lines
Now about the code complexity, it is even higher than pvops:
KVM pvops ========= =============1) Changes daily Never change
2) Patched at runtime Patched only at boot time early on
during module load
and multiple times
at every load of kvm-*.ko
Quoted 6 lines
3) The patching points to All patch destinations are linked into
code in kernel modules the kernelWhy exactly should we go through such a complication when it runs slower in the end and it's much more complex to implement and maintain and in fact even more complex than pvops already is?
For completeness, one advantage of patching would be to keep support for built-in Intel+AMD. The modpost patch should be pretty small, and since Jessica seemed quite open to it let's do that.
Thanks,
Paolo
Quoted 7 lines
Furthermore by linking the thing statically we'll also enable LTO and other gcc features which would never be possible with those indirect calls.
Thanks,
Andrea
Re: [PATCH 03/13] kvm: monolithic: fixup x86-32 build
In reply to Paolo Bonzini(+CC: Lucas De Marchi, the kmod maintainer)
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Jessica Yu
And I am
not sure what Masahiro (who takes care of all things kbuild-related) thinks of this idea. But before implementing all this, is there absolutely no way around having the duplicated exported symbols? (e.g., could the modules be configured/built in a mutally exclusive way? I'm lacking the context from the rest of the thread, so not sure which are the problematic modules.)
I do not think having a white-list in the modpost is maintainable.
Quoted 6 lines
The problematic modules are
kvm_intelandkvm_amd, so we cannot build them in a mutually exclusive way (but we know it won't make sense to load both). We will have to build only one of them when built into vmlinux, but the module case must support building both.
Quoted 8 lines
Currently we put the common symbols in kvm.ko, and kvm.ko acts as a kind of "library" for kvm_intel.ko and kvm_amd.ko. The problem is that kvm_intel.ko and kvm_amd.ko currently pass a large array of function pointers to kvm.ko, and Andrea measured a substantial performance penalty from retpolines when kvm.ko calls back through those pointers.
Therefore he would like to remove kvm.ko, and that would result in symbols exported from two modules.
If this is a general demand, I think we can relax the 'exported twice' warning; we can show the warning only when the previous export is from vmlinux.
However, I am not sure how the module dependency should be handled when the same symbol is exported from multiple modules.
Let's say the same symbol is exported from foo.ko and bar.ko and foo.ko appears first in modules.order .
In this case, I think the foo.ko should be considered to have a higher priority than bar.ko
The modpost records MODULE_INFO(depends, foo.ko) correctly, but modules.{dep, symbols} do not seem to reflect that. Maybe, depmod does not take multiple-export into consideration ??
If we change this, I want this working consistently.
Masahiro Yamada
Quoted 7 lines
I suppose we could use code patching mechanism to avoid the retpolines.
Andrea, what do you think about that? That would have the advantage
that we won't have to removekvm_x86_ops. :)Thanks,
Paolo
Signature · 2 lines
Best Regards Masahiro Yamada
PATCH 4/13 KVM: monolithic: x86: handle the request_immediate_exit variation
In reply to Andrea Arcangelirequest_immediate_exit is one of those few cases where the pointer to function of the method isn't fixed at build time and it requires special handling because hardware_setup() may override it at runtime.
--- arch/x86/kvm/vmx/vmx.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +4 -1
@@ -7103,7 +7103,10 @@ void kvm_x86_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry)void kvm_x86_request_immediate_exit (struct kvm_vcpu *vcpu) { -to_vmx (vcpu)->req_immediate_exit = true; +if (likely (enable_preemption_timer)) +to_vmx (vcpu)->req_immediate_exit = true; +else +__kvm_request_immediate_exit (vcpu); } int kvm_x86_check_intercept(struct kvm_vcpu *vcpu,
PATCH 5/13 KVM: monolithic: add more section prefixes
In reply to Andrea ArcangeliAdd more section prefixes because with the monolithic KVM model the section checker can now do a more accurate static analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.
--- arch/powerpc/kvm/book3s.c | 2 +- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 8 ++++---- virt/kvm/arm/arm.c | 2 +- virt/kvm/kvm_main.c | 6 +++--- 5 files changed, 11 insertions(+), 11 deletions(-)
arch/powerpc/kvm/book3s.c +1 -1
@@ -1067,7 +1067,7 @@ int kvm_irq_map_chip_pin(struct kvm *kvm, unsigned irqchip, unsigned pin)#endif /* CONFIG_KVM_XICS */ -static int kvmppc_book3s_init (void ) +static __init int kvmppc_book3s_init (void ) {int r;
arch/x86/kvm/x86.c +2 -2
@@ -9272,7 +9272,7 @@ void kvm_arch_hardware_disable(void)drop_user_return_notifiers (); } -int kvm_arch_hardware_setup (void ) +__init int kvm_arch_hardware_setup (void ) {int r;
@@ -9303,7 +9303,7 @@ void kvm_arch_hardware_unsetup(void)kvm_x86_hardware_unsetup (); } -int kvm_arch_check_processor_compat (void ) +__init int kvm_arch_check_processor_compat (void ) {return kvm_x86_check_processor_compatibility (); }
include/linux/kvm_host.h +4 -4
@@ -616,8 +616,8 @@ static inline void kvm_irqfd_exit(void) { }#endif -int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, -struct module *module); +__init int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, +struct module *module);void kvm_exit (void );void kvm_get_kvm (struct kvm *kvm);
@@ -867,9 +867,9 @@ void kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu);int kvm_arch_hardware_enable (void );void kvm_arch_hardware_disable (void ); -int kvm_arch_hardware_setup (void ); +__init int kvm_arch_hardware_setup (void );void kvm_arch_hardware_unsetup (void ); -int kvm_arch_check_processor_compat (void ); +__init int kvm_arch_check_processor_compat (void );int kvm_arch_vcpu_runnable (struct kvm_vcpu *vcpu);bool kvm_arch_vcpu_in_kernel (struct kvm_vcpu *vcpu);int kvm_arch_vcpu_should_kick (struct kvm_vcpu *vcpu);
virt/kvm/arm/arm.c +1 -1
@@ -1726,7 +1726,7 @@ void kvm_arch_exit(void)kvm_perf_teardown (); } -static int arm_init (void ) +static __init int arm_init (void ) {int rc =kvm_init (NULL ,sizeof (struct kvm_vcpu ),0 ,THIS_MODULE );return rc;
virt/kvm/kvm_main.c +3 -3
@@ -4246,13 +4246,13 @@ static void kvm_sched_out(struct preempt_notifier *pn,kvm_arch_vcpu_put (vcpu); } -static void check_processor_compat (void *rtn) +static __init void check_processor_compat (void *rtn) { *(int *)rtn =kvm_arch_check_processor_compat (); } -int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, -struct module *module) +__init int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, +struct module *module) {int r;int cpu;
Re: [PATCH 05/13] KVM: monolithic: add more section prefixes
Quoted 101 lines · Andrea Arcangeli
Add more section prefixes because with the monolithic KVM model the section checker can now do a more accurate static analysis at build time and this allows to build without
CONFIG_SECTION_MISMATCH_WARN_ONLY=n.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/powerpc/kvm/book3s.c | 2 +- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 8 ++++---- virt/kvm/arm/arm.c | 2 +- virt/kvm/kvm_main.c | 6 +++--- 5 files changed, 11 insertions(+), 11 deletions(-)arch/powerpc/kvm/book3s.c +1
-1@@ -1067,7 +1067,7 @@ int kvm_irq_map_chip_pin(struct kvm *kvm, unsigned irqchip, unsigned pin)#endif /* CONFIG_KVM_XICS */ -static int kvmppc_book3s_init (void ) +static __init int kvmppc_book3s_init (void ) {int r;arch/x86/kvm/x86.c +2
-2@@ -9272,7 +9272,7 @@ void kvm_arch_hardware_disable(void)drop_user_return_notifiers (); } -int kvm_arch_hardware_setup (void ) +__init int kvm_arch_hardware_setup (void ) {int r;@@ -9303,7 +9303,7 @@ void kvm_arch_hardware_unsetup(void)kvm_x86_hardware_unsetup (); } -int kvm_arch_check_processor_compat (void ) +__init int kvm_arch_check_processor_compat (void ) {return kvm_x86_check_processor_compatibility (); }include/linux/kvm_host.h +4
-4@@ -616,8 +616,8 @@ static inline void kvm_irqfd_exit(void) { }#endif -int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, -struct module *module); +__init int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, +struct module *module);void kvm_exit (void );void kvm_get_kvm (struct kvm *kvm);@@ -867,9 +867,9 @@ void kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu);int kvm_arch_hardware_enable (void );void kvm_arch_hardware_disable (void ); -int kvm_arch_hardware_setup (void ); +__init int kvm_arch_hardware_setup (void );void kvm_arch_hardware_unsetup (void ); -int kvm_arch_check_processor_compat (void ); +__init int kvm_arch_check_processor_compat (void );int kvm_arch_vcpu_runnable (struct kvm_vcpu *vcpu);bool kvm_arch_vcpu_in_kernel (struct kvm_vcpu *vcpu);int kvm_arch_vcpu_should_kick (struct kvm_vcpu *vcpu);virt/kvm/arm/arm.c +1
-1@@ -1726,7 +1726,7 @@ void kvm_arch_exit(void)kvm_perf_teardown (); } -static int arm_init (void ) +static __init int arm_init (void ) {int rc =kvm_init (NULL ,sizeof (struct kvm_vcpu ),0 ,THIS_MODULE );return rc;virt/kvm/kvm_main.c +1
-1@@ -4246,13 +4246,13 @@ static void kvm_sched_out(struct preempt_notifier *pn,kvm_arch_vcpu_put (vcpu); } -static void check_processor_compat (void *rtn) +static __init void check_processor_compat (void *rtn) {
virt/kvm/kvm_main.c static void kvm_sched_out(struct preempt_notifier *pn, · line 4258
Quoted 10 lines
virt/kvm/kvm_main.c +2
-2*(int *)rtn =kvm_arch_check_processor_compat (); } -int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, -struct module *module) +__init int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align, +struct module *module) {int r;int cpu;
Queued, thanks.
Paolo
PATCH 6/13 KVM: monolithic: x86: remove __exit section prefix from machine_unsetup
In reply to Andrea ArcangeliAdjusts the section prefixes of some KVM x86 code function because with the monolithic KVM model the section checker can now do a more accurate static analysis at build time and it found a potentially kernel crashing bug. This also allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.
The __exit removed from machine_unsetup is because kvm_arch_hardware_unsetup() is called by kvm_init() which is in the __init section. It's not allowed to call a function located in the __exit section and dropped during the kernel link from the __init section or the kernel will crash if that call is made.
--- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -2
@@ -1004,7 +1004,7 @@ extern int kvm_x86_hardware_enable(void);extern void kvm_x86_hardware_disable (void );extern __init int kvm_x86_check_processor_compatibility (void );extern __init int kvm_x86_hardware_setup (void ); -extern __exit void kvm_x86_hardware_unsetup(void); +extern void kvm_x86_hardware_unsetup (void );extern bool kvm_x86_cpu_has_accelerated_tpr (void );extern bool kvm_x86_has_emulated_msr (int index);extern void kvm_x86_cpuid_update (struct kvm_vcpu *vcpu);
@@ -1196,7 +1196,7 @@ struct kvm_x86_ops {void (*hardware_disable)(void );int (*check_processor_compatibility)(void );/* __init */ int (*hardware_setup)(void );/* __init */ -void (*hardware_unsetup)(void );/* __exit */ +void (*hardware_unsetup)(void );bool (*cpu_has_accelerated_tpr)(void );bool (*has_emulated_msr)(int index);void (*cpuid_update)(struct kvm_vcpu *vcpu);
arch/x86/kvm/svm.c +1 -1
@@ -1412,7 +1412,7 @@ __init int kvm_x86_hardware_setup(void)return r; } -__exit void kvm_x86_hardware_unsetup(void) +void kvm_x86_hardware_unsetup (void ) {int cpu;
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7737,7 +7737,7 @@ __init int kvm_x86_hardware_setup(void)return r; } -__exit void kvm_x86_hardware_unsetup(void) +void kvm_x86_hardware_unsetup (void ) {if (nested)nested_vmx_hardware_unsetup ();
Re: [PATCH 06/13] KVM: monolithic: x86: remove __exit section prefix from machine_unsetup
Quoted 60 lines · Andrea Arcangeli
Adjusts the section prefixes of some KVM x86 code function because with the monolithic KVM model the section checker can now do a more accurate static analysis at build time and it found a potentially kernel crashing bug. This also allows to build without
CONFIG_SECTION_MISMATCH_WARN_ONLY=n.The
__exitremoved frommachine_unsetupis becausekvm_arch_hardware_unsetup()is called bykvm_init()which is in the__initsection. It's not allowed to call a function located in the__exitsection and dropped during the kernel link from the__initsection or the kernel will crash if that call is made.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)arch/x86/include/asm/kvm_host.h +2
-2@@ -1004,7 +1004,7 @@ extern int kvm_x86_hardware_enable(void);extern void kvm_x86_hardware_disable (void );extern __init int kvm_x86_check_processor_compatibility (void );extern __init int kvm_x86_hardware_setup (void ); -extern __exit void kvm_x86_hardware_unsetup(void); +extern void kvm_x86_hardware_unsetup (void );extern bool kvm_x86_cpu_has_accelerated_tpr (void );extern bool kvm_x86_has_emulated_msr (int index);extern void kvm_x86_cpuid_update (struct kvm_vcpu *vcpu);@@ -1196,7 +1196,7 @@ struct kvm_x86_ops {void (*hardware_disable)(void );int (*check_processor_compatibility)(void );/* __init */ int (*hardware_setup)(void );/* __init */ -void (*hardware_unsetup)(void );/* __exit */ +void (*hardware_unsetup)(void );bool (*cpu_has_accelerated_tpr)(void );bool (*has_emulated_msr)(int index);void (*cpuid_update)(struct kvm_vcpu *vcpu);arch/x86/kvm/svm.c +1
-1@@ -1412,7 +1412,7 @@ __init int kvm_x86_hardware_setup(void)return r; } -__exit void kvm_x86_hardware_unsetup(void) +void kvm_x86_hardware_unsetup (void ) {int cpu;arch/x86/kvm/vmx/vmx.c +0
-0
arch/x86/kvm/vmx/vmx.c __init int kvm_x86_hardware_setup(void) · line 7743
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -7737,7 +7737,7 @@ __init int kvm_x86_hardware_setup(void)return r; } -__exit void kvm_x86_hardware_unsetup(void) +void kvm_x86_hardware_unsetup (void ) {if (nested)nested_vmx_hardware_unsetup ();
Queued, thanks.
Paolo
PATCH 7/13 KVM: monolithic: x86: remove __init section prefix from kvm_x86_cpu_has_kvm_support
In reply to Andrea ArcangeliAdjusts the section prefixes of some KVM x86 code function because with the monolithic KVM model the section checker can now do a more accurate static analysis at build time. This also allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.
The __init needs to be removed on vmx despite it's only svm calling it from kvm_x86_hardware_enable which is eventually called by hardware_enable_nolock() or there's a (potentially false positive) warning (false positive because this function isn't called in the vmx case). If this isn't needed the right cleanup isn't to put it in the __init section, but to drop it. As long as it's defined in vmx as a kvm_x86 operation, it's expectable that might eventually be called at runtime while hot plugging new CPUs.
--- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/vmx/vmx.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -2
@@ -998,7 +998,7 @@ struct kvm_lapic_irq {bool msi_redir_hint; }; -extern __init int kvm_x86_cpu_has_kvm_support (void ); +extern int kvm_x86_cpu_has_kvm_support (void );extern __init int kvm_x86_disabled_by_bios (void );extern int kvm_x86_hardware_enable (void );extern void kvm_x86_hardware_disable (void );
@@ -1190,7 +1190,7 @@ extern bool kvm_x86_apic_init_signal_blocked(struct kvm_vcpu *vcpu);extern int kvm_x86_enable_direct_tlbflush (struct kvm_vcpu *vcpu);struct kvm_x86_ops { -int (*cpu_has_kvm_support)(void );/* __init */ +int (*cpu_has_kvm_support)(void );int (*disabled_by_bios)(void );/* __init */ int (*hardware_enable)(void );void (*hardware_disable)(void );
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -2178,7 +2178,7 @@ void kvm_x86_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) } } -__init int kvm_x86_cpu_has_kvm_support (void ) +int kvm_x86_cpu_has_kvm_support (void ) {return cpu_has_vmx (); }
Paolo Bonzini
maintainerRe: [PATCH 07/13] KVM: monolithic: x86: remove __init section prefix from kvm_x86_cpu_has_kvm_support
Quoted 48 lines · Andrea Arcangeli
Adjusts the section prefixes of some KVM x86 code function because with the monolithic KVM model the section checker can now do a more accurate static analysis at build time. This also allows to build without
CONFIG_SECTION_MISMATCH_WARN_ONLY=n.The
__initneeds to be removed on vmx despite it's only svm calling it fromkvm_x86_hardware_enablewhich is eventually called byhardware_enable_nolock()or there's a (potentially false positive) warning (false positive because this function isn't called in the vmx case). If this isn't needed the right cleanup isn't to put it in the__initsection, but to drop it. As long as it's defined in vmx as akvm_x86operation, it's expectable that might eventually be called at runtime while hot plugging new CPUs.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/vmx/vmx.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)arch/x86/include/asm/kvm_host.h +2
-2@@ -998,7 +998,7 @@ struct kvm_lapic_irq {bool msi_redir_hint; }; -extern __init int kvm_x86_cpu_has_kvm_support (void ); +extern int kvm_x86_cpu_has_kvm_support (void );extern __init int kvm_x86_disabled_by_bios (void );extern int kvm_x86_hardware_enable (void );extern void kvm_x86_hardware_disable (void );@@ -1190,7 +1190,7 @@ extern bool kvm_x86_apic_init_signal_blocked(struct kvm_vcpu *vcpu);extern int kvm_x86_enable_direct_tlbflush (struct kvm_vcpu *vcpu);struct kvm_x86_ops { -int (*cpu_has_kvm_support)(void );/* __init */ +int (*cpu_has_kvm_support)(void );int (*disabled_by_bios)(void );/* __init */ int (*hardware_enable)(void );void (*hardware_disable)(void );arch/x86/kvm/vmx/vmx.c +0
-0
arch/x86/kvm/vmx/vmx.c void kvm_x86_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) · line 2184
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -2178,7 +2178,7 @@ void kvm_x86_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) } } -__init int kvm_x86_cpu_has_kvm_support (void ) +int kvm_x86_cpu_has_kvm_support (void ) {return cpu_has_vmx (); }
I think we should eliminate all the complications in cpu_has_svm(), so that svm_hardware_enable can use it. I'll post a patch.
Paolo
PATCH 8/13 KVM: monolithic: remove exports
In reply to Andrea ArcangeliThe exports would be duplicated across kvm-amd and kvm-intel if they're kept and that causes various harmless warnings.
The warnings aren't particularly concerning because the two modules can't load at the same time, but it's cleaner to remove the warnings by removing the exports.
This commit might break non-x86 archs, but it should be simple to make them monolithic too (if they're not already).
In the unlikely case there's a legit reason not to go monolithic in any arch and to keep kvm.ko around, we'll need a way to retain the common code exports. In which case this commit would need to be partially reversed and the exports in the kvm common code should then be done only conditionally to a new opt-in per-arch CONFIG option.
The following warning remains for now to be able to load the kvmgt and the powerpc version. These remaining warnings can be handled later.
WARNING: arch/x86/kvm/kvm-amd: 'kvm_debugfs_dir' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
This is needed by powerpc.
WARNING: arch/x86/kvm/kvm-amd: 'kvm_get_kvm' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_put_kvm' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'gfn_to_memslot' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_is_visible_gfn' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'gfn_to_pfn' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_read_guest' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_write_guest' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_slot_page_track_add_page' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_slot_page_track_remove_page' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_page_track_register_notifier' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_page_track_unregister_notifier' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
This is needed by kvmgt.c.
--- arch/x86/kvm/cpuid.c | 5 -- arch/x86/kvm/hyperv.c | 2 - arch/x86/kvm/irq.c | 4 -- arch/x86/kvm/irq_comm.c | 2 - arch/x86/kvm/lapic.c | 16 ------ arch/x86/kvm/mmu.c | 24 --------- arch/x86/kvm/mtrr.c | 2 - arch/x86/kvm/pmu.c | 3 -- arch/x86/kvm/x86.c | 106 ---------------------------------------- virt/kvm/eventfd.c | 1 - virt/kvm/kvm_main.c | 64 ------------------------ 11 files changed, 229 deletions(-)
arch/x86/kvm/cpuid.c +0 -5
@@ -50,7 +50,6 @@ bool kvm_mpx_supported(void)return ((host_xcr0 & (XFEATURE_MASK_BNDREGS |XFEATURE_MASK_BNDCSR )) &&kvm_x86_mpx_supported ()); } -EXPORT_SYMBOL_GPL (kvm_mpx_supported);u64 kvm_supported_xcr0 (void ) {
@@ -192,7 +191,6 @@ int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu) not_found:return 36 ; } -EXPORT_SYMBOL_GPL (cpuid_query_maxphyaddr);/* when an old userspace process fills a new kernel module */ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu,
@@ -971,7 +969,6 @@ struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu, }return best; } -EXPORT_SYMBOL_GPL (kvm_find_cpuid_entry);/* * If the basic or extended CPUID leaf requested is higher than the
@@ -1035,7 +1032,6 @@ bool kvm_cpuid(struct kvm_vcpu *vcpu, u32 *eax, u32 *ebx,trace_kvm_cpuid (function, *eax, *ebx, *ecx, *edx, found);return found; } -EXPORT_SYMBOL_GPL (kvm_cpuid);int kvm_emulate_cpuid (struct kvm_vcpu *vcpu) {
@@ -1053,4 +1049,3 @@ int kvm_emulate_cpuid(struct kvm_vcpu *vcpu)kvm_rdx_write (vcpu, edx);return kvm_skip_emulated_instruction (vcpu); } -EXPORT_SYMBOL_GPL (kvm_emulate_cpuid);
arch/x86/kvm/hyperv.c +0 -2
@@ -714,7 +714,6 @@ bool kvm_hv_assist_page_enabled(struct kvm_vcpu *vcpu)return false;return vcpu->arch.pv_eoi.msr_val &KVM_MSR_ENABLED ; } -EXPORT_SYMBOL_GPL (kvm_hv_assist_page_enabled);bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu,struct hv_vp_assist_page *assist_page)
@@ -724,7 +723,6 @@ bool kvm_hv_get_assist_page(struct kvm_vcpu *vcpu,return !kvm_read_guest_cached (vcpu->kvm, &vcpu->arch.pv_eoi.data, assist_page,sizeof (*assist_page)); } -EXPORT_SYMBOL_GPL (kvm_hv_get_assist_page);static void stimer_prepare_msg (struct kvm_vcpu_hv_stimer *stimer) {
arch/x86/kvm/irq.c +0 -4
@@ -26,7 +26,6 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)return 0 ; } -EXPORT_SYMBOL (kvm_cpu_has_pending_timer);/* * check if there is a pending userspace external interrupt
@@ -109,7 +108,6 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v)return kvm_apic_has_interrupt (v) !=-1 ;/* LAPIC */ } -EXPORT_SYMBOL_GPL (kvm_cpu_has_interrupt);/* * Read pending interrupt(from non-APIC source)
@@ -146,14 +144,12 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v)return kvm_get_apic_interrupt (v);/* APIC */ } -EXPORT_SYMBOL_GPL (kvm_cpu_get_interrupt);void kvm_inject_pending_timer_irqs (struct kvm_vcpu *vcpu) {if (lapic_in_kernel (vcpu))kvm_inject_apic_timer_irqs (vcpu); } -EXPORT_SYMBOL_GPL (kvm_inject_pending_timer_irqs);void __kvm_migrate_timers (struct kvm_vcpu *vcpu) {
arch/x86/kvm/irq_comm.c +0 -2
@@ -122,7 +122,6 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, irq->level =1 ; irq->shorthand =0 ; } -EXPORT_SYMBOL_GPL (kvm_set_msi_irq);static inline bool kvm_msi_route_invalid (struct kvm *kvm,struct kvm_kernel_irq_routing_entry *e)
@@ -346,7 +345,6 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq,return r ==1 ; } -EXPORT_SYMBOL_GPL (kvm_intr_is_single_vcpu);#define IOAPIC_ROUTING_ENTRY (irq) \ { .gsi = irq, .type = KVM_IRQ_ROUTING_IRQCHIP, \
arch/x86/kvm/lapic.c +0 -16
@@ -120,7 +120,6 @@ bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) {return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); } -EXPORT_SYMBOL_GPL (kvm_can_post_timer_interrupt);static bool kvm_use_posted_timer_interrupt (struct kvm_vcpu *vcpu) {
@@ -410,7 +409,6 @@ bool __kvm_apic_update_irr(u32 *pir, void *regs, int *max_irr)return ((max_updated_irr !=-1 ) && (max_updated_irr == *max_irr)); } -EXPORT_SYMBOL_GPL (__kvm_apic_update_irr);bool kvm_apic_update_irr (struct kvm_vcpu *vcpu,u32 *pir,int *max_irr) {
@@ -418,7 +416,6 @@ bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir, int *max_irr)return __kvm_apic_update_irr (pir, apic->regs, max_irr); } -EXPORT_SYMBOL_GPL (kvm_apic_update_irr);static inline int apic_search_irr (struct kvm_lapic *apic) {
@@ -542,7 +539,6 @@ int kvm_lapic_find_highest_irr(struct kvm_vcpu *vcpu)*/ return apic_find_highest_irr (vcpu->arch.apic); } -EXPORT_SYMBOL_GPL (kvm_lapic_find_highest_irr); static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode, int vector, int level, int trig_mode,
@@ -710,7 +706,6 @@ void kvm_apic_update_ppr(struct kvm_vcpu *vcpu) {apic_update_ppr (vcpu->arch.apic); } -EXPORT_SYMBOL_GPL (kvm_apic_update_ppr);static void apic_set_tpr (struct kvm_lapic *apic,u32 tpr) {
@@ -821,7 +816,6 @@ bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,return false; } } -EXPORT_SYMBOL_GPL (kvm_apic_match_dest);int kvm_vector_to_index (u32 vector,u32 dest_vcpus,const unsigned long *bitmap,u32 bitmap_size)
@@ -1194,7 +1188,6 @@ void kvm_apic_set_eoi_accelerated(struct kvm_vcpu *vcpu, int vector)kvm_ioapic_send_eoi (apic, vector);kvm_make_request (KVM_REQ_EVENT , apic->vcpu); } -EXPORT_SYMBOL_GPL (kvm_apic_set_eoi_accelerated);static void apic_send_ipi (struct kvm_lapic *apic,u32 icr_low,u32 icr_high) {
@@ -1353,7 +1346,6 @@ int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len, }return 0 ; } -EXPORT_SYMBOL_GPL (kvm_lapic_reg_read);static int apic_mmio_in_range (struct kvm_lapic *apic,gpa_t addr) {
@@ -1530,7 +1522,6 @@ void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)if (lapic_timer_int_injected (vcpu))__kvm_wait_lapic_expire (vcpu); } -EXPORT_SYMBOL_GPL (kvm_wait_lapic_expire);static void kvm_apic_inject_pending_timer_irqs (struct kvm_lapic *apic) {
@@ -1695,7 +1686,6 @@ bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu)return vcpu->arch.apic->lapic_timer.hv_timer_in_use; } -EXPORT_SYMBOL_GPL (kvm_lapic_hv_timer_in_use);static void cancel_hv_timer (struct kvm_lapic *apic) {
@@ -1796,13 +1786,11 @@ void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu) out:preempt_enable (); } -EXPORT_SYMBOL_GPL (kvm_lapic_expired_hv_timer);void kvm_lapic_switch_to_hv_timer (struct kvm_vcpu *vcpu) {restart_apic_timer (vcpu->arch.apic); } -EXPORT_SYMBOL_GPL (kvm_lapic_switch_to_hv_timer);void kvm_lapic_switch_to_sw_timer (struct kvm_vcpu *vcpu) {
@@ -1814,7 +1802,6 @@ void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu)start_sw_timer (apic);preempt_enable (); } -EXPORT_SYMBOL_GPL (kvm_lapic_switch_to_sw_timer);void kvm_lapic_restart_hv_timer (struct kvm_vcpu *vcpu) {
@@ -1984,7 +1971,6 @@ int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)return ret; } -EXPORT_SYMBOL_GPL (kvm_lapic_reg_write);static int apic_mmio_write (struct kvm_vcpu *vcpu,struct kvm_io_device *this,gpa_t address,int len,const void *data)
@@ -2023,7 +2009,6 @@ void kvm_lapic_set_eoi(struct kvm_vcpu *vcpu) {kvm_lapic_reg_write (vcpu->arch.apic,APIC_EOI ,0 ); } -EXPORT_SYMBOL_GPL (kvm_lapic_set_eoi);/* emulate APIC access in a trap manner */ void kvm_apic_write_nodecode (struct kvm_vcpu *vcpu,u32 offset)
@@ -2038,7 +2023,6 @@ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset)/* TODO: optimize to just emulate side effect w/o one more write */ kvm_lapic_reg_write (vcpu->arch.apic, offset, val); } -EXPORT_SYMBOL_GPL (kvm_apic_write_nodecode);void kvm_free_lapic (struct kvm_vcpu *vcpu) {
arch/x86/kvm/mmu.c +0 -24
@@ -317,7 +317,6 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask, u64 mmio_value, u64 access_mask) shadow_mmio_mask = mmio_mask |SPTE_SPECIAL_MASK ; shadow_mmio_access_mask = access_mask; } -EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_mask);static bool is_mmio_spte (u64 spte) {
@@ -498,7 +497,6 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask, shadow_acc_track_mask = acc_track_mask; shadow_me_mask = me_mask; } -EXPORT_SYMBOL_GPL (kvm_mmu_set_mask_ptes);static u8 kvm_get_shadow_phys_bits (void ) {
@@ -1731,7 +1729,6 @@ void kvm_mmu_clear_dirty_pt_masked(struct kvm *kvm, mask &= mask -1 ; } } -EXPORT_SYMBOL_GPL (kvm_mmu_clear_dirty_pt_masked);/** * kvm_arch_mmu_enable_log_dirty_pt_masked - enable dirty logging for selected
@@ -2888,7 +2885,6 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn)return r; } -EXPORT_SYMBOL_GPL (kvm_mmu_unprotect_page);static void kvm_unsync_page (struct kvm_vcpu *vcpu,struct kvm_mmu_page *sp) {
@@ -3658,7 +3654,6 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,kvm_mmu_commit_zap_page (vcpu->kvm, &invalid_list);spin_unlock (&vcpu->kvm->mmu_lock); } -EXPORT_SYMBOL_GPL (kvm_mmu_free_roots);static int mmu_check_root (struct kvm_vcpu *vcpu,gfn_t root_gfn) {
@@ -3883,7 +3878,6 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu)kvm_mmu_audit (vcpu,AUDIT_POST_SYNC );spin_unlock (&vcpu->kvm->mmu_lock); } -EXPORT_SYMBOL_GPL (kvm_mmu_sync_roots);static gpa_t nonpaging_gva_to_gpa (struct kvm_vcpu *vcpu,gva_t vaddr,u32 access,struct x86_exception *exception)
@@ -4151,7 +4145,6 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, }return r; } -EXPORT_SYMBOL_GPL (kvm_handle_page_fault);static bool check_hugepage_cache_consistency (struct kvm_vcpu *vcpu,gfn_t gfn,int level)
@@ -4331,7 +4324,6 @@ void kvm_mmu_new_cr3(struct kvm_vcpu *vcpu, gpa_t new_cr3, bool skip_tlb_flush)__kvm_mmu_new_cr3 (vcpu, new_cr3,kvm_mmu_calc_root_page_role (vcpu), skip_tlb_flush); } -EXPORT_SYMBOL_GPL (kvm_mmu_new_cr3);static unsigned long get_cr3 (struct kvm_vcpu *vcpu) {
@@ -4571,7 +4563,6 @@ reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, struct kvm_mmu *context) } } -EXPORT_SYMBOL_GPL (reset_shadow_zero_bits_mask);static inline bool boot_cpu_is_amd (void ) {
@@ -4991,7 +4982,6 @@ void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu) context->mmu_role.as_u64 = new_role.as_u64;reset_shadow_zero_bits_mask (vcpu, context); } -EXPORT_SYMBOL_GPL (kvm_init_shadow_mmu); static union kvm_mmu_role kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *vcpu, bool accessed_dirty,
@@ -5055,7 +5045,6 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly,reset_rsvds_bits_mask_ept (vcpu, context, execonly);reset_ept_shadow_zero_bits_mask (vcpu, context, execonly); } -EXPORT_SYMBOL_GPL (kvm_init_shadow_ept_mmu);static void init_kvm_softmmu (struct kvm_vcpu *vcpu) {
@@ -5135,7 +5124,6 @@ void kvm_init_mmu(struct kvm_vcpu *vcpu, bool reset_roots)else init_kvm_softmmu (vcpu); } -EXPORT_SYMBOL_GPL (kvm_init_mmu);static union kvm_mmu_page_role kvm_mmu_calc_root_page_role (struct kvm_vcpu *vcpu)
@@ -5155,7 +5143,6 @@ void kvm_mmu_reset_context(struct kvm_vcpu *vcpu)kvm_mmu_unload (vcpu);kvm_init_mmu (vcpu, true); } -EXPORT_SYMBOL_GPL (kvm_mmu_reset_context);int kvm_mmu_load (struct kvm_vcpu *vcpu) {
@@ -5173,7 +5160,6 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) out:return r; } -EXPORT_SYMBOL_GPL (kvm_mmu_load);void kvm_mmu_unload (struct kvm_vcpu *vcpu) {
@@ -5182,7 +5168,6 @@ void kvm_mmu_unload(struct kvm_vcpu *vcpu)kvm_mmu_free_roots (vcpu, &vcpu->arch.guest_mmu,KVM_MMU_ROOTS_ALL );WARN_ON (VALID_PAGE (vcpu->arch.guest_mmu.root_hpa)); } -EXPORT_SYMBOL_GPL (kvm_mmu_unload); static void mmu_pte_write_new_pte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, u64 *spte,
@@ -5394,7 +5379,6 @@ int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva)return r; } -EXPORT_SYMBOL_GPL (kvm_mmu_unprotect_page_virt);static int make_mmu_pages_available (struct kvm_vcpu *vcpu) {
@@ -5489,7 +5473,6 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code,return x86_emulate_instruction (vcpu, cr2, emulation_type, insn, insn_len); } -EXPORT_SYMBOL_GPL (kvm_mmu_page_fault);void kvm_mmu_invlpg (struct kvm_vcpu *vcpu,gva_t gva) {
@@ -5520,7 +5503,6 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)kvm_x86_tlb_flush_gva (vcpu, gva); ++vcpu->stat.invlpg; } -EXPORT_SYMBOL_GPL (kvm_mmu_invlpg);void kvm_mmu_invpcid_gva (struct kvm_vcpu *vcpu,gva_t gva,unsigned long pcid) {
@@ -5552,19 +5534,16 @@ void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid)* for them. */ } -EXPORT_SYMBOL_GPL (kvm_mmu_invpcid_gva);void kvm_enable_tdp (void ) { tdp_enabled = true; } -EXPORT_SYMBOL_GPL (kvm_enable_tdp);void kvm_disable_tdp (void ) { tdp_enabled = false; } -EXPORT_SYMBOL_GPL (kvm_disable_tdp);/* The return value indicates if tlb flush on all vcpus is needed. */
@@ -5963,7 +5942,6 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,kvm_flush_remote_tlbs_with_address (kvm, memslot->base_gfn, memslot->npages); } -EXPORT_SYMBOL_GPL (kvm_mmu_slot_leaf_clear_dirty);void kvm_mmu_slot_largepage_remove_write_access (struct kvm *kvm,struct kvm_memory_slot *memslot)
@@ -5982,7 +5960,6 @@ void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm,kvm_flush_remote_tlbs_with_address (kvm, memslot->base_gfn, memslot->npages); } -EXPORT_SYMBOL_GPL (kvm_mmu_slot_largepage_remove_write_access);void kvm_mmu_slot_set_dirty (struct kvm *kvm,struct kvm_memory_slot *memslot)
@@ -6000,7 +5977,6 @@ void kvm_mmu_slot_set_dirty(struct kvm *kvm,kvm_flush_remote_tlbs_with_address (kvm, memslot->base_gfn, memslot->npages); } -EXPORT_SYMBOL_GPL (kvm_mmu_slot_set_dirty);void kvm_mmu_zap_all (struct kvm *kvm) {
arch/x86/kvm/mtrr.c +0 -2
@@ -91,7 +91,6 @@ bool kvm_mtrr_valid(struct kvm_vcpu *vcpu, u32 msr, u64 data)return true; } -EXPORT_SYMBOL_GPL (kvm_mtrr_valid);static bool mtrr_is_enabled (struct kvm_mtrr *mtrr_state) {
@@ -686,7 +685,6 @@ u8 kvm_mtrr_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn)return type; } -EXPORT_SYMBOL_GPL (kvm_mtrr_get_guest_memory_type);bool kvm_mtrr_check_gfn_range_consistency (struct kvm_vcpu *vcpu,gfn_t gfn,int page_num)
arch/x86/kvm/pmu.c +0 -3
@@ -200,7 +200,6 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) (eventsel &HSW_IN_TX ), (eventsel & HSW_IN_TX_CHECKPOINTED)); } -EXPORT_SYMBOL_GPL (reprogram_gp_counter);void reprogram_fixed_counter (struct kvm_pmc *pmc,u8 ctrl,int idx) {
@@ -230,7 +229,6 @@ void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx) !(en_field &0x1 ),/* exclude kernel */ pmi, false, false); } -EXPORT_SYMBOL_GPL (reprogram_fixed_counter);void reprogram_counter (struct kvm_pmu *pmu,int pmc_idx) {
@@ -248,7 +246,6 @@ void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx)reprogram_fixed_counter (pmc, ctrl, idx); } } -EXPORT_SYMBOL_GPL (reprogram_counter);void kvm_pmu_handle_event (struct kvm_vcpu *vcpu) {
arch/x86/kvm/x86.c +0 -106
573 lines not on this page · raw message
virt/kvm/eventfd.c +0 -1
@@ -462,7 +462,6 @@ bool kvm_irq_has_notifier(struct kvm *kvm, unsigned irqchip, unsigned pin)return false; } -EXPORT_SYMBOL_GPL (kvm_irq_has_notifier);void kvm_notify_acked_gsi (struct kvm *kvm,int gsi) {
virt/kvm/kvm_main.c +0 -64
433 lines not on this page · raw message
PATCH 9/13 KVM: monolithic: x86: drop the kvm_pmu_ops structure
In reply to Andrea ArcangeliCleanup after the structure was finally left completely unused.
--- arch/x86/include/asm/kvm_host.h | 3 --- arch/x86/kvm/pmu.h | 20 -------------------- arch/x86/kvm/pmu_amd.c | 15 --------------- arch/x86/kvm/svm.c | 1 - arch/x86/kvm/vmx/pmu_intel.c | 15 --------------- arch/x86/kvm/vmx/vmx.c | 2 -- 6 files changed, 56 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -3
@@ -1345,9 +1345,6 @@ struct kvm_x86_ops { gfn_t offset, unsigned long mask);int (*write_log_dirty)(struct kvm_vcpu *vcpu); -/* pmu operations of sub-arch */ -const struct kvm_pmu_ops *pmu_ops; -/* * Architecture specific hooks for vCPU blocking due to * HLT instruction.
arch/x86/kvm/pmu.h +0 -20
@@ -36,23 +36,6 @@ extern void kvm_x86_pmu_refresh(struct kvm_vcpu *vcpu);extern void kvm_x86_pmu_init (struct kvm_vcpu *vcpu);extern void kvm_x86_pmu_reset (struct kvm_vcpu *vcpu); -struct kvm_pmu_ops { -unsigned (*find_arch_event)(struct kvm_pmu *pmu,u8 event_select, -u8 unit_mask); -unsigned (*find_fixed_event)(int idx); -bool (*pmc_is_enabled)(struct kvm_pmc *pmc); -struct kvm_pmc *(*pmc_idx_to_pmc)(struct kvm_pmu *pmu,int pmc_idx); -struct kvm_pmc *(*msr_idx_to_pmc)(struct kvm_vcpu *vcpu,unsigned idx, -u64 *mask); -int (*is_valid_msr_idx)(struct kvm_vcpu *vcpu,unsigned idx); -bool (*is_valid_msr)(struct kvm_vcpu *vcpu,u32 msr); -int (*get_msr)(struct kvm_vcpu *vcpu,u32 msr,u64 *data); -int (*set_msr)(struct kvm_vcpu *vcpu,struct msr_data *msr_info); -void (*refresh)(struct kvm_vcpu *vcpu); -void (*init)(struct kvm_vcpu *vcpu); -void (*reset)(struct kvm_vcpu *vcpu); -}; -static inline u64 pmc_bitmask (struct kvm_pmc *pmc) {struct kvm_pmu *pmu =pmc_to_pmu (pmc);
@@ -138,7 +121,4 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu);int kvm_vm_ioctl_set_pmu_event_filter (struct kvm *kvm,void __user *argp);bool is_vmware_backdoor_pmc (u32 pmc_idx); - -extern struct kvm_pmu_ops intel_pmu_ops; -extern struct kvm_pmu_ops amd_pmu_ops;#endif /* __KVM_X86_PMU_H */
arch/x86/kvm/pmu_amd.c +0 -15
@@ -300,18 +300,3 @@ void kvm_x86_pmu_reset(struct kvm_vcpu *vcpu) pmc->counter = pmc->eventsel =0 ; } } - -struct kvm_pmu_ops amd_pmu_ops = { - .find_arch_event = kvm_x86_pmu_find_arch_event, - .find_fixed_event = kvm_x86_pmu_find_fixed_event, - .pmc_is_enabled = kvm_x86_pmu_pmc_is_enabled, - .pmc_idx_to_pmc = kvm_x86_pmu_pmc_idx_to_pmc, - .msr_idx_to_pmc = kvm_x86_pmu_msr_idx_to_pmc, - .is_valid_msr_idx = kvm_x86_pmu_is_valid_msr_idx, - .is_valid_msr = kvm_x86_pmu_is_valid_msr, - .get_msr = kvm_x86_pmu_get_msr, - .set_msr = kvm_x86_pmu_set_msr, - .refresh = kvm_x86_pmu_refresh, - .init = kvm_x86_pmu_init, - .reset = kvm_x86_pmu_reset, -};
arch/x86/kvm/svm.c +0 -1
@@ -7290,7 +7290,6 @@ static struct kvm_x86_ops svm_x86_ops __ro_after_init = { .sched_in = kvm_x86_sched_in, - .pmu_ops = &amd_pmu_ops, .deliver_posted_interrupt = kvm_x86_deliver_posted_interrupt, .dy_apicv_has_pending_interrupt = kvm_x86_dy_apicv_has_pending_interrupt, .update_pi_irte = kvm_x86_update_pi_irte,
arch/x86/kvm/vmx/pmu_intel.c +0 -15
@@ -359,18 +359,3 @@ void kvm_x86_pmu_reset(struct kvm_vcpu *vcpu) pmu->fixed_ctr_ctrl = pmu->global_ctrl = pmu->global_status = pmu->global_ovf_ctrl =0 ; } - -struct kvm_pmu_ops intel_pmu_ops = { - .find_arch_event = kvm_x86_pmu_find_arch_event, - .find_fixed_event = kvm_x86_pmu_find_fixed_event, - .pmc_is_enabled = kvm_x86_pmu_pmc_is_enabled, - .pmc_idx_to_pmc = kvm_x86_pmu_pmc_idx_to_pmc, - .msr_idx_to_pmc = kvm_x86_pmu_msr_idx_to_pmc, - .is_valid_msr_idx = kvm_x86_pmu_is_valid_msr_idx, - .is_valid_msr = kvm_x86_pmu_is_valid_msr, - .get_msr = kvm_x86_pmu_get_msr, - .set_msr = kvm_x86_pmu_set_msr, - .refresh = kvm_x86_pmu_refresh, - .init = kvm_x86_pmu_init, - .reset = kvm_x86_pmu_reset, -};
arch/x86/kvm/vmx/vmx.c +0 -2
@@ -7897,8 +7897,6 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .pre_block = kvm_x86_pre_block, .post_block = kvm_x86_post_block, - .pmu_ops = &intel_pmu_ops, - .update_pi_irte = kvm_x86_update_pi_irte,#ifdef CONFIG_X86_64
PATCH 10/13 KVM: x86: optimize more exit handlers in vmx.c
In reply to Andrea ArcangeliEliminate wasteful call/ret non RETPOLINE case and unnecessary fentry dynamic tracing hooking points.
--- arch/x86/kvm/vmx/vmx.c | 30 +++++------------------------- 1 file changed, 5 insertions(+), 25 deletions(-)
arch/x86/kvm/vmx/vmx.c +5 -25
@@ -4694,7 +4694,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)return 0 ; } -static int handle_external_interrupt (struct kvm_vcpu *vcpu) +static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu) { ++vcpu->stat.irq_exits;return 1 ;
@@ -4965,21 +4965,6 @@ void kvm_x86_set_dr7(struct kvm_vcpu *vcpu, unsigned long val)vmcs_writel (GUEST_DR7 , val); } -static int handle_cpuid (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_cpuid (vcpu); -} - -static int handle_rdmsr (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_rdmsr (vcpu); -} - -static int handle_wrmsr (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_wrmsr (vcpu); -} -static int handle_tpr_below_threshold (struct kvm_vcpu *vcpu) {kvm_apic_update_ppr (vcpu);
@@ -4996,11 +4981,6 @@ static int handle_interrupt_window(struct kvm_vcpu *vcpu)return 1 ; } -static int handle_halt (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_halt (vcpu); -} -static int handle_vmcall (struct kvm_vcpu *vcpu) {return kvm_emulate_hypercall (vcpu);
@@ -5548,11 +5528,11 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_IO_INSTRUCTION] = handle_io, [EXIT_REASON_CR_ACCESS] = handle_cr, [EXIT_REASON_DR_ACCESS] = handle_dr, - [EXIT_REASON_CPUID] = handle_cpuid, - [EXIT_REASON_MSR_READ] = handle_rdmsr, - [EXIT_REASON_MSR_WRITE] = handle_wrmsr, + [EXIT_REASON_CPUID] = kvm_emulate_cpuid, + [EXIT_REASON_MSR_READ] = kvm_emulate_rdmsr, + [EXIT_REASON_MSR_WRITE] = kvm_emulate_wrmsr, [EXIT_REASON_PENDING_INTERRUPT] = handle_interrupt_window, - [EXIT_REASON_HLT] = handle_halt, + [EXIT_REASON_HLT] = kvm_emulate_halt, [EXIT_REASON_INVD] = handle_invd, [EXIT_REASON_INVLPG] = handle_invlpg, [EXIT_REASON_RDPMC] = handle_rdpmc,
Re: [PATCH 10/13] KVM: x86: optimize more exit handlers in vmx.c
Quoted 62 lines · Andrea Arcangeli
Eliminate wasteful call/ret non RETPOLINE case and unnecessary fentry dynamic tracing hooking points.
Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 30 +++++------------------------- 1 file changed, 5 insertions(+), 25 deletions(-)arch/x86/kvm/vmx/vmx.c +1
-23@@ -4694,7 +4694,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)return 0 ; } -static int handle_external_interrupt (struct kvm_vcpu *vcpu) +static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu) { ++vcpu->stat.irq_exits;return 1 ;@@ -4965,21 +4965,6 @@ void kvm_x86_set_dr7(struct kvm_vcpu *vcpu, unsigned long val)vmcs_writel (GUEST_DR7 , val); } -static int handle_cpuid (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_cpuid (vcpu); -} - -static int handle_rdmsr (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_rdmsr (vcpu); -} - -static int handle_wrmsr (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_wrmsr (vcpu); -} -static int handle_tpr_below_threshold (struct kvm_vcpu *vcpu) {kvm_apic_update_ppr (vcpu);@@ -4996,11 +4981,6 @@ static int handle_interrupt_window(struct kvm_vcpu *vcpu)return 1 ; } -static int handle_halt (struct kvm_vcpu *vcpu) -{ -return kvm_emulate_halt (vcpu); -} -static int handle_vmcall (struct kvm_vcpu *vcpu) {return kvm_emulate_hypercall (vcpu);@@ -5548,11 +5528,11 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_IO_INSTRUCTION] = handle_io, [EXIT_REASON_CR_ACCESS] = handle_cr, [EXIT_REASON_DR_ACCESS] = handle_dr, - [EXIT_REASON_CPUID] = handle_cpuid, - [EXIT_REASON_MSR_READ] = handle_rdmsr,
arch/x86/kvm/vmx/vmx.c static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { · line 5538
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +4
-2- [EXIT_REASON_MSR_WRITE] = handle_wrmsr, + [EXIT_REASON_CPUID] = kvm_emulate_cpuid, + [EXIT_REASON_MSR_READ] = kvm_emulate_rdmsr, + [EXIT_REASON_MSR_WRITE] = kvm_emulate_wrmsr, [EXIT_REASON_PENDING_INTERRUPT] = handle_interrupt_window, - [EXIT_REASON_HLT] = handle_halt, + [EXIT_REASON_HLT] = kvm_emulate_halt, [EXIT_REASON_INVD] = handle_invd, [EXIT_REASON_INVLPG] = handle_invlpg, [EXIT_REASON_RDPMC] = handle_rdpmc,
Queued, thanks.
Paolo
PATCH 11/13 KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers
In reply to Andrea ArcangeliIt's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.
Of course CONFIG_RETPOLINE already forbids gcc to use indirect jumps while compiling all switch() statements, however switch() would still allow the compiler to bisect the case value. It's more efficient to prioritize the most frequent vmexits instead.
The halt may be slow paths from the point of the guest, but not necessarily so from the point of the host if the host runs at full CPU capacity and no host CPU is ever left idle.
--- arch/x86/kvm/vmx/vmx.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +16 -2
@@ -5905,9 +5905,23 @@ int kvm_x86_handle_exit(struct kvm_vcpu *vcpu) }if (exit_reason < kvm_vmx_max_exit_handlers - && kvm_vmx_exit_handlers[exit_reason]) + && kvm_vmx_exit_handlers[exit_reason]) { +#ifdef CONFIG_RETPOLINE +if (exit_reason ==EXIT_REASON_MSR_WRITE ) +return kvm_emulate_wrmsr (vcpu); +else if (exit_reason ==EXIT_REASON_PREEMPTION_TIMER ) +return handle_preemption_timer (vcpu); +else if (exit_reason ==EXIT_REASON_PENDING_INTERRUPT ) +return handle_interrupt_window (vcpu); +else if (exit_reason ==EXIT_REASON_EXTERNAL_INTERRUPT ) +return handle_external_interrupt (vcpu); +else if (exit_reason ==EXIT_REASON_HLT ) +return kvm_emulate_halt (vcpu); +else if (exit_reason ==EXIT_REASON_EPT_MISCONFIG ) +return handle_ept_misconfig (vcpu); +#endif return kvm_vmx_exit_handlers[exit_reason](vcpu); -else { + }else {vcpu_unimpl (vcpu,"vmx: unexpected exit reason 0x%x\n" , exit_reason);dump_vmcs ();
Re: [PATCH 11/13] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers
Quoted 38 lines · Andrea Arcangeli
It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.
Of course
CONFIG_RETPOLINEalready forbids gcc to use indirect jumps while compiling allswitch()statements, howeverswitch()would still allow the compiler to bisect the case value. It's more efficient to prioritize the most frequent vmexits instead.The halt may be slow paths from the point of the guest, but not necessarily so from the point of the host if the host runs at full CPU capacity and no host CPU is ever left idle.
Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/vmx.c +11
-1@@ -5905,9 +5905,23 @@ int kvm_x86_handle_exit(struct kvm_vcpu *vcpu) }if (exit_reason < kvm_vmx_max_exit_handlers - && kvm_vmx_exit_handlers[exit_reason]) + && kvm_vmx_exit_handlers[exit_reason]) { +#ifdef CONFIG_RETPOLINE +if (exit_reason ==EXIT_REASON_MSR_WRITE ) +return kvm_emulate_wrmsr (vcpu); +else if (exit_reason ==EXIT_REASON_PREEMPTION_TIMER ) +return handle_preemption_timer (vcpu); +else if (exit_reason ==EXIT_REASON_PENDING_INTERRUPT ) +return handle_interrupt_window (vcpu); +else if (exit_reason ==EXIT_REASON_EXTERNAL_INTERRUPT ) +return handle_external_interrupt (vcpu); +else if (exit_reason ==EXIT_REASON_HLT )
arch/x86/kvm/vmx/vmx.c int kvm_x86_handle_exit(struct kvm_vcpu *vcpu) · line 5927
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +5
-1+return kvm_emulate_halt (vcpu); + else if (exit_reason == EXIT_REASON_EPT_MISCONFIG) + return handle_ept_misconfig(vcpu); +#endif return kvm_vmx_exit_handlers[exit_reason](vcpu); - else { + }else {vcpu_unimpl (vcpu,"vmx: unexpected exit reason 0x%x\n" , exit_reason);dump_vmcs ();
Queued, thanks.
Paolo
PATCH 12/13 KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers
In reply to Andrea ArcangeliIt's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.
After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a SVM host:
[..]
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get_update_offsets_now+70
hrtimer_interrupt+131
smp_apic_timer_interrupt+106
apic_timer_interrupt+15
start_sw_timer+359
restart_apic_timer+85
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 1940
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_r12+33
force_qs_rnp+217
rcu_gp_kthread+1270
kthread+268
ret_from_fork+34
]: 4644
@[]: 25095
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
lapic_next_event+28
clockevents_program_event+148
hrtimer_start_range_ns+528
start_sw_timer+356
restart_apic_timer+85
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 41474
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
clockevents_program_event+148
hrtimer_start_range_ns+528
start_sw_timer+356
restart_apic_timer+85
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 41474
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get+58
clockevents_program_event+84
hrtimer_start_range_ns+528
start_sw_timer+356
restart_apic_timer+85
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 41887
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
lapic_next_event+28
clockevents_program_event+148
hrtimer_try_to_cancel+168
hrtimer_cancel+21
kvm_set_lapic_tscdeadline_msr+43
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 42723
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
clockevents_program_event+148
hrtimer_try_to_cancel+168
hrtimer_cancel+21
kvm_set_lapic_tscdeadline_msr+43
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 42766
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get+58
clockevents_program_event+84
hrtimer_try_to_cancel+168
hrtimer_cancel+21
kvm_set_lapic_tscdeadline_msr+43
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 42848
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
ktime_get+58
start_sw_timer+279
restart_apic_timer+85
kvm_set_msr_common+1497
msr_interception+142
vcpu_enter_guest+684
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68]: 499845
@total: 1780243
SVM has no TSC based programmable preemption timer so it is invoking ktime_get() frequently.
--- arch/x86/kvm/svm.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/x86/kvm/svm.c +12 -0
@@ -4995,6 +4995,18 @@ int kvm_x86_handle_exit(struct kvm_vcpu *vcpu)return 0 ; } +#ifdef CONFIG_RETPOLINE +if (exit_code ==SVM_EXIT_MSR ) +return msr_interception (svm); +else if (exit_code ==SVM_EXIT_VINTR ) +return interrupt_window_interception (svm); +else if (exit_code ==SVM_EXIT_INTR ) +return intr_interception (svm); +else if (exit_code ==SVM_EXIT_HLT ) +return halt_interception (svm); +else if (exit_code ==SVM_EXIT_NPF ) +return npf_interception (svm); +#endif return svm_exit_handlers[exit_code](svm); }
Re: [PATCH 12/13] KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers
Quoted 199 lines · Andrea Arcangeli
It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.
After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a SVM host:
[..]
@[trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get_update_offsets_now+70 hrtimer_interrupt+131 smp_apic_timer_interrupt+106 apic_timer_interrupt+15 start_sw_timer+359 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 1940 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_r12+33 force_qs_rnp+217 rcu_gp_kthread+1270 kthread+268 ret_from_fork+34 ]: 4644 @[]: 25095 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 lapic_next_event+28 clockevents_program_event+148 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 41474 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 clockevents_program_event+148 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 41474 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 clockevents_program_event+84 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 41887 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 lapic_next_event+28 clockevents_program_event+148 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 42723 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 clockevents_program_event+148 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 42766 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 clockevents_program_event+84 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 42848 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 start_sw_timer+279 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68]: 499845
@total: 1780243
SVM has no TSC based programmable preemption timer so it is invoking
ktime_get()frequently.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/kvm/svm.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/x86/kvm/svm.c +5
-0@@ -4995,6 +4995,18 @@ int kvm_x86_handle_exit(struct kvm_vcpu *vcpu)return 0 ; } +#ifdef CONFIG_RETPOLINE +if (exit_code ==SVM_EXIT_MSR ) +return msr_interception (svm); +else if (exit_code ==SVM_EXIT_VINTR ) +return interrupt_window_interception (svm);
arch/x86/kvm/svm.c int kvm_x86_handle_exit(struct kvm_vcpu *vcpu) · line 5012
Quoted 10 lines
arch/x86/kvm/svm.c +7
-0+ else if (exit_code == SVM_EXIT_INTR) + return intr_interception(svm); + else if (exit_code == SVM_EXIT_HLT) + return halt_interception(svm); + else if (exit_code == SVM_EXIT_NPF) + return npf_interception(svm); +#endif return svm_exit_handlers[exit_code](svm); }
Queued, thanks (BTW, I still disagree about HLT exits but okay).
Paolo
PATCH 13/13 x86: retpolines: eliminate retpoline from msr event handlers
In reply to Andrea ArcangeliIt's enough to check the value and issue the direct call.
After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a VMX host:
[..]
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68
]: 267
@[]: 2256
@[
trace_retpoline+1
__trace_retpoline+30
__x86_indirect_thunk_rax+33
__kvm_wait_lapic_expire+284
vmx_vcpu_run.part.97+1091
vcpu_enter_guest+377
kvm_arch_vcpu_ioctl_run+261
kvm_vcpu_ioctl+559
do_vfs_ioctl+164
ksys_ioctl+96
__x64_sys_ioctl+22
do_syscall_64+89
entry_SYSCALL_64_after_hwframe+68]: 2390
@[]: 33410
@total: 315707
Note the highest hit above is __delay so probably not worth optimizing even if it would be more frequent than 2k hits per sec.
--- arch/x86/events/intel/core.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/x86/events/intel/core.c +11 -0
@@ -3323,8 +3323,19 @@ static int intel_pmu_hw_config(struct perf_event *event)return 0 ; } +#ifdef CONFIG_RETPOLINE +static struct perf_guest_switch_msr *core_guest_get_msrs (int *nr); +static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr); +#endif +struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr) { +#ifdef CONFIG_RETPOLINE +if (x86_pmu.guest_get_msrs == intel_guest_get_msrs) +return intel_guest_get_msrs (nr); +else if (x86_pmu.guest_get_msrs == core_guest_get_msrs) +return core_guest_get_msrs (nr); +#endif if (x86_pmu.guest_get_msrs)return x86_pmu.guest_get_msrs (nr); *nr =0 ;
Re: [PATCH 13/13] x86: retpolines: eliminate retpoline from msr event handlers
Quoted 56 lines · Andrea Arcangeli
It's enough to check the value and issue the direct call.
After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a VMX host:
[..]
@[trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 267 @[]: 2256 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 __kvm_wait_lapic_expire+284 vmx_vcpu_run.part.97+1091 vcpu_enter_guest+377 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68]: 2390
@[]: 33410@total: 315707
Note the highest hit above is
__delayso probably not worth optimizing even if it would be more frequent than 2k hits per sec.Signed-off-by Andrea Arcangeli <[email protected]>--- arch/x86/events/intel/core.c | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/x86/events/intel/core.c +5
-0@@ -3323,8 +3323,19 @@ static int intel_pmu_hw_config(struct perf_event *event)return 0 ; } +#ifdef CONFIG_RETPOLINE +static struct perf_guest_switch_msr *core_guest_get_msrs (int *nr); +static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr); +#endif +struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr)
arch/x86/events/intel/core.c static int intel_pmu_hw_config(struct perf_event *event) · line 3341
Quoted 10 lines
arch/x86/events/intel/core.c +6
-0{ +#ifdef CONFIG_RETPOLINE +if (x86_pmu.guest_get_msrs == intel_guest_get_msrs) +return intel_guest_get_msrs (nr); +else if (x86_pmu.guest_get_msrs == core_guest_get_msrs) +return core_guest_get_msrs (nr); +#endif if (x86_pmu.guest_get_msrs)return x86_pmu.guest_get_msrs (nr); *nr =0 ;
Queued, thanks.
Paolo