Skip to contentSkip to conversation

KVM monolithic v1

PATCH In review Andrea Arcangeli 4 participants 68 messages Sep 20, 2019 – Sep 25, 2019 1 message not in the archive

Subsystems (6)
x86 KVM perf events KVM x86 +2 more subsystems
lore
Share & tools
lore
#0 lore raw

cover KVM monolithic v1

Hello,

This patchset micro optimizes the vmexit to increase performance by dropping the kvm.ko kernel module.

All common KVM code gets linked twice into kvm-intel and kvm-amd with the only cons of using more disk space, but the pros of CPU (and RAM) runtime benefits.

This improves the vmexit performance by two digits percent on microbenchmarks with the spectre_v2 default mitigation on both VMX and SVM. With spectre_v2=off or with CPUs with IBRS_ALL in ARCH_CAPABILITIES this still improve performance but it's more of the order of 1%.

We'll still have to deal with CPUs without IBRS_ALL for a decade and reducing the vmexit latency is important to pass certain benchmarks with workloads that happen to trigger frequent vmexits without having to set spectre_v2=off in the host (which at least in theory would make the host kernel vulnerable from a spectre v2 attack from the guest, even through hyperthreading).

The first patch 1/17 should be splitted off from this series and it's intended to be merged separately, it's included here only to avoid any possible erroneous measurement if using kexec for testing, in turn if using kexec it's recommended to include it in the baseline measurements too.

A git clonable branch for quick testing is available here:

https://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git/log/?h=kvm-mono1

Thanks,
Andrea

Andrea Arcangeli (17):
x86: spec_ctrl: fix SPEC_CTRL initialization after kexec
KVM: monolithic: x86: convert the kvm_x86_ops methods to external
functions
KVM: monolithic: x86: handle the request_immediate_exit variation
KVM: monolithic: x86: convert the kvm_pmu_ops methods to external
functions
KVM: monolithic: x86: enable the kvm_x86_ops external functions
KVM: monolithic: x86: enable the kvm_pmu_ops external functions
KVM: monolithic: x86: adjust the section prefixes
KVM: monolithic: adjust the section prefixes in the KVM common code
KVM: monolithic: x86: remove kvm.ko
KVM: monolithic: x86: use the external functions instead of

    kvm_x86_ops

KVM: monolithic: x86: remove exports
KVM: monolithic: remove exports from KVM common code
KVM: monolithic: x86: drop the kvm_pmu_ops structure
KVM: monolithic: x86: inline more exit handlers in vmx.c
KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers
KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers
x86: retpolines: eliminate retpoline from msr event handlers

 arch/x86/events/intel/core.c     |  11 +
 arch/x86/include/asm/kvm_host.h  |  15 +-
 arch/x86/include/asm/kvm_ops.h   | 166 ++++++++
 arch/x86/include/asm/msr-index.h |   2 +
 arch/x86/kernel/cpu/bugs.c       |  20 +-
 arch/x86/kvm/Makefile            |   5 +-
 arch/x86/kvm/cpuid.c             |  27 +-
 arch/x86/kvm/hyperv.c            |   8 +-
 arch/x86/kvm/irq.c               |   4 -
 arch/x86/kvm/irq_comm.c          |   2 -
 arch/x86/kvm/kvm_cache_regs.h    |  10 +-
 arch/x86/kvm/lapic.c             |  44 +-
 arch/x86/kvm/mmu.c               |  50 +--
 arch/x86/kvm/mmu.h               |   4 +-
 arch/x86/kvm/mtrr.c              |   2 -
 arch/x86/kvm/pmu.c               |  27 +-
 arch/x86/kvm/pmu.h               |  21 +-
 arch/x86/kvm/pmu_amd.c           |  15 +-
 arch/x86/kvm/pmu_amd_ops.c       |  68 ++++
 arch/x86/kvm/pmu_ops.h           |  22 +
 arch/x86/kvm/svm.c               |  19 +-
 arch/x86/kvm/svm_ops.c           | 672 ++++++++++++++++++++++++++++++
 arch/x86/kvm/trace.h             |   4 +-
 arch/x86/kvm/vmx/pmu_intel.c     |  17 +-
 arch/x86/kvm/vmx/pmu_intel_ops.c |  68 ++++
 arch/x86/kvm/vmx/vmx.c           |  36 +-
 arch/x86/kvm/vmx/vmx_ops.c       | 675 +++++++++++++++++++++++++++++++
 arch/x86/kvm/x86.c               | 409 +++++++------------
 arch/x86/kvm/x86.h               |   2 +-
 virt/kvm/eventfd.c               |   1 -
 virt/kvm/kvm_main.c              |  71 +---
 31 files changed, 1982 insertions(+), 515 deletions(-)
 create mode 100644 arch/x86/include/asm/kvm_ops.h
 create mode 100644 arch/x86/kvm/pmu_amd_ops.c
 create mode 100644 arch/x86/kvm/pmu_ops.h
 create mode 100644 arch/x86/kvm/svm_ops.c
 create mode 100644 arch/x86/kvm/vmx/pmu_intel_ops.c
 create mode 100644 arch/x86/kvm/vmx/vmx_ops.c
#1 lore raw

PATCH 1/17 x86: spec_ctrl: fix SPEC_CTRL initialization after kexec

We can't assume the SPEC_CTRL msr is zero at boot because it could be left enabled by a previous kernel booted with spec_store_bypass_disable=on.

Without this fix a boot with spec_store_bypass_disable=on followed by a kexec boot with spec_store_bypass_disable=off would erroneously and unexpectedly leave bit 2 set in SPEC_CTRL.

---
 arch/x86/include/asm/msr-index.h |  2 ++
 arch/x86/kernel/cpu/bugs.c       | 20 +++++++++++++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)
arch/x86/include/asm/msr-index.h +2 -0
index 20ce682a2540..3ba95728a6fe 100644
@@ -47,6 +47,8 @@
 #define SPEC_CTRL_STIBP			BIT(SPEC_CTRL_STIBP_SHIFT)	/* STIBP mask */
 #define SPEC_CTRL_SSBD_SHIFT		2	   /* Speculative Store Bypass Disable bit */
 #define SPEC_CTRL_SSBD			BIT(SPEC_CTRL_SSBD_SHIFT)	/* Speculative Store Bypass Disable */
+#define SPEC_CTRL_ALL			(SPEC_CTRL_IBRS|SPEC_CTRL_STIBP| \
+					 SPEC_CTRL_SSBD) /* all known bits */
 
 #define MSR_IA32_PRED_CMD		0x00000049 /* Prediction Command */
 #define PRED_CMD_IBPB			BIT(0)	   /* Indirect Branch Prediction Barrier */
arch/x86/kernel/cpu/bugs.c +19 -1
index 91c2561b905f..e3922dcf252f 100644
@@ -92,8 +92,26 @@ void __init check_bugs(void)
 	 * have unknown values. AMD64_LS_CFG MSR is cached in the early AMD
 	 * init code as it is not enumerated and depends on the family.
 	 */
-	if (boot_cpu_has(X86_FEATURE_MSR_SPEC_CTRL))
+	if (boot_cpu_has(X86_FEATURE_MSR_SPEC_CTRL)) {
 		rdmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base);
+		/*
+		 * Clear the non reserved bits from x86_spec_ctrl_base
+		 * to fix kexec. Otherwise for example SSBD could be
+		 * left enabled despite booting with
+		 * spec_store_bypass_disable=off because SSBD would be
+		 * erroneously mistaken as a reserved bit set by the
+		 * BIOS when in fact it was set by a previous kernel
+		 * booted with spec_store_bypass_disable=on. Careful
+		 * however not to write SPEC_CTRL unnecessarily to
+		 * keep the virt MSR intercept enabled as long as
+		 * possible.
+		 */
+		if (x86_spec_ctrl_base & SPEC_CTRL_ALL) {
+			/* all known bits must not be set at boot, clear it */
+			x86_spec_ctrl_base &= ~SPEC_CTRL_ALL;
+			wrmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base);
+		}
+	}
 
 	/* Allow STIBP in MSR_SPEC_CTRL if supported */
 	if (boot_cpu_has(X86_FEATURE_STIBP))
maintainer
#2 lore raw

Re: [PATCH 01/17] x86: spec_ctrl: fix SPEC_CTRL initialization after kexec

Quoted 9 lines · Andrea Arcangeli

We can't assume the SPEC_CTRL msr is zero at boot because it could be left enabled by a previous kernel booted with spec_store_bypass_disable=on.

Without this fix a boot with spec_store_bypass_disable=on followed by a kexec boot with spec_store_bypass_disable=off would erroneously and unexpectedly leave bit 2 set in SPEC_CTRL.

Can you send this out separately, so that Thomas et al. can pick it up as a bug fix?

Thanks,

Paolo

#3 lore raw

Re: [PATCH 01/17] x86: spec_ctrl: fix SPEC_CTRL initialization after kexec

Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Andrea Arcangeli

We can't assume the SPEC_CTRL msr is zero at boot because it could be left enabled by a previous kernel booted with spec_store_bypass_disable=on.

Quoted 8 lines
Quoted 5 lines

Without this fix a boot with spec_store_bypass_disable=on followed by a kexec boot with spec_store_bypass_disable=off would erroneously and unexpectedly leave bit 2 set in SPEC_CTRL.

Can you send this out separately, so that Thomas et al. can pick it up as a bug fix?

Can all off the patches that are not directly related to the monolithic conversion be sent separately? AFAICT, patches 01, 03, 07, 08, 14, 15, 16 and 17 are not required or dependent on the conversion to a monolithic module. That's almost half the series...

#4 lore raw

Re: [PATCH 01/17] x86: spec_ctrl: fix SPEC_CTRL initialization after kexec

Hello,

Quoted 6 lines · Sean Christopherson
Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Andrea Arcangeli

We can't assume the SPEC_CTRL msr is zero at boot because it could be left enabled by a previous kernel booted with spec_store_bypass_disable=on.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Without this fix a boot with spec_store_bypass_disable=on followed by a kexec boot with spec_store_bypass_disable=off would erroneously and unexpectedly leave bit 2 set in SPEC_CTRL.

Can you send this out separately, so that Thomas et al. can pick it up as a bug fix?

As specified in the cover letter 1/17 was already intended to be merged separately. I just keep this included in case people had the idea of using kexec to benchmark this work, because I was bitten by that bug myself and it wasted a few days worth of benchmarks.

Quoted 4 lines

Can all off the patches that are not directly related to the monolithic conversion be sent separately? AFAICT, patches 01, 03, 07, 08, 14, 15, 16 and 17 are not required or dependent on the conversion to a monolithic module. That's almost half the series...

03 07 08 are directly related to the monolithic conversion as the subject of the patch clarifies. In fact I should try to reorder 7/8 in front to make things more bisectable under all config options.

Per subject of the patch, 14 is also an optimization that while not a strict requirement, is somewhat related to the monolithic conversion because in fact it may naturally disappear if I rename the vmx/svm functions directly.

15 16 17 don't have the monolithic tag in the subject of the patch and they're obviously unrelated to the monolithic conversion, but when I did the first research on this idea of dropping kvm.ko a couple of months ago, things didn't really work well until I got rid of those few last retpolines too. If felt as if the large retpoline regression wasn't linear with the number of retpolines executed for each vmexit, and that it was more linear with the percentage of vmexits that hit on any number of retpolines. So while they're not part of the monolithic conversion I assumed they're required to run any meaningful benchmark.

I can drop 15 16 17 from further submits of course, after clarifying benchmark should be only run on the v1 full set I posted earlier, or they wouldn't be meaningful.

Thanks,
Andrea

#5 lore raw

Re: [PATCH 01/17] x86: spec_ctrl: fix SPEC_CTRL initialization after kexec

Quoted 15 lines · Andrea Arcangeli

Per subject of the patch, 14 is also an optimization that while not a strict requirement, is somewhat related to the monolithic conversion because in fact it may naturally disappear if I rename the vmx/svm functions directly.

15 16 17 don't have the monolithic tag in the subject of the patch and they're obviously unrelated to the monolithic conversion, but when I did the first research on this idea of dropping kvm.ko a couple of months ago, things didn't really work well until I got rid of those few last retpolines too. If felt as if the large retpoline regression wasn't linear with the number of retpolines executed for each vmexit, and that it was more linear with the percentage of vmexits that hit on any number of retpolines. So while they're not part of the monolithic conversion I assumed they're required to run any meaningful benchmark.

Quoted 3 lines

I can drop 15 16 17 from further submits of course, after clarifying benchmark should be only run on the v1 full set I posted earlier, or they wouldn't be meaningful.

I like the patches, I'd just prefer that they be sent in a separate series so they can churn independently.

#6 lore raw

PATCH 2/17 KVM: monolithic: x86: convert the kvm_x86_ops methods to external functions

In reply to Andrea Arcangeli

This replaces all kvm_x86_ops pointer to functions with regular external functions that don't require indirect calls.

This is the first commit of a series that aim to replace the modular kvm.ko kernel module with a monolithic kvm-intel/kvm-amd model. This change has the only possible cons of wasting some disk space in /lib/modules/. The pros are that it saves CPUS and some minor RAM which are more scarse resources than disk space.

The pointer to function virtual template model cannot provide any runtime benefit because kvm-intel and kvm-amd can't be loaded at the same time.

In practice this optimization results in a double digit percent reduction in the vmexit latency with the default retpoline spectre v2 mitigation enabled in the host.

When the host is booted with spectre_v2=off this still results in a measurable improvement of the order of 1% depending on the architecture and workload. Supposedly userland workloads in guest making lots of use of the BTB will benefit more.

To reduce the rejecting parts while tracking upstream, this doesn't attempt to entirely remove the kvm_x86_ops structure yet, that is meant for a later cleanup. The pmu ops have been already cleaned up in this patchset because it was left completely unused right after the conversion from pointer to functions to external functions.

Further incremental minor optimizations that weren't possible before are now enabled by the monolithic model. For example it would be possible later to convert some of the small external methods to inline functions from the {svm,vmx}_ops.c file to kvm_ops.h. However that will require more Makefile tweaks.

This is a list of the most common retpolines executed in KVM on VMX under a guest workload triggering a high resolution timer SIGALRM flood before the monolithic KVM patchset is applied.

[..]
@[

    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    cancel_hv_timer.isra.46+44
    restart_apic_timer+295
    kvm_set_msr_common+1435
    vmx_set_msr+478
    handle_wrmsr+85
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 65382
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+1646
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 66164
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_read_l1_tsc+41
    __kvm_wait_lapic_expire+60
    vmx_vcpu_run.part.88+1091
    vcpu_enter_guest+423
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 66199
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+4958
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 66227
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    restart_apic_timer+99
    kvm_set_msr_common+1435
    vmx_set_msr+478
    handle_wrmsr+85
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 130619
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_read_l1_tsc+41
    vmx_set_hv_timer+81
    restart_apic_timer+99
    kvm_set_msr_common+1435
    vmx_set_msr+478
    handle_wrmsr+85
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 130665
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_skip_emulated_instruction+49
    handle_wrmsr+102
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 131020
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_skip_emulated_instruction+82
    handle_wrmsr+102
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 131025
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    handle_wrmsr+85
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 131043
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    skip_emulated_instruction+48
    kvm_skip_emulated_instruction+82
    handle_wrmsr+102
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 131046
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+4009
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 132405
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rcx+33
    vcpu_enter_guest+1689
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 197697
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vmx_vcpu_run.part.88+358
    vcpu_enter_guest+423
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 198736
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+575
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 198771
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+423
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 198793
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+486
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 198801
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+168
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 198848
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68

]: 397680

@total: 3816655

Here the same but on SVM:

[..]
@[

    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    clockevents_program_event+148
    hrtimer_start_range_ns+528
    start_sw_timer+356
    restart_apic_timer+111
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 36031
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    lapic_next_event+28
    clockevents_program_event+148
    hrtimer_start_range_ns+528
    start_sw_timer+356
    restart_apic_timer+111
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 36063
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get+58
    clockevents_program_event+84
    hrtimer_try_to_cancel+168
    hrtimer_cancel+21
    kvm_set_lapic_tscdeadline_msr+43
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 36134
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    lapic_next_event+28
    clockevents_program_event+148
    hrtimer_try_to_cancel+168
    hrtimer_cancel+21
    kvm_set_lapic_tscdeadline_msr+43
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 36146
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    clockevents_program_event+148
    hrtimer_try_to_cancel+168
    hrtimer_cancel+21
    kvm_set_lapic_tscdeadline_msr+43
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 36190
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get+58
    clockevents_program_event+84
    hrtimer_start_range_ns+528
    start_sw_timer+356
    restart_apic_timer+111
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 36281
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+1646
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 37752
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_read_l1_tsc+41
    __kvm_wait_lapic_expire+60
    svm_vcpu_run+1276
]: 37886
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+4958
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 37957
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_read_l1_tsc+41
    start_sw_timer+302
    restart_apic_timer+111
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 74358
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get+58
    start_sw_timer+279
    restart_apic_timer+111
    kvm_set_msr_common+1435
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 74558
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_skip_emulated_instruction+82
    msr_interception+356
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 74713
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_skip_emulated_instruction+49
    msr_interception+356
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 74757
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    msr_interception+138
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 74795
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    kvm_get_rflags+28
    svm_interrupt_allowed+50
    vcpu_enter_guest+4009
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 75647
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+4009
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 75812
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rcx+33
    vcpu_enter_guest+1689
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 112579
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+575
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 113371
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+423
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 113386
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+486
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 113414
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+168
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 113601
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    vcpu_enter_guest+772
    kvm_arch_vcpu_ioctl_run+263
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68

]: 227076

@total: 3829460

---
 arch/x86/include/asm/kvm_ops.h | 166 ++++++++
 arch/x86/kvm/svm_ops.c         | 672 +++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx_ops.c     | 672 +++++++++++++++++++++++++++++++++
 3 files changed, 1510 insertions(+)
 create mode 100644 arch/x86/include/asm/kvm_ops.h
 create mode 100644 arch/x86/kvm/svm_ops.c
 create mode 100644 arch/x86/kvm/vmx/vmx_ops.c
arch/x86/include/asm/kvm_ops.h new file +166 -0
new file mode 100644
index 000000000000..1ec5380c9b67
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASM_X86_KVM_OPS_H
+#define _ASM_X86_KVM_OPS_H
+
+extern __init int kvm_x86_ops_cpu_has_kvm_support(void);
+extern __init int kvm_x86_ops_disabled_by_bios(void);
+extern int kvm_x86_ops_hardware_enable(void);
+extern void kvm_x86_ops_hardware_disable(void);
+extern __init int kvm_x86_ops_check_processor_compatibility(void);
+extern __init int kvm_x86_ops_hardware_setup(void);
+extern void kvm_x86_ops_hardware_unsetup(void);
+extern bool kvm_x86_ops_cpu_has_accelerated_tpr(void);
+extern bool kvm_x86_ops_has_emulated_msr(int index);
+extern void kvm_x86_ops_cpuid_update(struct kvm_vcpu *vcpu);
+extern struct kvm *kvm_x86_ops_vm_alloc(void);
+extern void kvm_x86_ops_vm_free(struct kvm *kvm);
+extern int kvm_x86_ops_vm_init(struct kvm *kvm);
+extern void kvm_x86_ops_vm_destroy(struct kvm *kvm);
+extern struct kvm_vcpu *kvm_x86_ops_vcpu_create(struct kvm *kvm, unsigned id);
+extern void kvm_x86_ops_vcpu_free(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
+extern void kvm_x86_ops_prepare_guest_switch(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_vcpu_load(struct kvm_vcpu *vcpu, int cpu);
+extern void kvm_x86_ops_vcpu_put(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_update_bp_intercept(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr);
+extern int kvm_x86_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr);
+extern u64 kvm_x86_ops_get_segment_base(struct kvm_vcpu *vcpu, int seg);
+extern void kvm_x86_ops_get_segment(struct kvm_vcpu *vcpu,
+				    struct kvm_segment *var, int seg);
+extern int kvm_x86_ops_get_cpl(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_segment(struct kvm_vcpu *vcpu,
+				    struct kvm_segment *var, int seg);
+extern void kvm_x86_ops_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db,
+					 int *l);
+extern void kvm_x86_ops_decache_cr0_guest_bits(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_decache_cr3(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_decache_cr4_guest_bits(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0);
+extern void kvm_x86_ops_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3);
+extern int kvm_x86_ops_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4);
+extern void kvm_x86_ops_set_efer(struct kvm_vcpu *vcpu, u64 efer);
+extern void kvm_x86_ops_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
+extern void kvm_x86_ops_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
+extern void kvm_x86_ops_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
+extern void kvm_x86_ops_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
+extern u64 kvm_x86_ops_get_dr6(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_dr6(struct kvm_vcpu *vcpu, unsigned long value);
+extern void kvm_x86_ops_sync_dirty_debug_regs(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_dr7(struct kvm_vcpu *vcpu, unsigned long value);
+extern void kvm_x86_ops_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg);
+extern unsigned long kvm_x86_ops_get_rflags(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_rflags(struct kvm_vcpu *vcpu,
+				   unsigned long rflags);
+extern void kvm_x86_ops_tlb_flush(struct kvm_vcpu *vcpu, bool invalidate_gpa);
+extern int kvm_x86_ops_tlb_remote_flush(struct kvm *kvm);
+extern int kvm_x86_ops_tlb_remote_flush_with_range(struct kvm *kvm,
+						   struct kvm_tlb_range *range);
+extern void kvm_x86_ops_tlb_flush_gva(struct kvm_vcpu *vcpu, gva_t addr);
+extern void kvm_x86_ops_run(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_handle_exit(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_skip_emulated_instruction(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask);
+extern u32 kvm_x86_ops_get_interrupt_shadow(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_patch_hypercall(struct kvm_vcpu *vcpu,
+					unsigned char *hypercall_addr);
+extern void kvm_x86_ops_set_irq(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_nmi(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_queue_exception(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_cancel_injection(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_interrupt_allowed(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_nmi_allowed(struct kvm_vcpu *vcpu);
+extern bool kvm_x86_ops_get_nmi_mask(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked);
+extern void kvm_x86_ops_enable_nmi_window(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_enable_irq_window(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr,
+					     int irr);
+extern bool kvm_x86_ops_get_enable_apicv(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr);
+extern void kvm_x86_ops_hwapic_isr_update(struct kvm_vcpu *vcpu, int isr);
+extern bool kvm_x86_ops_guest_apic_has_interrupt(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_load_eoi_exitmap(struct kvm_vcpu *vcpu,
+					 u64 *eoi_exit_bitmap);
+extern void kvm_x86_ops_set_virtual_apic_mode(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_set_apic_access_page_addr(struct kvm_vcpu *vcpu,
+						  hpa_t hpa);
+extern void kvm_x86_ops_deliver_posted_interrupt(struct kvm_vcpu *vcpu,
+						 int vector);
+extern int kvm_x86_ops_sync_pir_to_irr(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_set_tss_addr(struct kvm *kvm, unsigned int addr);
+extern int kvm_x86_ops_set_identity_map_addr(struct kvm *kvm, u64 ident_addr);
+extern int kvm_x86_ops_get_tdp_level(struct kvm_vcpu *vcpu);
+extern u64 kvm_x86_ops_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn,
+				   bool is_mmio);
+extern int kvm_x86_ops_get_lpage_level(void);
+extern bool kvm_x86_ops_rdtscp_supported(void);
+extern bool kvm_x86_ops_invpcid_supported(void);
+extern void kvm_x86_ops_set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long cr3);
+extern void kvm_x86_ops_set_supported_cpuid(u32 func,
+					    struct kvm_cpuid_entry2 *entry);
+extern bool kvm_x86_ops_has_wbinvd_exit(void);
+extern u64 kvm_x86_ops_read_l1_tsc_offset(struct kvm_vcpu *vcpu);
+extern u64 kvm_x86_ops_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset);
+extern void kvm_x86_ops_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1,
+				      u64 *info2);
+extern int kvm_x86_ops_check_intercept(struct kvm_vcpu *vcpu,
+				       struct x86_instruction_info *info,
+				       enum x86_intercept_stage stage);
+extern void kvm_x86_ops_handle_exit_irqoff(struct kvm_vcpu *vcpu);
+extern bool kvm_x86_ops_mpx_supported(void);
+extern bool kvm_x86_ops_xsaves_supported(void);
+extern bool kvm_x86_ops_umip_emulated(void);
+extern bool kvm_x86_ops_pt_supported(void);
+extern int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu,
+					   bool external_intr);
+extern void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu);
+extern void kvm_x86_ops_slot_enable_log_dirty(struct kvm *kvm,
+					      struct kvm_memory_slot *slot);
+extern void kvm_x86_ops_slot_disable_log_dirty(struct kvm *kvm,
+					       struct kvm_memory_slot *slot);
+extern void kvm_x86_ops_flush_log_dirty(struct kvm *kvm);
+extern void kvm_x86_ops_enable_log_dirty_pt_masked(struct kvm *kvm,
+						   struct kvm_memory_slot *slot,
+						   gfn_t offset,
+						   unsigned long mask);
+extern int kvm_x86_ops_write_log_dirty(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_pre_block(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_post_block(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_vcpu_blocking(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_vcpu_unblocking(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
+				      uint32_t guest_irq, bool set);
+extern void kvm_x86_ops_apicv_post_state_restore(struct kvm_vcpu *vcpu);
+extern bool kvm_x86_ops_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_set_hv_timer(struct kvm_vcpu *vcpu,
+				    u64 guest_deadline_tsc, bool *expired);
+extern void kvm_x86_ops_cancel_hv_timer(struct kvm_vcpu *vcpu);
+extern void kvm_x86_ops_setup_mce(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_get_nested_state(struct kvm_vcpu *vcpu,
+					struct kvm_nested_state __user *user_kvm_nested_state,
+					unsigned user_data_size);
+extern int kvm_x86_ops_set_nested_state(struct kvm_vcpu *vcpu,
+					struct kvm_nested_state __user *user_kvm_nested_state,
+					struct kvm_nested_state *kvm_state);
+extern void kvm_x86_ops_get_vmcs12_pages(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_smi_allowed(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate);
+extern int kvm_x86_ops_pre_leave_smm(struct kvm_vcpu *vcpu,
+				     const char *smstate);
+extern int kvm_x86_ops_enable_smi_window(struct kvm_vcpu *vcpu);
+extern int kvm_x86_ops_mem_enc_op(struct kvm *kvm, void __user *argp);
+extern int kvm_x86_ops_mem_enc_reg_region(struct kvm *kvm,
+					  struct kvm_enc_region *argp);
+extern int kvm_x86_ops_mem_enc_unreg_region(struct kvm *kvm,
+					    struct kvm_enc_region *argp);
+extern int kvm_x86_ops_get_msr_feature(struct kvm_msr_entry *entry);
+extern int kvm_x86_ops_nested_enable_evmcs(struct kvm_vcpu *vcpu,
+					   uint16_t *vmcs_version);
+extern uint16_t kvm_x86_ops_nested_get_evmcs_version(struct kvm_vcpu *vcpu);
+extern bool kvm_x86_ops_need_emulation_on_page_fault(struct kvm_vcpu *vcpu);
+extern bool kvm_x86_ops_apic_init_signal_blocked(struct kvm_vcpu *vcpu);
+
+#endif /* _ASM_X86_KVM_OPS_H */
arch/x86/kvm/svm_ops.c new file +672 -0

672 lines not on this page · raw message

arch/x86/kvm/vmx/vmx_ops.c new file +672 -0

672 lines not on this page · raw message

maintainer
#7 lore raw

Re: [PATCH 02/17] KVM: monolithic: x86: convert the kvm_x86_ops methods to external functions

Quoted 7 lines · Andrea Arcangeli
arch/x86/kvm/svm_ops.c new file +1 -0
new file mode 100644
index 000000000000..2aaabda92179
@@ -0,0 +1,672 @@
+// SPDX-License-Identifier: GPL-2.0-only

arch/x86/kvm/svm_ops.c line 11

Quoted 10 lines
arch/x86/kvm/svm_ops.c new file +10 -0
+/*
+ *  arch/x86/kvm/svm_ops.c
+ *
+ *  Copyright 2019 Red Hat, Inc.
+ */
+
+int kvm_x86_ops_cpu_has_kvm_support(void)
+{
+	return has_svm();
+}

Can you just rename all the functions in vmx/ and svm.c, instead of adding forwarders?

Thanks,

Paolo

Quoted 1,339 lines
arch/x86/kvm/svm_ops.c new file +660 -0
+int kvm_x86_ops_disabled_by_bios(void)
+{
+	return is_disabled();
+}
+
+int kvm_x86_ops_hardware_enable(void)
+{
+	return svm_hardware_enable();
+}
+
+void kvm_x86_ops_hardware_disable(void)
+{
+	svm_hardware_disable();
+}
+
+__init int kvm_x86_ops_check_processor_compatibility(void)
+{
+	return svm_check_processor_compat();
+}
+
+__init int kvm_x86_ops_hardware_setup(void)
+{
+	return svm_hardware_setup();
+}
+
+void kvm_x86_ops_hardware_unsetup(void)
+{
+	svm_hardware_unsetup();
+}
+
+bool kvm_x86_ops_cpu_has_accelerated_tpr(void)
+{
+	return svm_cpu_has_accelerated_tpr();
+}
+
+bool kvm_x86_ops_has_emulated_msr(int index)
+{
+	return svm_has_emulated_msr(index);
+}
+
+void kvm_x86_ops_cpuid_update(struct kvm_vcpu *vcpu)
+{
+	svm_cpuid_update(vcpu);
+}
+
+struct kvm *kvm_x86_ops_vm_alloc(void)
+{
+	return svm_vm_alloc();
+}
+
+void kvm_x86_ops_vm_free(struct kvm *kvm)
+{
+	svm_vm_free(kvm);
+}
+
+int kvm_x86_ops_vm_init(struct kvm *kvm)
+{
+	return avic_vm_init(kvm);
+}
+
+void kvm_x86_ops_vm_destroy(struct kvm *kvm)
+{
+	svm_vm_destroy(kvm);
+}
+
+struct kvm_vcpu *kvm_x86_ops_vcpu_create(struct kvm *kvm, unsigned id)
+{
+	return svm_create_vcpu(kvm, id);
+}
+
+void kvm_x86_ops_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	svm_free_vcpu(vcpu);
+}
+
+void kvm_x86_ops_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+	svm_vcpu_reset(vcpu, init_event);
+}
+
+void kvm_x86_ops_prepare_guest_switch(struct kvm_vcpu *vcpu)
+{
+	svm_prepare_guest_switch(vcpu);
+}
+
+void kvm_x86_ops_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	svm_vcpu_load(vcpu, cpu);
+}
+
+void kvm_x86_ops_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	svm_vcpu_put(vcpu);
+}
+
+void kvm_x86_ops_update_bp_intercept(struct kvm_vcpu *vcpu)
+{
+	update_bp_intercept(vcpu);
+}
+
+int kvm_x86_ops_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
+{
+	return svm_get_msr(vcpu, msr);
+}
+
+int kvm_x86_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
+{
+	return svm_set_msr(vcpu, msr);
+}
+
+u64 kvm_x86_ops_get_segment_base(struct kvm_vcpu *vcpu, int seg)
+{
+	return svm_get_segment_base(vcpu, seg);
+}
+
+void kvm_x86_ops_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var,
+			     int seg)
+{
+	svm_get_segment(vcpu, var, seg);
+}
+
+int kvm_x86_ops_get_cpl(struct kvm_vcpu *vcpu)
+{
+	return svm_get_cpl(vcpu);
+}
+
+void kvm_x86_ops_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var,
+			     int seg)
+{
+	svm_set_segment(vcpu, var, seg);
+}
+
+void kvm_x86_ops_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l)
+{
+	kvm_get_cs_db_l_bits(vcpu, db, l);
+}
+
+void kvm_x86_ops_decache_cr0_guest_bits(struct kvm_vcpu *vcpu)
+{
+	svm_decache_cr0_guest_bits(vcpu);
+}
+
+void kvm_x86_ops_decache_cr3(struct kvm_vcpu *vcpu)
+{
+	svm_decache_cr3(vcpu);
+}
+
+void kvm_x86_ops_decache_cr4_guest_bits(struct kvm_vcpu *vcpu)
+{
+	svm_decache_cr4_guest_bits(vcpu);
+}
+
+void kvm_x86_ops_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
+{
+	svm_set_cr0(vcpu, cr0);
+}
+
+void kvm_x86_ops_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
+{
+	svm_set_cr3(vcpu, cr3);
+}
+
+int kvm_x86_ops_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
+{
+	return svm_set_cr4(vcpu, cr4);
+}
+
+void kvm_x86_ops_set_efer(struct kvm_vcpu *vcpu, u64 efer)
+{
+	svm_set_efer(vcpu, efer);
+}
+
+void kvm_x86_ops_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	svm_get_idt(vcpu, dt);
+}
+
+void kvm_x86_ops_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	svm_set_idt(vcpu, dt);
+}
+
+void kvm_x86_ops_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	svm_get_gdt(vcpu, dt);
+}
+
+void kvm_x86_ops_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	svm_set_gdt(vcpu, dt);
+}
+
+u64 kvm_x86_ops_get_dr6(struct kvm_vcpu *vcpu)
+{
+	return svm_get_dr6(vcpu);
+}
+
+void kvm_x86_ops_set_dr6(struct kvm_vcpu *vcpu, unsigned long value)
+{
+	svm_set_dr6(vcpu, value);
+}
+
+void kvm_x86_ops_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)
+{
+	svm_sync_dirty_debug_regs(vcpu);
+}
+
+void kvm_x86_ops_set_dr7(struct kvm_vcpu *vcpu, unsigned long value)
+{
+	svm_set_dr7(vcpu, value);
+}
+
+void kvm_x86_ops_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
+{
+	svm_cache_reg(vcpu, reg);
+}
+
+unsigned long kvm_x86_ops_get_rflags(struct kvm_vcpu *vcpu)
+{
+	return svm_get_rflags(vcpu);
+}
+
+void kvm_x86_ops_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
+{
+	svm_set_rflags(vcpu, rflags);
+}
+
+void kvm_x86_ops_tlb_flush(struct kvm_vcpu *vcpu, bool invalidate_gpa)
+{
+	svm_flush_tlb(vcpu, invalidate_gpa);
+}
+
+int kvm_x86_ops_tlb_remote_flush(struct kvm *kvm)
+{
+	return kvm_x86_ops->tlb_remote_flush(kvm);
+}
+
+int kvm_x86_ops_tlb_remote_flush_with_range(struct kvm *kvm,
+					    struct kvm_tlb_range *range)
+{
+	return kvm_x86_ops->tlb_remote_flush_with_range(kvm, range);
+}
+
+void kvm_x86_ops_tlb_flush_gva(struct kvm_vcpu *vcpu, gva_t addr)
+{
+	svm_flush_tlb_gva(vcpu, addr);
+}
+
+void kvm_x86_ops_run(struct kvm_vcpu *vcpu)
+{
+	svm_vcpu_run(vcpu);
+}
+
+int kvm_x86_ops_handle_exit(struct kvm_vcpu *vcpu)
+{
+	return handle_exit(vcpu);
+}
+
+int kvm_x86_ops_skip_emulated_instruction(struct kvm_vcpu *vcpu)
+{
+	return skip_emulated_instruction(vcpu);
+}
+
+void kvm_x86_ops_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask)
+{
+	svm_set_interrupt_shadow(vcpu, mask);
+}
+
+u32 kvm_x86_ops_get_interrupt_shadow(struct kvm_vcpu *vcpu)
+{
+	return svm_get_interrupt_shadow(vcpu);
+}
+
+void kvm_x86_ops_patch_hypercall(struct kvm_vcpu *vcpu,
+				 unsigned char *hypercall_addr)
+{
+	svm_patch_hypercall(vcpu, hypercall_addr);
+}
+
+void kvm_x86_ops_set_irq(struct kvm_vcpu *vcpu)
+{
+	svm_set_irq(vcpu);
+}
+
+void kvm_x86_ops_set_nmi(struct kvm_vcpu *vcpu)
+{
+	svm_inject_nmi(vcpu);
+}
+
+void kvm_x86_ops_queue_exception(struct kvm_vcpu *vcpu)
+{
+	svm_queue_exception(vcpu);
+}
+
+void kvm_x86_ops_cancel_injection(struct kvm_vcpu *vcpu)
+{
+	svm_cancel_injection(vcpu);
+}
+
+int kvm_x86_ops_interrupt_allowed(struct kvm_vcpu *vcpu)
+{
+	return svm_interrupt_allowed(vcpu);
+}
+
+int kvm_x86_ops_nmi_allowed(struct kvm_vcpu *vcpu)
+{
+	return svm_nmi_allowed(vcpu);
+}
+
+bool kvm_x86_ops_get_nmi_mask(struct kvm_vcpu *vcpu)
+{
+	return svm_get_nmi_mask(vcpu);
+}
+
+void kvm_x86_ops_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked)
+{
+	svm_set_nmi_mask(vcpu, masked);
+}
+
+void kvm_x86_ops_enable_nmi_window(struct kvm_vcpu *vcpu)
+{
+	enable_nmi_window(vcpu);
+}
+
+void kvm_x86_ops_enable_irq_window(struct kvm_vcpu *vcpu)
+{
+	enable_irq_window(vcpu);
+}
+
+void kvm_x86_ops_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr)
+{
+	update_cr8_intercept(vcpu, tpr, irr);
+}
+
+bool kvm_x86_ops_get_enable_apicv(struct kvm_vcpu *vcpu)
+{
+	return svm_get_enable_apicv(vcpu);
+}
+
+void kvm_x86_ops_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
+{
+	svm_refresh_apicv_exec_ctrl(vcpu);
+}
+
+void kvm_x86_ops_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr)
+{
+	svm_hwapic_irr_update(vcpu, max_irr);
+}
+
+void kvm_x86_ops_hwapic_isr_update(struct kvm_vcpu *vcpu, int isr)
+{
+	svm_hwapic_isr_update(vcpu, isr);
+}
+
+bool kvm_x86_ops_guest_apic_has_interrupt(struct kvm_vcpu *vcpu)
+{
+	return kvm_x86_ops->guest_apic_has_interrupt(vcpu);
+}
+
+void kvm_x86_ops_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
+{
+	svm_load_eoi_exitmap(vcpu, eoi_exit_bitmap);
+}
+
+void kvm_x86_ops_set_virtual_apic_mode(struct kvm_vcpu *vcpu)
+{
+	svm_set_virtual_apic_mode(vcpu);
+}
+
+void kvm_x86_ops_set_apic_access_page_addr(struct kvm_vcpu *vcpu, hpa_t hpa)
+{
+	kvm_x86_ops->set_apic_access_page_addr(vcpu, hpa);
+}
+
+void kvm_x86_ops_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)
+{
+	svm_deliver_avic_intr(vcpu, vector);
+}
+
+int kvm_x86_ops_sync_pir_to_irr(struct kvm_vcpu *vcpu)
+{
+	return kvm_lapic_find_highest_irr(vcpu);
+}
+
+int kvm_x86_ops_set_tss_addr(struct kvm *kvm, unsigned int addr)
+{
+	return svm_set_tss_addr(kvm, addr);
+}
+
+int kvm_x86_ops_set_identity_map_addr(struct kvm *kvm, u64 ident_addr)
+{
+	return svm_set_identity_map_addr(kvm, ident_addr);
+}
+
+int kvm_x86_ops_get_tdp_level(struct kvm_vcpu *vcpu)
+{
+	return get_npt_level(vcpu);
+}
+
+u64 kvm_x86_ops_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
+{
+	return svm_get_mt_mask(vcpu, gfn, is_mmio);
+}
+
+int kvm_x86_ops_get_lpage_level(void)
+{
+	return svm_get_lpage_level();
+}
+
+bool kvm_x86_ops_rdtscp_supported(void)
+{
+	return svm_rdtscp_supported();
+}
+
+bool kvm_x86_ops_invpcid_supported(void)
+{
+	return svm_invpcid_supported();
+}
+
+void kvm_x86_ops_set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
+{
+	set_tdp_cr3(vcpu, cr3);
+}
+
+void kvm_x86_ops_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry)
+{
+	svm_set_supported_cpuid(func, entry);
+}
+
+bool kvm_x86_ops_has_wbinvd_exit(void)
+{
+	return svm_has_wbinvd_exit();
+}
+
+u64 kvm_x86_ops_read_l1_tsc_offset(struct kvm_vcpu *vcpu)
+{
+	return svm_read_l1_tsc_offset(vcpu);
+}
+
+u64 kvm_x86_ops_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset)
+{
+	return svm_write_l1_tsc_offset(vcpu, offset);
+}
+
+void kvm_x86_ops_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1, u64 *info2)
+{
+	svm_get_exit_info(vcpu, info1, info2);
+}
+
+int kvm_x86_ops_check_intercept(struct kvm_vcpu *vcpu,
+				struct x86_instruction_info *info,
+				enum x86_intercept_stage stage)
+{
+	return svm_check_intercept(vcpu, info, stage);
+}
+
+void kvm_x86_ops_handle_exit_irqoff(struct kvm_vcpu *vcpu)
+{
+	svm_handle_exit_irqoff(vcpu);
+}
+
+bool kvm_x86_ops_mpx_supported(void)
+{
+	return svm_mpx_supported();
+}
+
+bool kvm_x86_ops_xsaves_supported(void)
+{
+	return svm_xsaves_supported();
+}
+
+bool kvm_x86_ops_umip_emulated(void)
+{
+	return svm_umip_emulated();
+}
+
+bool kvm_x86_ops_pt_supported(void)
+{
+	return svm_pt_supported();
+}
+
+int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr)
+{
+	return kvm_x86_ops->check_nested_events(vcpu, external_intr);
+}
+
+void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu)
+{
+	__kvm_request_immediate_exit(vcpu);
+}
+
+void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu)
+{
+	svm_sched_in(kvm, cpu);
+}
+
+void kvm_x86_ops_slot_enable_log_dirty(struct kvm *kvm,
+				       struct kvm_memory_slot *slot)
+{
+	kvm_x86_ops->slot_enable_log_dirty(kvm, slot);
+}
+
+void kvm_x86_ops_slot_disable_log_dirty(struct kvm *kvm,
+					struct kvm_memory_slot *slot)
+{
+	kvm_x86_ops->slot_disable_log_dirty(kvm, slot);
+}
+
+void kvm_x86_ops_flush_log_dirty(struct kvm *kvm)
+{
+	kvm_x86_ops->flush_log_dirty(kvm);
+}
+
+void kvm_x86_ops_enable_log_dirty_pt_masked(struct kvm *kvm,
+					    struct kvm_memory_slot *slot,
+					    gfn_t offset, unsigned long mask)
+{
+	kvm_x86_ops->enable_log_dirty_pt_masked(kvm, slot, offset, mask);
+}
+
+int kvm_x86_ops_write_log_dirty(struct kvm_vcpu *vcpu)
+{
+	return kvm_x86_ops->write_log_dirty(vcpu);
+}
+
+int kvm_x86_ops_pre_block(struct kvm_vcpu *vcpu)
+{
+	return kvm_x86_ops->pre_block(vcpu);
+}
+
+void kvm_x86_ops_post_block(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->post_block(vcpu);
+}
+
+void kvm_x86_ops_vcpu_blocking(struct kvm_vcpu *vcpu)
+{
+	svm_vcpu_blocking(vcpu);
+}
+
+void kvm_x86_ops_vcpu_unblocking(struct kvm_vcpu *vcpu)
+{
+	svm_vcpu_unblocking(vcpu);
+}
+
+int kvm_x86_ops_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
+			       uint32_t guest_irq, bool set)
+{
+	return svm_update_pi_irte(kvm, host_irq, guest_irq, set);
+}
+
+void kvm_x86_ops_apicv_post_state_restore(struct kvm_vcpu *vcpu)
+{
+	avic_post_state_restore(vcpu);
+}
+
+bool kvm_x86_ops_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu)
+{
+	return svm_dy_apicv_has_pending_interrupt(vcpu);
+}
+
+int kvm_x86_ops_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc,
+			     bool *expired)
+{
+	return kvm_x86_ops->set_hv_timer(vcpu, guest_deadline_tsc, expired);
+}
+
+void kvm_x86_ops_cancel_hv_timer(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->cancel_hv_timer(vcpu);
+}
+
+void kvm_x86_ops_setup_mce(struct kvm_vcpu *vcpu)
+{
+	svm_setup_mce(vcpu);
+}
+
+int kvm_x86_ops_get_nested_state(struct kvm_vcpu *vcpu,
+				 struct kvm_nested_state __user *user_kvm_nested_state,
+				 unsigned user_data_size)
+{
+	return kvm_x86_ops->get_nested_state(vcpu, user_kvm_nested_state,
+					     user_data_size);
+}
+
+int kvm_x86_ops_set_nested_state(struct kvm_vcpu *vcpu,
+				 struct kvm_nested_state __user *user_kvm_nested_state,
+				 struct kvm_nested_state *kvm_state)
+{
+	return kvm_x86_ops->set_nested_state(vcpu, user_kvm_nested_state,
+					     kvm_state);
+}
+
+void kvm_x86_ops_get_vmcs12_pages(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->get_vmcs12_pages(vcpu);
+}
+
+int kvm_x86_ops_smi_allowed(struct kvm_vcpu *vcpu)
+{
+	return svm_smi_allowed(vcpu);
+}
+
+int kvm_x86_ops_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate)
+{
+	return svm_pre_enter_smm(vcpu, smstate);
+}
+
+int kvm_x86_ops_pre_leave_smm(struct kvm_vcpu *vcpu, const char *smstate)
+{
+	return svm_pre_leave_smm(vcpu, smstate);
+}
+
+int kvm_x86_ops_enable_smi_window(struct kvm_vcpu *vcpu)
+{
+	return enable_smi_window(vcpu);
+}
+
+int kvm_x86_ops_mem_enc_op(struct kvm *kvm, void __user *argp)
+{
+	return svm_mem_enc_op(kvm, argp);
+}
+
+int kvm_x86_ops_mem_enc_reg_region(struct kvm *kvm,
+				   struct kvm_enc_region *argp)
+{
+	return svm_register_enc_region(kvm, argp);
+}
+
+int kvm_x86_ops_mem_enc_unreg_region(struct kvm *kvm,
+				     struct kvm_enc_region *argp)
+{
+	return svm_unregister_enc_region(kvm, argp);
+}
+
+int kvm_x86_ops_get_msr_feature(struct kvm_msr_entry *entry)
+{
+	return svm_get_msr_feature(entry);
+}
+
+int kvm_x86_ops_nested_enable_evmcs(struct kvm_vcpu *vcpu,
+				    uint16_t *vmcs_version)
+{
+	return nested_enable_evmcs(vcpu, vmcs_version);
+}
+
+uint16_t kvm_x86_ops_nested_get_evmcs_version(struct kvm_vcpu *vcpu)
+{
+	return kvm_x86_ops->nested_get_evmcs_version(vcpu);
+}
+
+bool kvm_x86_ops_need_emulation_on_page_fault(struct kvm_vcpu *vcpu)
+{
+	return svm_need_emulation_on_page_fault(vcpu);
+}
+
+bool kvm_x86_ops_apic_init_signal_blocked(struct kvm_vcpu *vcpu)
+{
+	return svm_apic_init_signal_blocked(vcpu);
+}
arch/x86/kvm/vmx/vmx_ops.c new file +672 -0
new file mode 100644
index 000000000000..cdcad73935d9
@@ -0,0 +1,672 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  arch/x86/kvm/vmx/vmx_ops.c
+ *
+ *  Copyright 2019 Red Hat, Inc.
+ */
+
+__init int kvm_x86_ops_cpu_has_kvm_support(void)
+{
+	return cpu_has_kvm_support();
+}
+
+__init int kvm_x86_ops_disabled_by_bios(void)
+{
+	return vmx_disabled_by_bios();
+}
+
+int kvm_x86_ops_hardware_enable(void)
+{
+	return hardware_enable();
+}
+
+void kvm_x86_ops_hardware_disable(void)
+{
+	hardware_disable();
+}
+
+__init int kvm_x86_ops_check_processor_compatibility(void)
+{
+	return vmx_check_processor_compat();
+}
+
+__init int kvm_x86_ops_hardware_setup(void)
+{
+	return hardware_setup();
+}
+
+void kvm_x86_ops_hardware_unsetup(void)
+{
+	hardware_unsetup();
+}
+
+bool kvm_x86_ops_cpu_has_accelerated_tpr(void)
+{
+	return report_flexpriority();
+}
+
+bool kvm_x86_ops_has_emulated_msr(int index)
+{
+	return vmx_has_emulated_msr(index);
+}
+
+void kvm_x86_ops_cpuid_update(struct kvm_vcpu *vcpu)
+{
+	vmx_cpuid_update(vcpu);
+}
+
+struct kvm *kvm_x86_ops_vm_alloc(void)
+{
+	return vmx_vm_alloc();
+}
+
+void kvm_x86_ops_vm_free(struct kvm *kvm)
+{
+	vmx_vm_free(kvm);
+}
+
+int kvm_x86_ops_vm_init(struct kvm *kvm)
+{
+	return vmx_vm_init(kvm);
+}
+
+void kvm_x86_ops_vm_destroy(struct kvm *kvm)
+{
+	kvm_x86_ops->vm_destroy(kvm);
+}
+
+struct kvm_vcpu *kvm_x86_ops_vcpu_create(struct kvm *kvm, unsigned id)
+{
+	return vmx_create_vcpu(kvm, id);
+}
+
+void kvm_x86_ops_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	vmx_free_vcpu(vcpu);
+}
+
+void kvm_x86_ops_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+	vmx_vcpu_reset(vcpu, init_event);
+}
+
+void kvm_x86_ops_prepare_guest_switch(struct kvm_vcpu *vcpu)
+{
+	vmx_prepare_switch_to_guest(vcpu);
+}
+
+void kvm_x86_ops_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	vmx_vcpu_load(vcpu, cpu);
+}
+
+void kvm_x86_ops_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	vmx_vcpu_put(vcpu);
+}
+
+void kvm_x86_ops_update_bp_intercept(struct kvm_vcpu *vcpu)
+{
+	update_exception_bitmap(vcpu);
+}
+
+int kvm_x86_ops_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
+{
+	return vmx_get_msr(vcpu, msr);
+}
+
+int kvm_x86_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
+{
+	return vmx_set_msr(vcpu, msr);
+}
+
+u64 kvm_x86_ops_get_segment_base(struct kvm_vcpu *vcpu, int seg)
+{
+	return vmx_get_segment_base(vcpu, seg);
+}
+
+void kvm_x86_ops_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var,
+			     int seg)
+{
+	vmx_get_segment(vcpu, var, seg);
+}
+
+int kvm_x86_ops_get_cpl(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_cpl(vcpu);
+}
+
+void kvm_x86_ops_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var,
+			     int seg)
+{
+	vmx_set_segment(vcpu, var, seg);
+}
+
+void kvm_x86_ops_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l)
+{
+	vmx_get_cs_db_l_bits(vcpu, db, l);
+}
+
+void kvm_x86_ops_decache_cr0_guest_bits(struct kvm_vcpu *vcpu)
+{
+	vmx_decache_cr0_guest_bits(vcpu);
+}
+
+void kvm_x86_ops_decache_cr3(struct kvm_vcpu *vcpu)
+{
+	vmx_decache_cr3(vcpu);
+}
+
+void kvm_x86_ops_decache_cr4_guest_bits(struct kvm_vcpu *vcpu)
+{
+	vmx_decache_cr4_guest_bits(vcpu);
+}
+
+void kvm_x86_ops_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
+{
+	vmx_set_cr0(vcpu, cr0);
+}
+
+void kvm_x86_ops_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
+{
+	vmx_set_cr3(vcpu, cr3);
+}
+
+int kvm_x86_ops_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
+{
+	return vmx_set_cr4(vcpu, cr4);
+}
+
+void kvm_x86_ops_set_efer(struct kvm_vcpu *vcpu, u64 efer)
+{
+	vmx_set_efer(vcpu, efer);
+}
+
+void kvm_x86_ops_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	vmx_get_idt(vcpu, dt);
+}
+
+void kvm_x86_ops_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	vmx_set_idt(vcpu, dt);
+}
+
+void kvm_x86_ops_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	vmx_get_gdt(vcpu, dt);
+}
+
+void kvm_x86_ops_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
+{
+	vmx_set_gdt(vcpu, dt);
+}
+
+u64 kvm_x86_ops_get_dr6(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_dr6(vcpu);
+}
+
+void kvm_x86_ops_set_dr6(struct kvm_vcpu *vcpu, unsigned long value)
+{
+	vmx_set_dr6(vcpu, value);
+}
+
+void kvm_x86_ops_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)
+{
+	vmx_sync_dirty_debug_regs(vcpu);
+}
+
+void kvm_x86_ops_set_dr7(struct kvm_vcpu *vcpu, unsigned long value)
+{
+	vmx_set_dr7(vcpu, value);
+}
+
+void kvm_x86_ops_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
+{
+	vmx_cache_reg(vcpu, reg);
+}
+
+unsigned long kvm_x86_ops_get_rflags(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_rflags(vcpu);
+}
+
+void kvm_x86_ops_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
+{
+	vmx_set_rflags(vcpu, rflags);
+}
+
+void kvm_x86_ops_tlb_flush(struct kvm_vcpu *vcpu, bool invalidate_gpa)
+{
+	vmx_flush_tlb(vcpu, invalidate_gpa);
+}
+
+int kvm_x86_ops_tlb_remote_flush(struct kvm *kvm)
+{
+	return kvm_x86_ops->tlb_remote_flush(kvm);
+}
+
+int kvm_x86_ops_tlb_remote_flush_with_range(struct kvm *kvm,
+					    struct kvm_tlb_range *range)
+{
+	return kvm_x86_ops->tlb_remote_flush_with_range(kvm, range);
+}
+
+void kvm_x86_ops_tlb_flush_gva(struct kvm_vcpu *vcpu, gva_t addr)
+{
+	vmx_flush_tlb_gva(vcpu, addr);
+}
+
+void kvm_x86_ops_run(struct kvm_vcpu *vcpu)
+{
+	vmx_vcpu_run(vcpu);
+}
+
+int kvm_x86_ops_handle_exit(struct kvm_vcpu *vcpu)
+{
+	return vmx_handle_exit(vcpu);
+}
+
+int kvm_x86_ops_skip_emulated_instruction(struct kvm_vcpu *vcpu)
+{
+	return __skip_emulated_instruction(vcpu);
+}
+
+void kvm_x86_ops_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask)
+{
+	vmx_set_interrupt_shadow(vcpu, mask);
+}
+
+u32 kvm_x86_ops_get_interrupt_shadow(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_interrupt_shadow(vcpu);
+}
+
+void kvm_x86_ops_patch_hypercall(struct kvm_vcpu *vcpu,
+				 unsigned char *hypercall_addr)
+{
+	vmx_patch_hypercall(vcpu, hypercall_addr);
+}
+
+void kvm_x86_ops_set_irq(struct kvm_vcpu *vcpu)
+{
+	vmx_inject_irq(vcpu);
+}
+
+void kvm_x86_ops_set_nmi(struct kvm_vcpu *vcpu)
+{
+	vmx_inject_nmi(vcpu);
+}
+
+void kvm_x86_ops_queue_exception(struct kvm_vcpu *vcpu)
+{
+	vmx_queue_exception(vcpu);
+}
+
+void kvm_x86_ops_cancel_injection(struct kvm_vcpu *vcpu)
+{
+	vmx_cancel_injection(vcpu);
+}
+
+int kvm_x86_ops_interrupt_allowed(struct kvm_vcpu *vcpu)
+{
+	return vmx_interrupt_allowed(vcpu);
+}
+
+int kvm_x86_ops_nmi_allowed(struct kvm_vcpu *vcpu)
+{
+	return vmx_nmi_allowed(vcpu);
+}
+
+bool kvm_x86_ops_get_nmi_mask(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_nmi_mask(vcpu);
+}
+
+void kvm_x86_ops_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked)
+{
+	vmx_set_nmi_mask(vcpu, masked);
+}
+
+void kvm_x86_ops_enable_nmi_window(struct kvm_vcpu *vcpu)
+{
+	enable_nmi_window(vcpu);
+}
+
+void kvm_x86_ops_enable_irq_window(struct kvm_vcpu *vcpu)
+{
+	enable_irq_window(vcpu);
+}
+
+void kvm_x86_ops_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr)
+{
+	update_cr8_intercept(vcpu, tpr, irr);
+}
+
+bool kvm_x86_ops_get_enable_apicv(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_enable_apicv(vcpu);
+}
+
+void kvm_x86_ops_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
+{
+	vmx_refresh_apicv_exec_ctrl(vcpu);
+}
+
+void kvm_x86_ops_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr)
+{
+	vmx_hwapic_irr_update(vcpu, max_irr);
+}
+
+void kvm_x86_ops_hwapic_isr_update(struct kvm_vcpu *vcpu, int isr)
+{
+	vmx_hwapic_isr_update(vcpu, isr);
+}
+
+bool kvm_x86_ops_guest_apic_has_interrupt(struct kvm_vcpu *vcpu)
+{
+	return vmx_guest_apic_has_interrupt(vcpu);
+}
+
+void kvm_x86_ops_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
+{
+	vmx_load_eoi_exitmap(vcpu, eoi_exit_bitmap);
+}
+
+void kvm_x86_ops_set_virtual_apic_mode(struct kvm_vcpu *vcpu)
+{
+	vmx_set_virtual_apic_mode(vcpu);
+}
+
+void kvm_x86_ops_set_apic_access_page_addr(struct kvm_vcpu *vcpu, hpa_t hpa)
+{
+	vmx_set_apic_access_page_addr(vcpu, hpa);
+}
+
+void kvm_x86_ops_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)
+{
+	vmx_deliver_posted_interrupt(vcpu, vector);
+}
+
+int kvm_x86_ops_sync_pir_to_irr(struct kvm_vcpu *vcpu)
+{
+	return vmx_sync_pir_to_irr(vcpu);
+}
+
+int kvm_x86_ops_set_tss_addr(struct kvm *kvm, unsigned int addr)
+{
+	return vmx_set_tss_addr(kvm, addr);
+}
+
+int kvm_x86_ops_set_identity_map_addr(struct kvm *kvm, u64 ident_addr)
+{
+	return vmx_set_identity_map_addr(kvm, ident_addr);
+}
+
+int kvm_x86_ops_get_tdp_level(struct kvm_vcpu *vcpu)
+{
+	return get_ept_level(vcpu);
+}
+
+u64 kvm_x86_ops_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
+{
+	return vmx_get_mt_mask(vcpu, gfn, is_mmio);
+}
+
+int kvm_x86_ops_get_lpage_level(void)
+{
+	return vmx_get_lpage_level();
+}
+
+bool kvm_x86_ops_rdtscp_supported(void)
+{
+	return vmx_rdtscp_supported();
+}
+
+bool kvm_x86_ops_invpcid_supported(void)
+{
+	return vmx_invpcid_supported();
+}
+
+void kvm_x86_ops_set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
+{
+	vmx_set_cr3(vcpu, cr3);
+}
+
+void kvm_x86_ops_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry)
+{
+	vmx_set_supported_cpuid(func, entry);
+}
+
+bool kvm_x86_ops_has_wbinvd_exit(void)
+{
+	return cpu_has_vmx_wbinvd_exit();
+}
+
+u64 kvm_x86_ops_read_l1_tsc_offset(struct kvm_vcpu *vcpu)
+{
+	return vmx_read_l1_tsc_offset(vcpu);
+}
+
+u64 kvm_x86_ops_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset)
+{
+	return vmx_write_l1_tsc_offset(vcpu, offset);
+}
+
+void kvm_x86_ops_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1, u64 *info2)
+{
+	vmx_get_exit_info(vcpu, info1, info2);
+}
+
+int kvm_x86_ops_check_intercept(struct kvm_vcpu *vcpu,
+				struct x86_instruction_info *info,
+				enum x86_intercept_stage stage)
+{
+	return vmx_check_intercept(vcpu, info, stage);
+}
+
+void kvm_x86_ops_handle_exit_irqoff(struct kvm_vcpu *vcpu)
+{
+	vmx_handle_exit_irqoff(vcpu);
+}
+
+bool kvm_x86_ops_mpx_supported(void)
+{
+	return vmx_mpx_supported();
+}
+
+bool kvm_x86_ops_xsaves_supported(void)
+{
+	return vmx_xsaves_supported();
+}
+
+bool kvm_x86_ops_umip_emulated(void)
+{
+	return vmx_umip_emulated();
+}
+
+bool kvm_x86_ops_pt_supported(void)
+{
+	return vmx_pt_supported();
+}
+
+int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr)
+{
+	return kvm_x86_ops->check_nested_events(vcpu, external_intr);
+}
+
+void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu)
+{
+	vmx_request_immediate_exit(vcpu);
+}
+
+void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu)
+{
+	vmx_sched_in(kvm, cpu);
+}
+
+void kvm_x86_ops_slot_enable_log_dirty(struct kvm *kvm,
+				       struct kvm_memory_slot *slot)
+{
+	vmx_slot_enable_log_dirty(kvm, slot);
+}
+
+void kvm_x86_ops_slot_disable_log_dirty(struct kvm *kvm,
+					struct kvm_memory_slot *slot)
+{
+	vmx_slot_disable_log_dirty(kvm, slot);
+}
+
+void kvm_x86_ops_flush_log_dirty(struct kvm *kvm)
+{
+	vmx_flush_log_dirty(kvm);
+}
+
+void kvm_x86_ops_enable_log_dirty_pt_masked(struct kvm *kvm,
+					    struct kvm_memory_slot *slot,
+					    gfn_t offset, unsigned long mask)
+{
+	vmx_enable_log_dirty_pt_masked(kvm, slot, offset, mask);
+}
+
+int kvm_x86_ops_write_log_dirty(struct kvm_vcpu *vcpu)
+{
+	return vmx_write_pml_buffer(vcpu);
+}
+
+int kvm_x86_ops_pre_block(struct kvm_vcpu *vcpu)
+{
+	return vmx_pre_block(vcpu);
+}
+
+void kvm_x86_ops_post_block(struct kvm_vcpu *vcpu)
+{
+	vmx_post_block(vcpu);
+}
+
+void kvm_x86_ops_vcpu_blocking(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->vcpu_blocking(vcpu);
+}
+
+void kvm_x86_ops_vcpu_unblocking(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->vcpu_unblocking(vcpu);
+}
+
+int kvm_x86_ops_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
+			       uint32_t guest_irq, bool set)
+{
+	return vmx_update_pi_irte(kvm, host_irq, guest_irq, set);
+}
+
+void kvm_x86_ops_apicv_post_state_restore(struct kvm_vcpu *vcpu)
+{
+	vmx_apicv_post_state_restore(vcpu);
+}
+
+bool kvm_x86_ops_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu)
+{
+	return vmx_dy_apicv_has_pending_interrupt(vcpu);
+}
+
+int kvm_x86_ops_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc,
+			     bool *expired)
+{
+	return vmx_set_hv_timer(vcpu, guest_deadline_tsc, expired);
+}
+
+void kvm_x86_ops_cancel_hv_timer(struct kvm_vcpu *vcpu)
+{
+	vmx_cancel_hv_timer(vcpu);
+}
+
+void kvm_x86_ops_setup_mce(struct kvm_vcpu *vcpu)
+{
+	vmx_setup_mce(vcpu);
+}
+
+int kvm_x86_ops_get_nested_state(struct kvm_vcpu *vcpu,
+				 struct kvm_nested_state __user *user_kvm_nested_state,
+				 unsigned user_data_size)
+{
+	return kvm_x86_ops->get_nested_state(vcpu, user_kvm_nested_state,
+					     user_data_size);
+}
+
+int kvm_x86_ops_set_nested_state(struct kvm_vcpu *vcpu,
+				 struct kvm_nested_state __user *user_kvm_nested_state,
+				 struct kvm_nested_state *kvm_state)
+{
+	return kvm_x86_ops->set_nested_state(vcpu, user_kvm_nested_state,
+					     kvm_state);
+}
+
+void kvm_x86_ops_get_vmcs12_pages(struct kvm_vcpu *vcpu)
+{
+	kvm_x86_ops->get_vmcs12_pages(vcpu);
+}
+
+int kvm_x86_ops_smi_allowed(struct kvm_vcpu *vcpu)
+{
+	return vmx_smi_allowed(vcpu);
+}
+
+int kvm_x86_ops_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate)
+{
+	return vmx_pre_enter_smm(vcpu, smstate);
+}
+
+int kvm_x86_ops_pre_leave_smm(struct kvm_vcpu *vcpu, const char *smstate)
+{
+	return vmx_pre_leave_smm(vcpu, smstate);
+}
+
+int kvm_x86_ops_enable_smi_window(struct kvm_vcpu *vcpu)
+{
+	return enable_smi_window(vcpu);
+}
+
+int kvm_x86_ops_mem_enc_op(struct kvm *kvm, void __user *argp)
+{
+	return kvm_x86_ops->mem_enc_op(kvm, argp);
+}
+
+int kvm_x86_ops_mem_enc_reg_region(struct kvm *kvm,
+				   struct kvm_enc_region *argp)
+{
+	return kvm_x86_ops->mem_enc_reg_region(kvm, argp);
+}
+
+int kvm_x86_ops_mem_enc_unreg_region(struct kvm *kvm,
+				     struct kvm_enc_region *argp)
+{
+	return kvm_x86_ops->mem_enc_unreg_region(kvm, argp);
+}
+
+int kvm_x86_ops_get_msr_feature(struct kvm_msr_entry *entry)
+{
+	return vmx_get_msr_feature(entry);
+}
+
+int kvm_x86_ops_nested_enable_evmcs(struct kvm_vcpu *vcpu,
+				    uint16_t *vmcs_version)
+{
+	return kvm_x86_ops->nested_enable_evmcs(vcpu, vmcs_version);
+}
+
+uint16_t kvm_x86_ops_nested_get_evmcs_version(struct kvm_vcpu *vcpu)
+{
+	return kvm_x86_ops->nested_get_evmcs_version(vcpu);
+}
+
+bool kvm_x86_ops_need_emulation_on_page_fault(struct kvm_vcpu *vcpu)
+{
+	return vmx_need_emulation_on_page_fault(vcpu);
+}
+
+bool kvm_x86_ops_apic_init_signal_blocked(struct kvm_vcpu *vcpu)
+{
+	return vmx_apic_init_signal_blocked(vcpu);
+}
#8 lore raw

Re: [PATCH 02/17] KVM: monolithic: x86: convert the kvm_x86_ops methods to external functions

Quoted 13 lines · Paolo Bonzini
Quoted 12 lines · Andrea Arcangeli
arch/x86/kvm/svm_ops.c new file +6 -0
new file mode 100644
index 000000000000..2aaabda92179
@@ -0,0 +1,672 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  arch/x86/kvm/svm_ops.c
+ *
+ *  Copyright 2019 Red Hat, Inc.
+ */
Quoted 8 lines

arch/x86/kvm/svm_ops.c line 11

Quoted 5 lines
arch/x86/kvm/svm_ops.c new file +5 -0
+
+int kvm_x86_ops_cpu_has_kvm_support(void)
+{
+	return has_svm();
+}

Can you just rename all the functions in vmx/ and svm.c, instead of adding forwarders?

Yeah, having kvm_x86_ be analogous to kvm_arch_ seems like the obvious approach. The necessary VMX and SVM renaming can be done in separate preparatory patches, and the conversion from kvm_x86_ops to direct calls would be fairly straightforward.

Alternatively, what if we use macros in the call sites, e.g. keep/require vmx_ and svm_ prefixes for all functions, renaming VMX and SVM code as needed? E.g.:

cpu_has_vmx_support -> vmx_supported_by_cpu
cpu_has_svm_support -> svm_supported_by_cpu

int vmx_disabled_by_bios(void)
int svm_disabled_by_bios(void)

  #define X86_OP(name) kvm_x86_vendor##_##name

int kvm_arch_init(void *opaque)

  {
	if (X86_OP(supported_by_cpu())) {
		printk(KERN_ERR "kvm: no hardware support\n");
		r = -EOPNOTSUPP;
		goto out;
	}
	if (X86_OP(disabled_by_bios())) {
		printk(KERN_ERR "kvm: disabled by bios\n");
		r = -EOPNOTSUPP;
		goto out;
	}	

}

Pros:
- Smaller patches due to less renaming in VMX and SVM
- Calls to vendor code are very obvious
- Stack traces contain vmx vs. svm instead of kvm_x86

Cons:
- Macros
- Annoying development environment, e.g. editors tend to struggle with macrofied funtion/variable names.

maintainer
#9 lore raw

Re: [PATCH 02/17] KVM: monolithic: x86: convert the kvm_x86_ops methods to external functions

Quoted 10 lines · Sean Christopherson

Alternatively, what if we use macros in the call sites, e.g. keep/require vmx_ and svm_ prefixes for all functions, renaming VMX and SVM code as needed? E.g.:

  #define X86_OP(name) kvm_x86_vendor##_##name

int kvm_arch_init(void *opaque)

  {
	if (X86_OP(supported_by_cpu())) {

Please no, the extra parentheses would be a mess to review.

Paolo

#10 lore raw

Re: [PATCH 02/17] KVM: monolithic: x86: convert the kvm_x86_ops methods to external functions

In reply to Paolo Bonzini
Quoted 13 lines · Paolo Bonzini
Quoted 12 lines · Andrea Arcangeli
arch/x86/kvm/svm_ops.c new file +6 -0
new file mode 100644
index 000000000000..2aaabda92179
@@ -0,0 +1,672 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  arch/x86/kvm/svm_ops.c
+ *
+ *  Copyright 2019 Red Hat, Inc.
+ */
Quoted 8 lines

arch/x86/kvm/svm_ops.c line 11

Quoted 5 lines
arch/x86/kvm/svm_ops.c new file +5 -0
+
+int kvm_x86_ops_cpu_has_kvm_support(void)
+{
+	return has_svm();
+}

Can you just rename all the functions in vmx/ and svm.c, instead of adding forwarders?

I can do that, I thought this was cleaner as it still retained the abstraction separated from the mixup of the rest of the vmx/svm code, but it'll work the same by dropping the abstraction in kvm_ops.h and just maintaining a common name between the svm.c and vmx.c files, gcc already built it that way after all.

#11 lore raw

PATCH 3/17 KVM: monolithic: x86: handle the request_immediate_exit variation

In reply to Andrea Arcangeli

request_immediate_exit is one of those few cases where the pointer to function of the method isn't fixed at build time and it requires special handling because hardware_setup() may override it at runtime.

---
 arch/x86/kvm/vmx/vmx_ops.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx_ops.c +4 -1
index cdcad73935d9..25d441432901 100644
@@ -498,7 +498,10 @@ int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr)
 
 void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu)
 {
-	vmx_request_immediate_exit(vcpu);
+	if (likely(enable_preemption_timer))
+		vmx_request_immediate_exit(vcpu);
+	else
+		__kvm_request_immediate_exit(vcpu);
 }
 
 void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu)
#12 lore raw

Re: [PATCH 03/17] KVM: monolithic: x86: handle the request_immediate_exit variation

Quoted 14 lines · Andrea Arcangeli

request_immediate_exit is one of those few cases where the pointer to function of the method isn't fixed at build time and it requires special handling because hardware_setup() may override it at runtime.

---
 arch/x86/kvm/vmx/vmx_ops.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx_ops.c +0 -0
index cdcad73935d9..25d441432901 100644

arch/x86/kvm/vmx/vmx_ops.c int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) · line 504

Quoted 8 lines
arch/x86/kvm/vmx/vmx_ops.c +4 -1
@@ -498,7 +498,10 @@ int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr)
 
 void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu)
 {
-	vmx_request_immediate_exit(vcpu);
+	if (likely(enable_preemption_timer))
+		vmx_request_immediate_exit(vcpu);
+	else
+		__kvm_request_immediate_exit(vcpu);

Rather than wrap this in VMX code, what if we instead take advantage of a monolithic module and add an inline to query enable_preemption_timer? That'd likely save a few CALL/RET/JMP instructions and eliminate __kvm_request_immediate_exit.

E.g. something like:

	if (req_immediate_exit) {
		kvm_make_request(KVM_REQ_EVENT, vcpu);
		if (kvm_x86_has_request_immediate_exit())
			kvm_x86_request_immediate_exit(vcpu);
		else
			smp_send_reschedule(vcpu->cpu);
	}
Quoted 3 lines
arch/x86/kvm/vmx/vmx_ops.c +0 -0
 }
 
 void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu)
#13 lore raw

Re: [PATCH 03/17] KVM: monolithic: x86: handle the request_immediate_exit variation

Quoted 31 lines · Sean Christopherson
Quoted 14 lines · Andrea Arcangeli

request_immediate_exit is one of those few cases where the pointer to function of the method isn't fixed at build time and it requires special handling because hardware_setup() may override it at runtime.

---
 arch/x86/kvm/vmx/vmx_ops.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx_ops.c +0 -0
index cdcad73935d9..25d441432901 100644

arch/x86/kvm/vmx/vmx_ops.c int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) · line 504

Quoted 8 lines
arch/x86/kvm/vmx/vmx_ops.c +4 -1
@@ -498,7 +498,10 @@ int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr)
 
 void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu)
 {
-	vmx_request_immediate_exit(vcpu);
+	if (likely(enable_preemption_timer))
+		vmx_request_immediate_exit(vcpu);
+	else
+		__kvm_request_immediate_exit(vcpu);

Rather than wrap this in VMX code, what if we instead take advantage of a monolithic module and add an inline to query enable_preemption_timer? That'd likely save a few CALL/RET/JMP instructions and eliminate __kvm_request_immediate_exit.

E.g. something like:

Quoted 7 lines
	if (req_immediate_exit) {
		kvm_make_request(KVM_REQ_EVENT, vcpu);
		if (kvm_x86_has_request_immediate_exit())
			kvm_x86_request_immediate_exit(vcpu);
		else
			smp_send_reschedule(vcpu->cpu);
	}

Yes, I mentioned the inlining possibilities in part of comment of 2/17:

===
Further incremental minor optimizations that weren't possible before are now enabled by the monolithic model. For example it would be possible later to convert some of the small external methods to inline functions from the {svm,vmx}_ops.c file to kvm_ops.h. However that will require more Makefile tweaks.
===

To implement your kvm_x86_has_request_immediate_exit() we need more Makefile tweaking, basically we need a -D__SVM__ -D__VMX__ kind of thing so we can put an #ifdef __SVM__ in the kvm_ops.h equivalent to put inline code there. Or some other better solution if you think of any, that was my only idea so far with regard to inlining.

If you can sort out the solution to enable the inlining of svm/vmx code that would be great.

However I think all optimizations allowed by the monolithic model that aren't related to the full retpoline overhead removal, should be kept incremental.

I rather prefer to keep the inner working of the kvm_x86_ops to be 100% functional equivalent in the initial conversion, so things remains also more bisectable just in case and to keep the optimizations incremental.

In addition to the inline optimization there is the cleanup of the kvm_x86_ops that is lots of work left to do. To do that it requires a lot of changes to every place that checks the kvm_x86_ops->something pointer or that isn't initialized statically. I didn't even try to implement that part because I wanted to get to a point that was fully equivalent for easier review and fewer risk of breakage. So I sent the patchset at the point when there were zero reptolines left running in the KVM code, but more work is required to fully leverage the monolithic approach and eliminate the now mostly dead code of kvm_x86_ops structure.

I would rather prioritize on the changes required to the full removal of kvm_x86_ops before further inline optimizations, but it would be fine to do inline optimizations first as long as they're not mixed up with an initial simpler conversion to monolithic model.

Thanks,
Andrea

#14 lore raw

Re: [PATCH 03/17] KVM: monolithic: x86: handle the request_immediate_exit variation

Quoted 43 lines · Andrea Arcangeli
Quoted 31 lines · Sean Christopherson
Quoted 14 lines · Andrea Arcangeli

request_immediate_exit is one of those few cases where the pointer to function of the method isn't fixed at build time and it requires special handling because hardware_setup() may override it at runtime.

---
 arch/x86/kvm/vmx/vmx_ops.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx_ops.c +0 -0
index cdcad73935d9..25d441432901 100644

arch/x86/kvm/vmx/vmx_ops.c int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) · line 504

Quoted 8 lines
arch/x86/kvm/vmx/vmx_ops.c +4 -1
@@ -498,7 +498,10 @@ int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr)
 
 void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu)
 {
-	vmx_request_immediate_exit(vcpu);
+	if (likely(enable_preemption_timer))
+		vmx_request_immediate_exit(vcpu);
+	else
+		__kvm_request_immediate_exit(vcpu);

Rather than wrap this in VMX code, what if we instead take advantage of a monolithic module and add an inline to query enable_preemption_timer? That'd likely save a few CALL/RET/JMP instructions and eliminate __kvm_request_immediate_exit.

E.g. something like:

Quoted 7 lines
	if (req_immediate_exit) {
		kvm_make_request(KVM_REQ_EVENT, vcpu);
		if (kvm_x86_has_request_immediate_exit())
			kvm_x86_request_immediate_exit(vcpu);
		else
			smp_send_reschedule(vcpu->cpu);
	}

Yes, I mentioned the inlining possibilities in part of comment of 2/17:

Quoted 7 lines

===
Further incremental minor optimizations that weren't possible before are now enabled by the monolithic model. For example it would be possible later to convert some of the small external methods to inline functions from the {svm,vmx}_ops.c file to kvm_ops.h. However that will require more Makefile tweaks.
===

With a straight rename to kvm_x86_<function>() instead of wrappers, we shouldn't need kvm_ops.c. kvm_ops.h might be helpful, but it'd be just as easy to keep them in kvm_host.h and would likely yield a more insightful diff[*].

Quoted 5 lines

To implement your kvm_x86_has_request_immediate_exit() we need more Makefile tweaking, basically we need a -D__SVM__ -D__VMX__ kind of thing so we can put an #ifdef __SVM__ in the kvm_ops.h equivalent to put inline code there. Or some other better solution if you think of any, that was my only idea so far with regard to inlining.

Hmm, I was thinking more along the lines of extending the kvm_host.h pattern down into vendor specific code, e.g. arch/x86/kvm/vmx/kvm_host.h. Probably with a different name though, two of those is confusing enough.

It'd still need Makefile changes, but we wouldn't litter the code with #ifdefs. Future enhancments can also take advantage of the per-vendor header to inline other things. Such a header would also make it possible to fully remove kvm_x86_ops in this series (I think).

[*] Tying into the thought above, if we go for a straight rename and
eliminate the conditionally-implemented kvm_x86_ops ahead of time, e.g. with inlines that return -EINVAL or something, then the conversion to direct calls can be a straight replacement of "kvm_x86_ops->" with "kvm_x86_" at the same time the declarations are changed from members of kvm_x86_ops to externs.

Actually, typing out the above made me realize the immediate exit code can be:

	if (req_immediate_exit) {
		kvm_make_request(KVM_REQ_EVENT, vcpu);
		if (kvm_x86_request_immediate_exit(vcpu))
			smp_send_reschedule(vcpu->cpu);
	}

Where kvm_x86_request_immediate_exit() returns 0 on success, e.g. the SVM implementation can be "return -EINVAL" or whatever is appropriate, which I assume the compiler can optimize out. Or maybe a boolean return is better in this case?

#15 lore raw

Re: [PATCH 03/17] KVM: monolithic: x86: handle the request_immediate_exit variation

Quoted 4 lines · Sean Christopherson

With a straight rename to kvm_x86_<function>() instead of wrappers, we shouldn't need kvm_ops.c. kvm_ops.h might be helpful, but it'd be just as easy to keep them in kvm_host.h and would likely yield a more insightful diff[*].

Yes, I already commented about this change Paolo asked.

Quoted 8 lines

Hmm, I was thinking more along the lines of extending the kvm_host.h pattern down into vendor specific code, e.g. arch/x86/kvm/vmx/kvm_host.h. Probably with a different name though, two of those is confusing enough.

It'd still need Makefile changes, but we wouldn't litter the code with #ifdefs. Future enhancments can also take advantage of the per-vendor header to inline other things. Such a header would also make it possible to fully remove kvm_x86_ops in this series (I think).

It's common in include/linux/* to include the same .h file that just implements the inlines depending on some #ifdef, but in this case it's simpler to have different .h files to keep the two versions more separated as they may be maintained by different groups, so including different .h file sounds better than -D__VMX__ -D__VMX__ agreed.

Quoted 16 lines

[*] Tying into the thought above, if we go for a straight rename and
eliminate the conditionally-implemented kvm_x86_ops ahead of time, e.g. with inlines that return -EINVAL or something, then the conversion to direct calls can be a straight replacement of "kvm_x86_ops->" with "kvm_x86_" at the same time the declarations are changed from members of kvm_x86_ops to externs.

Actually, typing out the above made me realize the immediate exit code can be:

	if (req_immediate_exit) {
		kvm_make_request(KVM_REQ_EVENT, vcpu);
		if (kvm_x86_request_immediate_exit(vcpu))
			smp_send_reschedule(vcpu->cpu);
	}
Quoted 4 lines

Where kvm_x86_request_immediate_exit() returns 0 on success, e.g. the SVM implementation can be "return -EINVAL" or whatever is appropriate, which I assume the compiler can optimize out. Or maybe a boolean return is better in this case?

While the final cleanup of kvm_x86_ops doesn't strictly require the inlining Makefile tricks to be functional just yet, it would be beneficial to remove some branch at runtime from non frequently invoked methods and to get the final optimal implementation sorted out during the initial cleanup of the structure, this should reduce the number of patches to get to the most optimal possible end result. So I think making the inline work in the Makefile as a dependency to remove kvm_x86_ops is a fine plan.

Thanks,
Andrea

#16 lore raw

PATCH 4/17 KVM: monolithic: x86: convert the kvm_pmu_ops methods to external functions

In reply to Andrea Arcangeli

This replaces all kvm_pmu_ops pointer to functions with regular external functions that don't require indirect calls.

---
 arch/x86/kvm/pmu_amd_ops.c       | 68 ++++++++++++++++++++++++++++++++
 arch/x86/kvm/pmu_ops.h           | 22 +++++++++++
 arch/x86/kvm/vmx/pmu_intel_ops.c | 68 ++++++++++++++++++++++++++++++++
 3 files changed, 158 insertions(+)
 create mode 100644 arch/x86/kvm/pmu_amd_ops.c
 create mode 100644 arch/x86/kvm/pmu_ops.h
 create mode 100644 arch/x86/kvm/vmx/pmu_intel_ops.c
arch/x86/kvm/pmu_amd_ops.c new file +68 -0
new file mode 100644
index 000000000000..d2e9a244caa8
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  arch/x86/kvm/pmu_amd_ops.c
+ *
+ *  Copyright 2019 Red Hat, Inc.
+ */
+
+unsigned kvm_pmu_ops_find_arch_event(struct kvm_pmu *pmu, u8 event_select,
+				     u8 unit_mask)
+{
+	return amd_find_arch_event(pmu, event_select, unit_mask);
+}
+
+unsigned kvm_pmu_ops_find_fixed_event(int idx)
+{
+	return amd_find_fixed_event(idx);
+}
+
+bool kvm_pmu_ops_pmc_is_enabled(struct kvm_pmc *pmc)
+{
+	return amd_pmc_is_enabled(pmc);
+}
+
+struct kvm_pmc *kvm_pmu_ops_pmc_idx_to_pmc(struct kvm_pmu *pmu, int pmc_idx)
+{
+	return amd_pmc_idx_to_pmc(pmu, pmc_idx);
+}
+
+struct kvm_pmc *kvm_pmu_ops_msr_idx_to_pmc(struct kvm_vcpu *vcpu, unsigned idx,
+					   u64 *mask)
+{
+	return amd_msr_idx_to_pmc(vcpu, idx, mask);
+}
+
+int kvm_pmu_ops_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx)
+{
+	return amd_is_valid_msr_idx(vcpu, idx);
+}
+
+bool kvm_pmu_ops_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
+{
+	return amd_is_valid_msr(vcpu, msr);
+}
+
+int kvm_pmu_ops_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data)
+{
+	return amd_pmu_get_msr(vcpu, msr, data);
+}
+
+int kvm_pmu_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
+{
+	return amd_pmu_set_msr(vcpu, msr_info);
+}
+
+void kvm_pmu_ops_refresh(struct kvm_vcpu *vcpu)
+{
+	amd_pmu_refresh(vcpu);
+}
+
+void kvm_pmu_ops_init(struct kvm_vcpu *vcpu)
+{
+	amd_pmu_init(vcpu);
+}
+
+void kvm_pmu_ops_reset(struct kvm_vcpu *vcpu)
+{
+	amd_pmu_reset(vcpu);
+}
arch/x86/kvm/pmu_ops.h new file +22 -0
new file mode 100644
index 000000000000..6230ce300cbe
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __KVM_X86_PMU_OPS_H
+#define __KVM_X86_PMU_OPS_H
+
+extern unsigned kvm_pmu_ops_find_arch_event(struct kvm_pmu *pmu,
+					    u8 event_select, u8 unit_mask);
+extern unsigned kvm_pmu_ops_find_fixed_event(int idx);
+extern bool kvm_pmu_ops_pmc_is_enabled(struct kvm_pmc *pmc);
+extern struct kvm_pmc *kvm_pmu_ops_pmc_idx_to_pmc(struct kvm_pmu *pmu,
+						  int pmc_idx);
+extern struct kvm_pmc *kvm_pmu_ops_msr_idx_to_pmc(struct kvm_vcpu *vcpu,
+						  unsigned idx, u64 *mask);
+extern int kvm_pmu_ops_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx);
+extern bool kvm_pmu_ops_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr);
+extern int kvm_pmu_ops_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data);
+extern int kvm_pmu_ops_set_msr(struct kvm_vcpu *vcpu,
+			       struct msr_data *msr_info);
+extern void kvm_pmu_ops_refresh(struct kvm_vcpu *vcpu);
+extern void kvm_pmu_ops_init(struct kvm_vcpu *vcpu);
+extern void kvm_pmu_ops_reset(struct kvm_vcpu *vcpu);
+
+#endif /* __KVM_X86_PMU_OPS_H */
arch/x86/kvm/vmx/pmu_intel_ops.c new file +68 -0

68 lines not on this page · raw message

#17 lore raw

PATCH 5/17 KVM: monolithic: x86: enable the kvm_x86_ops external functions

In reply to Andrea Arcangeli

Plug in the new external functions and their extern declarations in the respective kernel modules (kvm-intel and kvm-amd).

---
 arch/x86/include/asm/kvm_host.h | 2 ++
 arch/x86/kvm/svm.c              | 2 ++
 arch/x86/kvm/vmx/vmx.c          | 2 ++
 3 files changed, 6 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
index a3a3ec73fa2f..99b30affb8ad 100644
@@ -1002,6 +1002,8 @@ struct kvm_lapic_irq {
 	bool msi_redir_hint;
 };
 
+#include <asm/kvm_ops.h>
+
 struct kvm_x86_ops {
 	int (*cpu_has_kvm_support)(void);          /* __init */
 	int (*disabled_by_bios)(void);             /* __init */
arch/x86/kvm/svm.c +2 -0
index 04fe21849b6e..3041abbd643b 100644
@@ -7179,6 +7179,8 @@ static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu)
 		   (svm->vmcb->control.intercept & (1ULL << INTERCEPT_INIT));
 }
 
+#include "svm_ops.c"
+
 static struct kvm_x86_ops svm_x86_ops __ro_after_init = {
 	.cpu_has_kvm_support = has_svm,
 	.disabled_by_bios = is_disabled,
arch/x86/kvm/vmx/vmx.c +2 -0
index 4a99be1fae4e..9fc3969f6443 100644
@@ -7650,6 +7650,8 @@ static __exit void hardware_unsetup(void)
 	free_kvm_area();
 }
 
+#include "vmx_ops.c"
+
 static struct kvm_x86_ops vmx_x86_ops __ro_after_init = {
 	.cpu_has_kvm_support = cpu_has_kvm_support,
 	.disabled_by_bios = vmx_disabled_by_bios,
#18 lore raw

PATCH 6/17 KVM: monolithic: x86: enable the kvm_pmu_ops external functions

In reply to Andrea Arcangeli

Plug in the new external functions and their extern declarations in the respective kernel modules (kvm-intel and kvm-amd).

---
 arch/x86/kvm/pmu.h           | 2 ++
 arch/x86/kvm/pmu_amd.c       | 2 ++
 arch/x86/kvm/vmx/pmu_intel.c | 2 ++
 3 files changed, 6 insertions(+)
arch/x86/kvm/pmu.h +2 -0
index 58265f761c3b..a9c2f68a40cc 100644
@@ -19,6 +19,8 @@ struct kvm_event_hw_type_mapping {
 	unsigned event_type;
 };
 
+#include "pmu_ops.h"
+
 struct kvm_pmu_ops {
 	unsigned (*find_arch_event)(struct kvm_pmu *pmu, u8 event_select,
 				    u8 unit_mask);
arch/x86/kvm/pmu_amd.c +2 -0
index c8388389a3b0..12d1fa3ba35a 100644
@@ -301,6 +301,8 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu)
 	}
 }
 
+#include "pmu_amd_ops.c"
+
 struct kvm_pmu_ops amd_pmu_ops = {
 	.find_arch_event = amd_find_arch_event,
 	.find_fixed_event = amd_find_fixed_event,
arch/x86/kvm/vmx/pmu_intel.c +2 -0
index 4dea0e0e7e39..5ef36a75c31e 100644
@@ -358,6 +358,8 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu)
 		pmu->global_ovf_ctrl = 0;
 }
 
+#include "pmu_intel_ops.c"
+
 struct kvm_pmu_ops intel_pmu_ops = {
 	.find_arch_event = intel_find_arch_event,
 	.find_fixed_event = intel_find_fixed_event,
#19 lore raw

PATCH 7/17 KVM: monolithic: x86: adjust the section prefixes

In reply to Andrea Arcangeli

Adjusts the section prefixes of some KVM common code function because with the monolithic methods the section checker can now do a more accurate analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.

---
 arch/x86/kvm/svm.c     | 2 +-
 arch/x86/kvm/vmx/vmx.c | 2 +-
 arch/x86/kvm/x86.c     | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/svm.c +1 -1
index 3041abbd643b..5a48beb58083 100644
@@ -1407,7 +1407,7 @@ static __init int svm_hardware_setup(void)
 	return r;
 }
 
-static __exit void svm_hardware_unsetup(void)
+static void svm_hardware_unsetup(void)
 {
 	int cpu;
 
arch/x86/kvm/vmx/vmx.c +1 -1
index 9fc3969f6443..09e6a477e06f 100644
@@ -7642,7 +7642,7 @@ static __init int hardware_setup(void)
 	return r;
 }
 
-static __exit void hardware_unsetup(void)
+static void hardware_unsetup(void)
 {
 	if (nested)
 		nested_vmx_hardware_unsetup();
arch/x86/kvm/x86.c +2 -2
index dfd641243568..c04894b61384 100644
@@ -9206,7 +9206,7 @@ void kvm_arch_hardware_disable(void)
 	drop_user_return_notifiers();
 }
 
-int kvm_arch_hardware_setup(void)
+__init int kvm_arch_hardware_setup(void)
 {
 	int r;
 
@@ -9237,7 +9237,7 @@ void kvm_arch_hardware_unsetup(void)
 	kvm_x86_ops->hardware_unsetup();
 }
 
-int kvm_arch_check_processor_compat(void)
+__init int kvm_arch_check_processor_compat(void)
 {
 	return kvm_x86_ops->check_processor_compatibility();
 }
maintainer
#20 lore raw

Re: [PATCH 07/17] KVM: monolithic: x86: adjust the section prefixes

Quoted 6 lines · Andrea Arcangeli

Adjusts the section prefixes of some KVM common code function because with the monolithic methods the section checker can now do a more accurate analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.

I think it's the opposite---the checker is detecting *missing* section prefixes, for example vmx_exit, kvm_exit, kvm_arch_hardware_unsetup etc. could be marked __exit.

Paolo

#21 lore raw

Re: [PATCH 07/17] KVM: monolithic: x86: adjust the section prefixes

Hello,

Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Andrea Arcangeli

Adjusts the section prefixes of some KVM common code function because with the monolithic methods the section checker can now do a more accurate analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.

I think it's the opposite---the checker is detecting *missing* section prefixes, for example vmx_exit, kvm_exit, kvm_arch_hardware_unsetup etc. could be marked __exit.

I added the two missing __init.

The __exit removed from unsetup is because kvm_arch_hardware_unsetup is called by kvm_init, so unless somehow kvm_init can go in the exit section and be dropped too during the final kernel link (which would prevent KVM to initialize in the first place at kernel boot), it's not feasible to call a function located in the exit section and dropped during the kernel link from there.

As far as I can tell with upstream KVM if you hit the kvm_arch_hardware_unsetup function during kvm_init error path it'll crash the kernel at boot because of it.

Removing __exit fixes that potential upstream crash and upstream bug.

The comment header was short, I'll add more commentary to the commit header to reduce the confusion about why removing __exit is needed.

Thanks,
Andrea

maintainer
#22 lore raw

Re: [PATCH 07/17] KVM: monolithic: x86: adjust the section prefixes

Quoted 8 lines · Andrea Arcangeli

The __exit removed from unsetup is because kvm_arch_hardware_unsetup is called by kvm_init, so unless somehow kvm_init can go in the exit section and be dropped too during the final kernel link (which would prevent KVM to initialize in the first place at kernel boot), it's not feasible to call a function located in the exit section and dropped during the kernel link from there.

Quoted 5 lines

As far as I can tell with upstream KVM if you hit the kvm_arch_hardware_unsetup function during kvm_init error path it'll crash the kernel at boot because of it.

Removing __exit fixes that potential upstream crash and upstream bug.

You're right.

Quoted 2 lines

The comment header was short, I'll add more commentary to the commit header to reduce the confusion about why removing __exit is needed.

Yes, thanks!

Paolo

#23 lore raw

PATCH 8/17 KVM: monolithic: adjust the section prefixes in the KVM common code

In reply to Andrea Arcangeli

Adjusts the section prefixes of some KVM common code function because with the monolithic methods the section checker can now do a more accurate analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n.

---
 virt/kvm/kvm_main.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
virt/kvm/kvm_main.c +3 -3
index e6de3159e682..9aa448ea688f 100644
@@ -4235,13 +4235,13 @@ static void kvm_sched_out(struct preempt_notifier *pn,
 	kvm_arch_vcpu_put(vcpu);
 }
 
-static void check_processor_compat(void *rtn)
+static __init void check_processor_compat(void *rtn)
 {
 	*(int *)rtn = kvm_arch_check_processor_compat();
 }
 
-int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
-		  struct module *module)
+__init int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
+		    struct module *module)
 {
 	int r;
 	int cpu;
#24 lore raw

PATCH 9/17 KVM: monolithic: x86: remove kvm.ko

In reply to Andrea Arcangeli

This removes kvm.ko and it links and duplicates all kvm.ko objects to both kvm-amd and kvm-intel.

---
 arch/x86/kvm/Makefile | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
arch/x86/kvm/Makefile +2 -3
index 31ecf7a76d5a..68b81f381369 100644
@@ -12,9 +12,8 @@ kvm-y			+= x86.o mmu.o emulate.o i8259.o irq.o lapic.o \
 			   i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \
 			   hyperv.o page_track.o debugfs.o
 
-kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o
-kvm-amd-y		+= svm.o pmu_amd.o
+kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o $(kvm-y)
+kvm-amd-y		+= svm.o pmu_amd.o $(kvm-y)
 
-obj-$(CONFIG_KVM)	+= kvm.o
 obj-$(CONFIG_KVM_INTEL)	+= kvm-intel.o
 obj-$(CONFIG_KVM_AMD)	+= kvm-amd.o
#25 lore raw

PATCH 10/17 KVM: monolithic: x86: use the external functions instead of kvm_x86_ops

In reply to Andrea Arcangeli

Now that the new methods are plugged in and they are functional use them instead of invoking the pointer to functions through kvm_x86_ops.

---
 arch/x86/include/asm/kvm_host.h |  10 +-
 arch/x86/kvm/cpuid.c            |  22 +--
 arch/x86/kvm/hyperv.c           |   6 +-
 arch/x86/kvm/kvm_cache_regs.h   |  10 +-
 arch/x86/kvm/lapic.c            |  28 +--
 arch/x86/kvm/mmu.c              |  26 +--
 arch/x86/kvm/mmu.h              |   4 +-
 arch/x86/kvm/pmu.c              |  24 +--
 arch/x86/kvm/pmu.h              |   2 +-
 arch/x86/kvm/trace.h            |   4 +-
 arch/x86/kvm/vmx/pmu_intel.c    |   2 +-
 arch/x86/kvm/x86.c              | 304 ++++++++++++++++----------------
 arch/x86/kvm/x86.h              |   2 +-
 13 files changed, 222 insertions(+), 222 deletions(-)
arch/x86/include/asm/kvm_host.h +5 -5
index 99b30affb8ad..bd5f4f900288 100644
@@ -1230,19 +1230,19 @@ extern struct kmem_cache *x86_fpu_cache;
 #define __KVM_HAVE_ARCH_VM_ALLOC
 static inline struct kvm *kvm_arch_alloc_vm(void)
 {
-	return kvm_x86_ops->vm_alloc();
+	return kvm_x86_ops_vm_alloc();
 }
 
 static inline void kvm_arch_free_vm(struct kvm *kvm)
 {
-	return kvm_x86_ops->vm_free(kvm);
+	return kvm_x86_ops_vm_free(kvm);
 }
 
 #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLB
 static inline int kvm_arch_flush_remote_tlb(struct kvm *kvm)
 {
 	if (kvm_x86_ops->tlb_remote_flush &&
-	    !kvm_x86_ops->tlb_remote_flush(kvm))
+	    !kvm_x86_ops_tlb_remote_flush(kvm))
 		return 0;
 	else
 		return -ENOTSUPP;
@@ -1599,13 +1599,13 @@ static inline bool kvm_irq_is_postable(struct kvm_lapic_irq *irq)
 static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
 {
 	if (kvm_x86_ops->vcpu_blocking)
-		kvm_x86_ops->vcpu_blocking(vcpu);
+		kvm_x86_ops_vcpu_blocking(vcpu);
 }
 
 static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
 {
 	if (kvm_x86_ops->vcpu_unblocking)
-		kvm_x86_ops->vcpu_unblocking(vcpu);
+		kvm_x86_ops_vcpu_unblocking(vcpu);
 }
 
 static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
arch/x86/kvm/cpuid.c +11 -11
index dd5985eb61b4..d8d840fc703e 100644
@@ -48,7 +48,7 @@ static u32 xstate_required_size(u64 xstate_bv, bool compacted)
 bool kvm_mpx_supported(void)
 {
 	return ((host_xcr0 & (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_BNDCSR))
-		 && kvm_x86_ops->mpx_supported());
+		 && kvm_x86_ops_mpx_supported());
 }
 EXPORT_SYMBOL_GPL(kvm_mpx_supported);
 
@@ -232,7 +232,7 @@ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu,
 	vcpu->arch.cpuid_nent = cpuid->nent;
 	cpuid_fix_nx_cap(vcpu);
 	kvm_apic_set_version(vcpu);
-	kvm_x86_ops->cpuid_update(vcpu);
+	kvm_x86_ops_cpuid_update(vcpu);
 	r = kvm_update_cpuid(vcpu);
 
 out:
@@ -255,7 +255,7 @@ int kvm_vcpu_ioctl_set_cpuid2(struct kvm_vcpu *vcpu,
 		goto out;
 	vcpu->arch.cpuid_nent = cpuid->nent;
 	kvm_apic_set_version(vcpu);
-	kvm_x86_ops->cpuid_update(vcpu);
+	kvm_x86_ops_cpuid_update(vcpu);
 	r = kvm_update_cpuid(vcpu);
 out:
 	return r;
@@ -341,10 +341,10 @@ static int __do_cpuid_func_emulated(struct kvm_cpuid_entry2 *entry,
 
 static inline void do_cpuid_7_mask(struct kvm_cpuid_entry2 *entry, int index)
 {
-	unsigned f_invpcid = kvm_x86_ops->invpcid_supported() ? F(INVPCID) : 0;
+	unsigned f_invpcid = kvm_x86_ops_invpcid_supported() ? F(INVPCID) : 0;
 	unsigned f_mpx = kvm_mpx_supported() ? F(MPX) : 0;
-	unsigned f_umip = kvm_x86_ops->umip_emulated() ? F(UMIP) : 0;
-	unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0;
+	unsigned f_umip = kvm_x86_ops_umip_emulated() ? F(UMIP) : 0;
+	unsigned f_intel_pt = kvm_x86_ops_pt_supported() ? F(INTEL_PT) : 0;
 	unsigned f_la57;
 
 	/* cpuid 7.0.ebx */
@@ -426,16 +426,16 @@ static inline int __do_cpuid_func(struct kvm_cpuid_entry2 *entry, u32 function,
 	int r;
 	unsigned f_nx = is_efer_nx() ? F(NX) : 0;
 #ifdef CONFIG_X86_64
-	unsigned f_gbpages = (kvm_x86_ops->get_lpage_level() == PT_PDPE_LEVEL)
+	unsigned f_gbpages = (kvm_x86_ops_get_lpage_level() == PT_PDPE_LEVEL)
 				? F(GBPAGES) : 0;
 	unsigned f_lm = F(LM);
 #else
 	unsigned f_gbpages = 0;
 	unsigned f_lm = 0;
 #endif
-	unsigned f_rdtscp = kvm_x86_ops->rdtscp_supported() ? F(RDTSCP) : 0;
-	unsigned f_xsaves = kvm_x86_ops->xsaves_supported() ? F(XSAVES) : 0;
-	unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0;
+	unsigned f_rdtscp = kvm_x86_ops_rdtscp_supported() ? F(RDTSCP) : 0;
+	unsigned f_xsaves = kvm_x86_ops_xsaves_supported() ? F(XSAVES) : 0;
+	unsigned f_intel_pt = kvm_x86_ops_pt_supported() ? F(INTEL_PT) : 0;
 
 	/* cpuid 1.edx */
 	const u32 kvm_cpuid_1_edx_x86_features =
@@ -786,7 +786,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_entry2 *entry, u32 function,
 		break;
 	}
 
-	kvm_x86_ops->set_supported_cpuid(function, entry);
+	kvm_x86_ops_set_supported_cpuid(function, entry);
 
 	r = 0;
 
arch/x86/kvm/hyperv.c +3 -3
index fff790a3f4ee..e5177c5c126c 100644
@@ -1015,7 +1015,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,
 		addr = gfn_to_hva(kvm, gfn);
 		if (kvm_is_error_hva(addr))
 			return 1;
-		kvm_x86_ops->patch_hypercall(vcpu, instructions);
+		kvm_x86_ops_patch_hypercall(vcpu, instructions);
 		((unsigned char *)instructions)[3] = 0xc3; /* ret */
 		if (__copy_to_user((void __user *)addr, instructions, 4))
 			return 1;
@@ -1600,7 +1600,7 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 	 * hypercall generates UD from non zero cpl and real mode
 	 * per HYPER-V spec
 	 */
-	if (kvm_x86_ops->get_cpl(vcpu) != 0 || !is_protmode(vcpu)) {
+	if (kvm_x86_ops_get_cpl(vcpu) != 0 || !is_protmode(vcpu)) {
 		kvm_queue_exception(vcpu, UD_VECTOR);
 		return 1;
 	}
@@ -1794,7 +1794,7 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
 	int i, nent = ARRAY_SIZE(cpuid_entries);
 
 	if (kvm_x86_ops->nested_get_evmcs_version)
-		evmcs_ver = kvm_x86_ops->nested_get_evmcs_version(vcpu);
+		evmcs_ver = kvm_x86_ops_nested_get_evmcs_version(vcpu);
 
 	/* Skip NESTED_FEATURES if eVMCS is not supported */
 	if (!evmcs_ver)
arch/x86/kvm/kvm_cache_regs.h +5 -5
index 1cc6c47dc77e..a8e80edd93cb 100644
@@ -41,7 +41,7 @@ static inline unsigned long kvm_register_read(struct kvm_vcpu *vcpu,
 					      enum kvm_reg reg)
 {
 	if (!test_bit(reg, (unsigned long *)&vcpu->arch.regs_avail))
-		kvm_x86_ops->cache_reg(vcpu, reg);
+		kvm_x86_ops_cache_reg(vcpu, reg);
 
 	return vcpu->arch.regs[reg];
 }
@@ -81,7 +81,7 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index)
 
 	if (!test_bit(VCPU_EXREG_PDPTR,
 		      (unsigned long *)&vcpu->arch.regs_avail))
-		kvm_x86_ops->cache_reg(vcpu, (enum kvm_reg)VCPU_EXREG_PDPTR);
+		kvm_x86_ops_cache_reg(vcpu, (enum kvm_reg)VCPU_EXREG_PDPTR);
 
 	return vcpu->arch.walk_mmu->pdptrs[index];
 }
@@ -90,7 +90,7 @@ static inline ulong kvm_read_cr0_bits(struct kvm_vcpu *vcpu, ulong mask)
 {
 	ulong tmask = mask & KVM_POSSIBLE_CR0_GUEST_BITS;
 	if (tmask & vcpu->arch.cr0_guest_owned_bits)
-		kvm_x86_ops->decache_cr0_guest_bits(vcpu);
+		kvm_x86_ops_decache_cr0_guest_bits(vcpu);
 	return vcpu->arch.cr0 & mask;
 }
 
@@ -103,14 +103,14 @@ static inline ulong kvm_read_cr4_bits(struct kvm_vcpu *vcpu, ulong mask)
 {
 	ulong tmask = mask & KVM_POSSIBLE_CR4_GUEST_BITS;
 	if (tmask & vcpu->arch.cr4_guest_owned_bits)
-		kvm_x86_ops->decache_cr4_guest_bits(vcpu);
+		kvm_x86_ops_decache_cr4_guest_bits(vcpu);
 	return vcpu->arch.cr4 & mask;
 }
 
 static inline ulong kvm_read_cr3(struct kvm_vcpu *vcpu)
 {
 	if (!test_bit(VCPU_EXREG_CR3, (ulong *)&vcpu->arch.regs_avail))
-		kvm_x86_ops->decache_cr3(vcpu);
+		kvm_x86_ops_decache_cr3(vcpu);
 	return vcpu->arch.cr3;
 }
 
arch/x86/kvm/lapic.c +14 -14
index 8675458c2205..d8dce6d40174 100644
@@ -453,7 +453,7 @@ static inline void apic_clear_irr(int vec, struct kvm_lapic *apic)
 	if (unlikely(vcpu->arch.apicv_active)) {
 		/* need to update RVI */
 		kvm_lapic_clear_vector(vec, apic->regs + APIC_IRR);
-		kvm_x86_ops->hwapic_irr_update(vcpu,
+		kvm_x86_ops_hwapic_irr_update(vcpu,
 				apic_find_highest_irr(apic));
 	} else {
 		apic->irr_pending = false;
@@ -478,7 +478,7 @@ static inline void apic_set_isr(int vec, struct kvm_lapic *apic)
 	 * just set SVI.
 	 */
 	if (unlikely(vcpu->arch.apicv_active))
-		kvm_x86_ops->hwapic_isr_update(vcpu, vec);
+		kvm_x86_ops_hwapic_isr_update(vcpu, vec);
 	else {
 		++apic->isr_count;
 		BUG_ON(apic->isr_count > MAX_APIC_VECTOR);
@@ -526,7 +526,7 @@ static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)
 	 * and must be left alone.
 	 */
 	if (unlikely(vcpu->arch.apicv_active))
-		kvm_x86_ops->hwapic_isr_update(vcpu,
+		kvm_x86_ops_hwapic_isr_update(vcpu,
 					       apic_find_highest_isr(apic));
 	else {
 		--apic->isr_count;
@@ -669,7 +669,7 @@ static int apic_has_interrupt_for_ppr(struct kvm_lapic *apic, u32 ppr)
 {
 	int highest_irr;
 	if (apic->vcpu->arch.apicv_active)
-		highest_irr = kvm_x86_ops->sync_pir_to_irr(apic->vcpu);
+		highest_irr = kvm_x86_ops_sync_pir_to_irr(apic->vcpu);
 	else
 		highest_irr = apic_find_highest_irr(apic);
 	if (highest_irr == -1 || (highest_irr & 0xF0) <= ppr)
@@ -1059,7 +1059,7 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
 		}
 
 		if (vcpu->arch.apicv_active)
-			kvm_x86_ops->deliver_posted_interrupt(vcpu, vector);
+			kvm_x86_ops_deliver_posted_interrupt(vcpu, vector);
 		else {
 			kvm_lapic_set_irr(vector, apic);
 
@@ -1704,7 +1704,7 @@ static void cancel_hv_timer(struct kvm_lapic *apic)
 {
 	WARN_ON(preemptible());
 	WARN_ON(!apic->lapic_timer.hv_timer_in_use);
-	kvm_x86_ops->cancel_hv_timer(apic->vcpu);
+	kvm_x86_ops_cancel_hv_timer(apic->vcpu);
 	apic->lapic_timer.hv_timer_in_use = false;
 }
 
@@ -1721,7 +1721,7 @@ static bool start_hv_timer(struct kvm_lapic *apic)
 	if (!ktimer->tscdeadline)
 		return false;
 
-	if (kvm_x86_ops->set_hv_timer(vcpu, ktimer->tscdeadline, &expired))
+	if (kvm_x86_ops_set_hv_timer(vcpu, ktimer->tscdeadline, &expired))
 		return false;
 
 	ktimer->hv_timer_in_use = true;
@@ -2141,7 +2141,7 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)
 		kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
 
 	if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE))
-		kvm_x86_ops->set_virtual_apic_mode(vcpu);
+		kvm_x86_ops_set_virtual_apic_mode(vcpu);
 
 	apic->base_address = apic->vcpu->arch.apic_base &
 			     MSR_IA32_APICBASE_BASE;
@@ -2204,9 +2204,9 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
 	vcpu->arch.pv_eoi.msr_val = 0;
 	apic_update_ppr(apic);
 	if (vcpu->arch.apicv_active) {
-		kvm_x86_ops->apicv_post_state_restore(vcpu);
-		kvm_x86_ops->hwapic_irr_update(vcpu, -1);
-		kvm_x86_ops->hwapic_isr_update(vcpu, -1);
+		kvm_x86_ops_apicv_post_state_restore(vcpu);
+		kvm_x86_ops_hwapic_irr_update(vcpu, -1);
+		kvm_x86_ops_hwapic_isr_update(vcpu, -1);
 	}
 
 	vcpu->arch.apic_arb_prio = 0;
@@ -2458,10 +2458,10 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 				1 : count_vectors(apic->regs + APIC_ISR);
 	apic->highest_isr_cache = -1;
 	if (vcpu->arch.apicv_active) {
-		kvm_x86_ops->apicv_post_state_restore(vcpu);
-		kvm_x86_ops->hwapic_irr_update(vcpu,
+		kvm_x86_ops_apicv_post_state_restore(vcpu);
+		kvm_x86_ops_hwapic_irr_update(vcpu,
 				apic_find_highest_irr(apic));
-		kvm_x86_ops->hwapic_isr_update(vcpu,
+		kvm_x86_ops_hwapic_isr_update(vcpu,
 				apic_find_highest_isr(apic));
 	}
 	kvm_make_request(KVM_REQ_EVENT, vcpu);
arch/x86/kvm/mmu.c +13 -13
index a10af9c87f8a..affeba031937 100644
@@ -283,7 +283,7 @@ static void kvm_flush_remote_tlbs_with_range(struct kvm *kvm,
 	int ret = -ENOTSUPP;
 
 	if (range && kvm_x86_ops->tlb_remote_flush_with_range)
-		ret = kvm_x86_ops->tlb_remote_flush_with_range(kvm, range);
+		ret = kvm_x86_ops_tlb_remote_flush_with_range(kvm, range);
 
 	if (ret)
 		kvm_flush_remote_tlbs(kvm);
@@ -1265,7 +1265,7 @@ static int mapping_level(struct kvm_vcpu *vcpu, gfn_t large_gfn,
 	if (host_level == PT_PAGE_TABLE_LEVEL)
 		return host_level;
 
-	max_level = min(kvm_x86_ops->get_lpage_level(), host_level);
+	max_level = min(kvm_x86_ops_get_lpage_level(), host_level);
 
 	for (level = PT_DIRECTORY_LEVEL; level <= max_level; ++level)
 		if (__mmu_gfn_lpage_is_disallowed(large_gfn, level, slot))
@@ -1719,7 +1719,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
 				gfn_t gfn_offset, unsigned long mask)
 {
 	if (kvm_x86_ops->enable_log_dirty_pt_masked)
-		kvm_x86_ops->enable_log_dirty_pt_masked(kvm, slot, gfn_offset,
+		kvm_x86_ops_enable_log_dirty_pt_masked(kvm, slot, gfn_offset,
 				mask);
 	else
 		kvm_mmu_write_protect_pt_masked(kvm, slot, gfn_offset, mask);
@@ -1735,7 +1735,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
 int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu)
 {
 	if (kvm_x86_ops->write_log_dirty)
-		return kvm_x86_ops->write_log_dirty(vcpu);
+		return kvm_x86_ops_write_log_dirty(vcpu);
 
 	return 0;
 }
@@ -2987,7 +2987,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
 	if (level > PT_PAGE_TABLE_LEVEL)
 		spte |= PT_PAGE_SIZE_MASK;
 	if (tdp_enabled)
-		spte |= kvm_x86_ops->get_mt_mask(vcpu, gfn,
+		spte |= kvm_x86_ops_get_mt_mask(vcpu, gfn,
 			kvm_is_mmio_pfn(pfn));
 
 	if (host_writable)
@@ -4258,7 +4258,7 @@ static bool fast_cr3_switch(struct kvm_vcpu *vcpu, gpa_t new_cr3,
 			kvm_make_request(KVM_REQ_LOAD_CR3, vcpu);
 			if (!skip_tlb_flush) {
 				kvm_make_request(KVM_REQ_MMU_SYNC, vcpu);
-				kvm_x86_ops->tlb_flush(vcpu, true);
+				kvm_x86_ops_tlb_flush(vcpu, true);
 			}
 
 			/*
@@ -4853,7 +4853,7 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, bool base_only)
 	union kvm_mmu_role role = kvm_calc_mmu_role_common(vcpu, base_only);
 
 	role.base.ad_disabled = (shadow_accessed_mask == 0);
-	role.base.level = kvm_x86_ops->get_tdp_level(vcpu);
+	role.base.level = kvm_x86_ops_get_tdp_level(vcpu);
 	role.base.direct = true;
 	role.base.gpte_is_8_bytes = true;
 
@@ -4875,7 +4875,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu)
 	context->sync_page = nonpaging_sync_page;
 	context->invlpg = nonpaging_invlpg;
 	context->update_pte = nonpaging_update_pte;
-	context->shadow_root_level = kvm_x86_ops->get_tdp_level(vcpu);
+	context->shadow_root_level = kvm_x86_ops_get_tdp_level(vcpu);
 	context->direct_map = true;
 	context->set_cr3 = kvm_x86_ops->set_tdp_cr3;
 	context->get_cr3 = get_cr3;
@@ -5132,7 +5132,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)
 	if (r)
 		goto out;
 	kvm_mmu_load_cr3(vcpu);
-	kvm_x86_ops->tlb_flush(vcpu, true);
+	kvm_x86_ops_tlb_flush(vcpu, true);
 out:
 	return r;
 }
@@ -5446,7 +5446,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code,
 	 * guest, with the exception of AMD Erratum 1096 which is unrecoverable.
 	 */
 	if (unlikely(insn && !insn_len)) {
-		if (!kvm_x86_ops->need_emulation_on_page_fault(vcpu))
+		if (!kvm_x86_ops_need_emulation_on_page_fault(vcpu))
 			return 1;
 	}
 
@@ -5492,7 +5492,7 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)
 		if (VALID_PAGE(mmu->prev_roots[i].hpa))
 			mmu->invlpg(vcpu, gva, mmu->prev_roots[i].hpa);
 
-	kvm_x86_ops->tlb_flush_gva(vcpu, gva);
+	kvm_x86_ops_tlb_flush_gva(vcpu, gva);
 	++vcpu->stat.invlpg;
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_invlpg);
@@ -5517,7 +5517,7 @@ void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid)
 	}
 
 	if (tlb_flush)
-		kvm_x86_ops->tlb_flush_gva(vcpu, gva);
+		kvm_x86_ops_tlb_flush_gva(vcpu, gva);
 
 	++vcpu->stat.invlpg;
 
@@ -5634,7 +5634,7 @@ static int alloc_mmu_pages(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu)
 	 * SVM's 32-bit NPT support, TDP paging doesn't use PAE paging and can
 	 * skip allocating the PDP table.
 	 */
-	if (tdp_enabled && kvm_x86_ops->get_tdp_level(vcpu) > PT32E_ROOT_LEVEL)
+	if (tdp_enabled && kvm_x86_ops_get_tdp_level(vcpu) > PT32E_ROOT_LEVEL)
 		return 0;
 
 	page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_DMA32);
arch/x86/kvm/mmu.h +2 -2
index 11f8ec89433b..c941980f58a4 100644
@@ -157,8 +157,8 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,
 				  unsigned pte_access, unsigned pte_pkey,
 				  unsigned pfec)
 {
-	int cpl = kvm_x86_ops->get_cpl(vcpu);
-	unsigned long rflags = kvm_x86_ops->get_rflags(vcpu);
+	int cpl = kvm_x86_ops_get_cpl(vcpu);
+	unsigned long rflags = kvm_x86_ops_get_rflags(vcpu);
 
 	/*
 	 * If CPL < 3, SMAP prevention are disabled if EFLAGS.AC = 1.
arch/x86/kvm/pmu.c +12 -12
index 46875bbd0419..363d3f86fb99 100644
@@ -183,7 +183,7 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)
 			  ARCH_PERFMON_EVENTSEL_CMASK |
 			  HSW_IN_TX |
 			  HSW_IN_TX_CHECKPOINTED))) {
-		config = kvm_x86_ops->pmu_ops->find_arch_event(pmc_to_pmu(pmc),
+		config = kvm_pmu_ops_find_arch_event(pmc_to_pmu(pmc),
 						      event_select,
 						      unit_mask);
 		if (config != PERF_COUNT_HW_MAX)
@@ -225,7 +225,7 @@ void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx)
 	}
 
 	pmc_reprogram_counter(pmc, PERF_TYPE_HARDWARE,
-			      kvm_x86_ops->pmu_ops->find_fixed_event(idx),
+			      kvm_pmu_ops_find_fixed_event(idx),
 			      !(en_field & 0x2), /* exclude user */
 			      !(en_field & 0x1), /* exclude kernel */
 			      pmi, false, false);
@@ -234,7 +234,7 @@ EXPORT_SYMBOL_GPL(reprogram_fixed_counter);
 
 void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx)
 {
-	struct kvm_pmc *pmc = kvm_x86_ops->pmu_ops->pmc_idx_to_pmc(pmu, pmc_idx);
+	struct kvm_pmc *pmc = kvm_pmu_ops_pmc_idx_to_pmc(pmu, pmc_idx);
 
 	if (!pmc)
 		return;
@@ -259,7 +259,7 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu)
 	bitmask = pmu->reprogram_pmi;
 
 	for_each_set_bit(bit, (unsigned long *)&bitmask, X86_PMC_IDX_MAX) {
-		struct kvm_pmc *pmc = kvm_x86_ops->pmu_ops->pmc_idx_to_pmc(pmu, bit);
+		struct kvm_pmc *pmc = kvm_pmu_ops_pmc_idx_to_pmc(pmu, bit);
 
 		if (unlikely(!pmc || !pmc->perf_event)) {
 			clear_bit(bit, (unsigned long *)&pmu->reprogram_pmi);
@@ -273,7 +273,7 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu)
 /* check if idx is a valid index to access PMU */
 int kvm_pmu_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx)
 {
-	return kvm_x86_ops->pmu_ops->is_valid_msr_idx(vcpu, idx);
+	return kvm_pmu_ops_is_valid_msr_idx(vcpu, idx);
 }
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx)
@@ -323,7 +323,7 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
 	if (is_vmware_backdoor_pmc(idx))
 		return kvm_pmu_rdpmc_vmware(vcpu, idx, data);
 
-	pmc = kvm_x86_ops->pmu_ops->msr_idx_to_pmc(vcpu, idx, &mask);
+	pmc = kvm_pmu_ops_msr_idx_to_pmc(vcpu, idx, &mask);
 	if (!pmc)
 		return 1;
 
@@ -339,17 +339,17 @@ void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu)
 
 bool kvm_pmu_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 {
-	return kvm_x86_ops->pmu_ops->is_valid_msr(vcpu, msr);
+	return kvm_pmu_ops_is_valid_msr(vcpu, msr);
 }
 
 int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data)
 {
-	return kvm_x86_ops->pmu_ops->get_msr(vcpu, msr, data);
+	return kvm_pmu_ops_get_msr(vcpu, msr, data);
 }
 
 int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
-	return kvm_x86_ops->pmu_ops->set_msr(vcpu, msr_info);
+	return kvm_pmu_ops_set_msr(vcpu, msr_info);
 }
 
 /* refresh PMU settings. This function generally is called when underlying
@@ -358,7 +358,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
  */
 void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 {
-	kvm_x86_ops->pmu_ops->refresh(vcpu);
+	kvm_pmu_ops_refresh(vcpu);
 }
 
 void kvm_pmu_reset(struct kvm_vcpu *vcpu)
@@ -366,7 +366,7 @@ void kvm_pmu_reset(struct kvm_vcpu *vcpu)
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
 	irq_work_sync(&pmu->irq_work);
-	kvm_x86_ops->pmu_ops->reset(vcpu);
+	kvm_pmu_ops_reset(vcpu);
 }
 
 void kvm_pmu_init(struct kvm_vcpu *vcpu)
@@ -374,7 +374,7 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
 	memset(pmu, 0, sizeof(*pmu));
-	kvm_x86_ops->pmu_ops->init(vcpu);
+	kvm_pmu_ops_init(vcpu);
 	init_irq_work(&pmu->irq_work, kvm_pmi_trigger_fn);
 	kvm_pmu_refresh(vcpu);
 }
arch/x86/kvm/pmu.h +1 -1
index a9c2f68a40cc..09e80e8ee21a 100644
@@ -78,7 +78,7 @@ static inline bool pmc_is_fixed(struct kvm_pmc *pmc)
 
 static inline bool pmc_is_enabled(struct kvm_pmc *pmc)
 {
-	return kvm_x86_ops->pmu_ops->pmc_is_enabled(pmc);
+	return kvm_pmu_ops_pmc_is_enabled(pmc);
 }
 
 /* returns general purpose PMC with the specified MSR. Note that it can be
arch/x86/kvm/trace.h +2 -2
index 7c741a0c5f80..f9994efc4c44 100644
@@ -240,7 +240,7 @@ TRACE_EVENT(kvm_exit,
 		__entry->guest_rip	= kvm_rip_read(vcpu);
 		__entry->isa            = isa;
 		__entry->vcpu_id        = vcpu->vcpu_id;
-		kvm_x86_ops->get_exit_info(vcpu, &__entry->info1,
+		kvm_x86_ops_get_exit_info(vcpu, &__entry->info1,
 					   &__entry->info2);
 	),
 
@@ -744,7 +744,7 @@ TRACE_EVENT(kvm_emulate_insn,
 		),
 
 	TP_fast_assign(
-		__entry->csbase = kvm_x86_ops->get_segment_base(vcpu, VCPU_SREG_CS);
+		__entry->csbase = kvm_x86_ops_get_segment_base(vcpu, VCPU_SREG_CS);
 		__entry->len = vcpu->arch.emulate_ctxt.fetch.ptr
 			       - vcpu->arch.emulate_ctxt.fetch.data;
 		__entry->rip = vcpu->arch.emulate_ctxt._eip - __entry->len;
arch/x86/kvm/vmx/pmu_intel.c +1 -1
index 5ef36a75c31e..bfa765842772 100644
@@ -305,7 +305,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->global_ovf_ctrl_mask = pmu->global_ctrl_mask
 			& ~(MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF |
 			    MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD);
-	if (kvm_x86_ops->pt_supported())
+	if (kvm_x86_ops_pt_supported())
 		pmu->global_ovf_ctrl_mask &=
 				~MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI;
 
arch/x86/kvm/x86.c +152 -152

1,122 lines not on this page · raw message

arch/x86/kvm/x86.h +1 -1
index b5274e2a53cf..6e4e2d75f402 100644
@@ -96,7 +96,7 @@ static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu)
 
 	if (!is_long_mode(vcpu))
 		return false;
-	kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l);
+	kvm_x86_ops_get_cs_db_l_bits(vcpu, &cs_db, &cs_l);
 	return cs_l;
 }
 
maintainer
#26 lore raw

Re: [PATCH 10/17] KVM: monolithic: x86: use the external functions instead of kvm_x86_ops

Quoted 4 lines · Andrea Arcangeli

Now that the new methods are plugged in and they are functional use them instead of invoking the pointer to functions through kvm_x86_ops.

Quoted 14 lines
---
 arch/x86/include/asm/kvm_host.h |  10 +-
 arch/x86/kvm/cpuid.c            |  22 +--
 arch/x86/kvm/hyperv.c           |   6 +-
 arch/x86/kvm/kvm_cache_regs.h   |  10 +-
 arch/x86/kvm/lapic.c            |  28 +--
 arch/x86/kvm/mmu.c              |  26 +--
 arch/x86/kvm/mmu.h              |   4 +-
 arch/x86/kvm/pmu.c              |  24 +--
 arch/x86/kvm/pmu.h              |   2 +-
 arch/x86/kvm/trace.h            |   4 +-
 arch/x86/kvm/vmx/pmu_intel.c    |   2 +-
 arch/x86/kvm/x86.c              | 304 ++++++++++++++++----------------
 arch/x86/kvm/x86.h              |   2 +-

Let's make the prefix kvm_x86_ instead of kvm_x86_ops_.

Paolo

#27 lore raw

PATCH 11/17 KVM: monolithic: x86: remove exports

In reply to Andrea Arcangeli

The exports would be duplicated across kvm-amd and kvm-intel if they're kept, this cleanup various harmless warnings about it.

---
 arch/x86/kvm/cpuid.c    |   5 --
 arch/x86/kvm/hyperv.c   |   2 -
 arch/x86/kvm/irq.c      |   4 --
 arch/x86/kvm/irq_comm.c |   2 -
 arch/x86/kvm/lapic.c    |  16 -------
 arch/x86/kvm/mmu.c      |  24 ----------
 arch/x86/kvm/mtrr.c     |   2 -
 arch/x86/kvm/pmu.c      |   3 --
 arch/x86/kvm/x86.c      | 101 ----------------------------------------
 9 files changed, 159 deletions(-)
arch/x86/kvm/cpuid.c +0 -5
index d8d840fc703e..e4fa9c629b0b 100644
@@ -50,7 +50,6 @@ bool kvm_mpx_supported(void)
 	return ((host_xcr0 & (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_BNDCSR))
 		 && kvm_x86_ops_mpx_supported());
 }
-EXPORT_SYMBOL_GPL(kvm_mpx_supported);
 
 u64 kvm_supported_xcr0(void)
 {
@@ -192,7 +191,6 @@ int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu)
 not_found:
 	return 36;
 }
-EXPORT_SYMBOL_GPL(cpuid_query_maxphyaddr);
 
 /* when an old userspace process fills a new kernel module */
 int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu,
@@ -960,7 +958,6 @@ struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu,
 	}
 	return best;
 }
-EXPORT_SYMBOL_GPL(kvm_find_cpuid_entry);
 
 /*
  * If no match is found, check whether we exceed the vCPU's limit
@@ -1011,7 +1008,6 @@ bool kvm_cpuid(struct kvm_vcpu *vcpu, u32 *eax, u32 *ebx,
 	trace_kvm_cpuid(function, *eax, *ebx, *ecx, *edx, entry_found);
 	return entry_found;
 }
-EXPORT_SYMBOL_GPL(kvm_cpuid);
 
 int kvm_emulate_cpuid(struct kvm_vcpu *vcpu)
 {
@@ -1029,4 +1025,3 @@ int kvm_emulate_cpuid(struct kvm_vcpu *vcpu)
 	kvm_rdx_write(vcpu, edx);
 	return kvm_skip_emulated_instruction(vcpu);
 }
-EXPORT_SYMBOL_GPL(kvm_emulate_cpuid);
arch/x86/kvm/hyperv.c +0 -2
index e5177c5c126c..d875d60339c7 100644
@@ -711,7 +711,6 @@ bool kvm_hv_assist_page_enabled(struct kvm_vcpu *vcpu)
 		return false;
 	return vcpu->arch.pv_eoi.msr_val & KVM_MSR_ENABLED;
 }
-EXPORT_SYMBOL_GPL(kvm_hv_assist_page_enabled);
 
 bool kvm_hv_get_assist_page(struct kvm_vcpu *vcpu,
 			    struct hv_vp_assist_page *assist_page)
@@ -721,7 +720,6 @@ bool kvm_hv_get_assist_page(struct kvm_vcpu *vcpu,
 	return !kvm_read_guest_cached(vcpu->kvm, &vcpu->arch.pv_eoi.data,
 				      assist_page, sizeof(*assist_page));
 }
-EXPORT_SYMBOL_GPL(kvm_hv_get_assist_page);
 
 static void stimer_prepare_msg(struct kvm_vcpu_hv_stimer *stimer)
 {
arch/x86/kvm/irq.c +0 -4
index e330e7d125f7..ba4300f36a32 100644
@@ -26,7 +26,6 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 
 	return 0;
 }
-EXPORT_SYMBOL(kvm_cpu_has_pending_timer);
 
 /*
  * check if there is a pending userspace external interrupt
@@ -109,7 +108,6 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v)
 
 	return kvm_apic_has_interrupt(v) != -1;	/* LAPIC */
 }
-EXPORT_SYMBOL_GPL(kvm_cpu_has_interrupt);
 
 /*
  * Read pending interrupt(from non-APIC source)
@@ -146,14 +144,12 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v)
 
 	return kvm_get_apic_interrupt(v);	/* APIC */
 }
-EXPORT_SYMBOL_GPL(kvm_cpu_get_interrupt);
 
 void kvm_inject_pending_timer_irqs(struct kvm_vcpu *vcpu)
 {
 	if (lapic_in_kernel(vcpu))
 		kvm_inject_apic_timer_irqs(vcpu);
 }
-EXPORT_SYMBOL_GPL(kvm_inject_pending_timer_irqs);
 
 void __kvm_migrate_timers(struct kvm_vcpu *vcpu)
 {
arch/x86/kvm/irq_comm.c +0 -2
index 8ecd48d31800..64a13d5fcc9f 100644
@@ -122,7 +122,6 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 	irq->level = 1;
 	irq->shorthand = 0;
 }
-EXPORT_SYMBOL_GPL(kvm_set_msi_irq);
 
 static inline bool kvm_msi_route_invalid(struct kvm *kvm,
 		struct kvm_kernel_irq_routing_entry *e)
@@ -346,7 +345,6 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq,
 
 	return r == 1;
 }
-EXPORT_SYMBOL_GPL(kvm_intr_is_single_vcpu);
 
 #define IOAPIC_ROUTING_ENTRY(irq) \
 	{ .gsi = irq, .type = KVM_IRQ_ROUTING_IRQCHIP,	\
arch/x86/kvm/lapic.c +0 -16
index d8dce6d40174..d32840fe45f7 100644
@@ -122,7 +122,6 @@ bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu)
 {
 	return pi_inject_timer && kvm_vcpu_apicv_active(vcpu);
 }
-EXPORT_SYMBOL_GPL(kvm_can_post_timer_interrupt);
 
 static bool kvm_use_posted_timer_interrupt(struct kvm_vcpu *vcpu)
 {
@@ -412,7 +411,6 @@ bool __kvm_apic_update_irr(u32 *pir, void *regs, int *max_irr)
 	return ((max_updated_irr != -1) &&
 		(max_updated_irr == *max_irr));
 }
-EXPORT_SYMBOL_GPL(__kvm_apic_update_irr);
 
 bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir, int *max_irr)
 {
@@ -420,7 +418,6 @@ bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir, int *max_irr)
 
 	return __kvm_apic_update_irr(pir, apic->regs, max_irr);
 }
-EXPORT_SYMBOL_GPL(kvm_apic_update_irr);
 
 static inline int apic_search_irr(struct kvm_lapic *apic)
 {
@@ -544,7 +541,6 @@ int kvm_lapic_find_highest_irr(struct kvm_vcpu *vcpu)
 	 */
 	return apic_find_highest_irr(vcpu->arch.apic);
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_find_highest_irr);
 
 static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
 			     int vector, int level, int trig_mode,
@@ -712,7 +708,6 @@ void kvm_apic_update_ppr(struct kvm_vcpu *vcpu)
 {
 	apic_update_ppr(vcpu->arch.apic);
 }
-EXPORT_SYMBOL_GPL(kvm_apic_update_ppr);
 
 static void apic_set_tpr(struct kvm_lapic *apic, u32 tpr)
 {
@@ -823,7 +818,6 @@ bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
 		return false;
 	}
 }
-EXPORT_SYMBOL_GPL(kvm_apic_match_dest);
 
 int kvm_vector_to_index(u32 vector, u32 dest_vcpus,
 		       const unsigned long *bitmap, u32 bitmap_size)
@@ -1196,7 +1190,6 @@ void kvm_apic_set_eoi_accelerated(struct kvm_vcpu *vcpu, int vector)
 	kvm_ioapic_send_eoi(apic, vector);
 	kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
 }
-EXPORT_SYMBOL_GPL(kvm_apic_set_eoi_accelerated);
 
 static void apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high)
 {
@@ -1355,7 +1348,6 @@ int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len,
 	}
 	return 0;
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_reg_read);
 
 static int apic_mmio_in_range(struct kvm_lapic *apic, gpa_t addr)
 {
@@ -1533,7 +1525,6 @@ void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)
 	if (lapic_timer_int_injected(vcpu))
 		__kvm_wait_lapic_expire(vcpu);
 }
-EXPORT_SYMBOL_GPL(kvm_wait_lapic_expire);
 
 static void kvm_apic_inject_pending_timer_irqs(struct kvm_lapic *apic)
 {
@@ -1698,7 +1689,6 @@ bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu)
 
 	return vcpu->arch.apic->lapic_timer.hv_timer_in_use;
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_hv_timer_in_use);
 
 static void cancel_hv_timer(struct kvm_lapic *apic)
 {
@@ -1799,13 +1789,11 @@ void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu)
 out:
 	preempt_enable();
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_expired_hv_timer);
 
 void kvm_lapic_switch_to_hv_timer(struct kvm_vcpu *vcpu)
 {
 	restart_apic_timer(vcpu->arch.apic);
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_switch_to_hv_timer);
 
 void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu)
 {
@@ -1817,7 +1805,6 @@ void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu)
 		start_sw_timer(apic);
 	preempt_enable();
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_switch_to_sw_timer);
 
 void kvm_lapic_restart_hv_timer(struct kvm_vcpu *vcpu)
 {
@@ -1987,7 +1974,6 @@ int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)
 
 	return ret;
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_reg_write);
 
 static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
 			    gpa_t address, int len, const void *data)
@@ -2026,7 +2012,6 @@ void kvm_lapic_set_eoi(struct kvm_vcpu *vcpu)
 {
 	kvm_lapic_reg_write(vcpu->arch.apic, APIC_EOI, 0);
 }
-EXPORT_SYMBOL_GPL(kvm_lapic_set_eoi);
 
 /* emulate APIC access in a trap manner */
 void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset)
@@ -2041,7 +2026,6 @@ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset)
 	/* TODO: optimize to just emulate side effect w/o one more write */
 	kvm_lapic_reg_write(vcpu->arch.apic, offset, val);
 }
-EXPORT_SYMBOL_GPL(kvm_apic_write_nodecode);
 
 void kvm_free_lapic(struct kvm_vcpu *vcpu)
 {
arch/x86/kvm/mmu.c +0 -24
index affeba031937..bb2d05eb9271 100644
@@ -308,7 +308,6 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask, u64 mmio_value, u64 access_mask)
 	shadow_mmio_mask = mmio_mask | SPTE_SPECIAL_MASK;
 	shadow_mmio_access_mask = access_mask;
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask);
 
 static bool is_mmio_spte(u64 spte)
 {
@@ -474,7 +473,6 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 	shadow_acc_track_mask = acc_track_mask;
 	shadow_me_mask = me_mask;
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
 
 static u8 kvm_get_shadow_phys_bits(void)
 {
@@ -1702,7 +1700,6 @@ void kvm_mmu_clear_dirty_pt_masked(struct kvm *kvm,
 		mask &= mask - 1;
 	}
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_clear_dirty_pt_masked);
 
 /**
  * kvm_arch_mmu_enable_log_dirty_pt_masked - enable dirty logging for selected
@@ -2853,7 +2850,6 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn)
 
 	return r;
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_unprotect_page);
 
 static void kvm_unsync_page(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)
 {
@@ -3621,7 +3617,6 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,
 	kvm_mmu_commit_zap_page(vcpu->kvm, &invalid_list);
 	spin_unlock(&vcpu->kvm->mmu_lock);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_free_roots);
 
 static int mmu_check_root(struct kvm_vcpu *vcpu, gfn_t root_gfn)
 {
@@ -3846,7 +3841,6 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu)
 	kvm_mmu_audit(vcpu, AUDIT_POST_SYNC);
 	spin_unlock(&vcpu->kvm->mmu_lock);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_sync_roots);
 
 static gpa_t nonpaging_gva_to_gpa(struct kvm_vcpu *vcpu, gva_t vaddr,
 				  u32 access, struct x86_exception *exception)
@@ -4114,7 +4108,6 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code,
 	}
 	return r;
 }
-EXPORT_SYMBOL_GPL(kvm_handle_page_fault);
 
 static bool
 check_hugepage_cache_consistency(struct kvm_vcpu *vcpu, gfn_t gfn, int level)
@@ -4294,7 +4287,6 @@ void kvm_mmu_new_cr3(struct kvm_vcpu *vcpu, gpa_t new_cr3, bool skip_tlb_flush)
 	__kvm_mmu_new_cr3(vcpu, new_cr3, kvm_mmu_calc_root_page_role(vcpu),
 			  skip_tlb_flush);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_new_cr3);
 
 static unsigned long get_cr3(struct kvm_vcpu *vcpu)
 {
@@ -4534,7 +4526,6 @@ reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, struct kvm_mmu *context)
 	}
 
 }
-EXPORT_SYMBOL_GPL(reset_shadow_zero_bits_mask);
 
 static inline bool boot_cpu_is_amd(void)
 {
@@ -4954,7 +4945,6 @@ void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu)
 	context->mmu_role.as_u64 = new_role.as_u64;
 	reset_shadow_zero_bits_mask(vcpu, context);
 }
-EXPORT_SYMBOL_GPL(kvm_init_shadow_mmu);
 
 static union kvm_mmu_role
 kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *vcpu, bool accessed_dirty,
@@ -5018,7 +5008,6 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly,
 	reset_rsvds_bits_mask_ept(vcpu, context, execonly);
 	reset_ept_shadow_zero_bits_mask(vcpu, context, execonly);
 }
-EXPORT_SYMBOL_GPL(kvm_init_shadow_ept_mmu);
 
 static void init_kvm_softmmu(struct kvm_vcpu *vcpu)
 {
@@ -5098,7 +5087,6 @@ void kvm_init_mmu(struct kvm_vcpu *vcpu, bool reset_roots)
 	else
 		init_kvm_softmmu(vcpu);
 }
-EXPORT_SYMBOL_GPL(kvm_init_mmu);
 
 static union kvm_mmu_page_role
 kvm_mmu_calc_root_page_role(struct kvm_vcpu *vcpu)
@@ -5118,7 +5106,6 @@ void kvm_mmu_reset_context(struct kvm_vcpu *vcpu)
 	kvm_mmu_unload(vcpu);
 	kvm_init_mmu(vcpu, true);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_reset_context);
 
 int kvm_mmu_load(struct kvm_vcpu *vcpu)
 {
@@ -5136,7 +5123,6 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)
 out:
 	return r;
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_load);
 
 void kvm_mmu_unload(struct kvm_vcpu *vcpu)
 {
@@ -5145,7 +5131,6 @@ void kvm_mmu_unload(struct kvm_vcpu *vcpu)
 	kvm_mmu_free_roots(vcpu, &vcpu->arch.guest_mmu, KVM_MMU_ROOTS_ALL);
 	WARN_ON(VALID_PAGE(vcpu->arch.guest_mmu.root_hpa));
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_unload);
 
 static void mmu_pte_write_new_pte(struct kvm_vcpu *vcpu,
 				  struct kvm_mmu_page *sp, u64 *spte,
@@ -5357,7 +5342,6 @@ int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva)
 
 	return r;
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_unprotect_page_virt);
 
 static int make_mmu_pages_available(struct kvm_vcpu *vcpu)
 {
@@ -5464,7 +5448,6 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code,
 		BUG();
 	}
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_page_fault);
 
 void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)
 {
@@ -5495,7 +5478,6 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva)
 	kvm_x86_ops_tlb_flush_gva(vcpu, gva);
 	++vcpu->stat.invlpg;
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_invlpg);
 
 void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid)
 {
@@ -5527,19 +5509,16 @@ void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid)
 	 * for them.
 	 */
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_invpcid_gva);
 
 void kvm_enable_tdp(void)
 {
 	tdp_enabled = true;
 }
-EXPORT_SYMBOL_GPL(kvm_enable_tdp);
 
 void kvm_disable_tdp(void)
 {
 	tdp_enabled = false;
 }
-EXPORT_SYMBOL_GPL(kvm_disable_tdp);
 
 
 /* The return value indicates if tlb flush on all vcpus is needed. */
@@ -5920,7 +5899,6 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
 		kvm_flush_remote_tlbs_with_address(kvm, memslot->base_gfn,
 				memslot->npages);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_slot_leaf_clear_dirty);
 
 void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm,
 					struct kvm_memory_slot *memslot)
@@ -5939,7 +5917,6 @@ void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm,
 		kvm_flush_remote_tlbs_with_address(kvm, memslot->base_gfn,
 				memslot->npages);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_slot_largepage_remove_write_access);
 
 void kvm_mmu_slot_set_dirty(struct kvm *kvm,
 			    struct kvm_memory_slot *memslot)
@@ -5957,7 +5934,6 @@ void kvm_mmu_slot_set_dirty(struct kvm *kvm,
 		kvm_flush_remote_tlbs_with_address(kvm, memslot->base_gfn,
 				memslot->npages);
 }
-EXPORT_SYMBOL_GPL(kvm_mmu_slot_set_dirty);
 
 static void __kvm_mmu_zap_all(struct kvm *kvm, bool mmio_only)
 {
arch/x86/kvm/mtrr.c +0 -2
index 25ce3edd1872..477f7141f793 100644
@@ -91,7 +91,6 @@ bool kvm_mtrr_valid(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 
 	return true;
 }
-EXPORT_SYMBOL_GPL(kvm_mtrr_valid);
 
 static bool mtrr_is_enabled(struct kvm_mtrr *mtrr_state)
 {
@@ -686,7 +685,6 @@ u8 kvm_mtrr_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn)
 
 	return type;
 }
-EXPORT_SYMBOL_GPL(kvm_mtrr_get_guest_memory_type);
 
 bool kvm_mtrr_check_gfn_range_consistency(struct kvm_vcpu *vcpu, gfn_t gfn,
 					  int page_num)
arch/x86/kvm/pmu.c +0 -3
index 363d3f86fb99..f87546e1d97d 100644
@@ -200,7 +200,6 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)
 			      (eventsel & HSW_IN_TX),
 			      (eventsel & HSW_IN_TX_CHECKPOINTED));
 }
-EXPORT_SYMBOL_GPL(reprogram_gp_counter);
 
 void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx)
 {
@@ -230,7 +229,6 @@ void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx)
 			      !(en_field & 0x1), /* exclude kernel */
 			      pmi, false, false);
 }
-EXPORT_SYMBOL_GPL(reprogram_fixed_counter);
 
 void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx)
 {
@@ -248,7 +246,6 @@ void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx)
 		reprogram_fixed_counter(pmc, ctrl, idx);
 	}
 }
-EXPORT_SYMBOL_GPL(reprogram_counter);
 
 void kvm_pmu_handle_event(struct kvm_vcpu *vcpu)
 {
arch/x86/kvm/x86.c +0 -101

546 lines not on this page · raw message

#28 lore raw

PATCH 12/17 KVM: monolithic: remove exports from KVM common code

In reply to Andrea Arcangeli

The exports would be duplicated across kvm-amd and kvm-intel if they're kept and that causes various harmless.

The warnings aren't particularly concerning because the two modules can't load at the same time, but it's cleaner to remove the warnings by removing the exports.

This commit might break non-x86 archs, but it should be simple to make them monolithic too (if they're not already).

In the unlikely case there's a legit reason not to go monolithic in any arch and to keep kvm.ko around, we'll need a way to retain the exports. In which case this commit would need to be reversed and the exports in the kvm common code should then be done only conditionally to a new opt-in per-arch CONFIG option.

The following warning remains for now to be able to load the kvmgt driver. These remaining warnings can be handled later.

WARNING: arch/x86/kvm/kvm-amd: 'kvm_get_kvm' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_put_kvm' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'gfn_to_memslot' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_is_visible_gfn' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'gfn_to_pfn' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_read_guest' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_write_guest' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_slot_page_track_add_page' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_slot_page_track_remove_page' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_page_track_register_notifier' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko
WARNING: arch/x86/kvm/kvm-amd: 'kvm_page_track_unregister_notifier' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko a

---
 virt/kvm/eventfd.c  |  1 -
 virt/kvm/kvm_main.c | 65 ---------------------------------------------
 2 files changed, 66 deletions(-)
virt/kvm/eventfd.c +0 -1
index 67b6fc153e9c..4c1a8abd1458 100644
@@ -462,7 +462,6 @@ bool kvm_irq_has_notifier(struct kvm *kvm, unsigned irqchip, unsigned pin)
 
 	return false;
 }
-EXPORT_SYMBOL_GPL(kvm_irq_has_notifier);
 
 void kvm_notify_acked_gsi(struct kvm *kvm, int gsi)
 {
virt/kvm/kvm_main.c +0 -65

438 lines not on this page · raw message

#29 lore raw

PATCH 13/17 KVM: monolithic: x86: drop the kvm_pmu_ops structure

In reply to Andrea Arcangeli

Cleanup after this was finally left fully unused.

---
 arch/x86/include/asm/kvm_host.h |  3 ---
 arch/x86/kvm/pmu.h              | 19 -------------------
 arch/x86/kvm/pmu_amd.c          | 15 ---------------
 arch/x86/kvm/svm.c              |  1 -
 arch/x86/kvm/vmx/pmu_intel.c    | 15 ---------------
 arch/x86/kvm/vmx/vmx.c          |  2 --
 6 files changed, 55 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -3
index bd5f4f900288..000d4e5a5664 100644
@@ -1160,9 +1160,6 @@ struct kvm_x86_ops {
 					   gfn_t offset, unsigned long mask);
 	int (*write_log_dirty)(struct kvm_vcpu *vcpu);
 
-	/* pmu operations of sub-arch */
-	const struct kvm_pmu_ops *pmu_ops;
-
 	/*
 	 * Architecture specific hooks for vCPU blocking due to
 	 * HLT instruction.
arch/x86/kvm/pmu.h +0 -19
index 09e80e8ee21a..513366c8b794 100644
@@ -21,23 +21,6 @@ struct kvm_event_hw_type_mapping {
 
 #include "pmu_ops.h"
 
-struct kvm_pmu_ops {
-	unsigned (*find_arch_event)(struct kvm_pmu *pmu, u8 event_select,
-				    u8 unit_mask);
-	unsigned (*find_fixed_event)(int idx);
-	bool (*pmc_is_enabled)(struct kvm_pmc *pmc);
-	struct kvm_pmc *(*pmc_idx_to_pmc)(struct kvm_pmu *pmu, int pmc_idx);
-	struct kvm_pmc *(*msr_idx_to_pmc)(struct kvm_vcpu *vcpu, unsigned idx,
-					  u64 *mask);
-	int (*is_valid_msr_idx)(struct kvm_vcpu *vcpu, unsigned idx);
-	bool (*is_valid_msr)(struct kvm_vcpu *vcpu, u32 msr);
-	int (*get_msr)(struct kvm_vcpu *vcpu, u32 msr, u64 *data);
-	int (*set_msr)(struct kvm_vcpu *vcpu, struct msr_data *msr_info);
-	void (*refresh)(struct kvm_vcpu *vcpu);
-	void (*init)(struct kvm_vcpu *vcpu);
-	void (*reset)(struct kvm_vcpu *vcpu);
-};
-
 static inline u64 pmc_bitmask(struct kvm_pmc *pmc)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
@@ -124,6 +107,4 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
-extern struct kvm_pmu_ops intel_pmu_ops;
-extern struct kvm_pmu_ops amd_pmu_ops;
 #endif /* __KVM_X86_PMU_H */
arch/x86/kvm/pmu_amd.c +0 -15
index 12d1fa3ba35a..0e01b48b3b0b 100644
@@ -302,18 +302,3 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu)
 }
 
 #include "pmu_amd_ops.c"
-
-struct kvm_pmu_ops amd_pmu_ops = {
-	.find_arch_event = amd_find_arch_event,
-	.find_fixed_event = amd_find_fixed_event,
-	.pmc_is_enabled = amd_pmc_is_enabled,
-	.pmc_idx_to_pmc = amd_pmc_idx_to_pmc,
-	.msr_idx_to_pmc = amd_msr_idx_to_pmc,
-	.is_valid_msr_idx = amd_is_valid_msr_idx,
-	.is_valid_msr = amd_is_valid_msr,
-	.get_msr = amd_pmu_get_msr,
-	.set_msr = amd_pmu_set_msr,
-	.refresh = amd_pmu_refresh,
-	.init = amd_pmu_init,
-	.reset = amd_pmu_reset,
-};
arch/x86/kvm/svm.c +0 -1
index 5a48beb58083..ca9fd1762519 100644
@@ -7298,7 +7298,6 @@ static struct kvm_x86_ops svm_x86_ops __ro_after_init = {
 
 	.sched_in = svm_sched_in,
 
-	.pmu_ops = &amd_pmu_ops,
 	.deliver_posted_interrupt = svm_deliver_avic_intr,
 	.dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt,
 	.update_pi_irte = svm_update_pi_irte,
arch/x86/kvm/vmx/pmu_intel.c +0 -15
index bfa765842772..e7c1253f47a2 100644
@@ -359,18 +359,3 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu)
 }
 
 #include "pmu_intel_ops.c"
-
-struct kvm_pmu_ops intel_pmu_ops = {
-	.find_arch_event = intel_find_arch_event,
-	.find_fixed_event = intel_find_fixed_event,
-	.pmc_is_enabled = intel_pmc_is_enabled,
-	.pmc_idx_to_pmc = intel_pmc_idx_to_pmc,
-	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
-	.is_valid_msr_idx = intel_is_valid_msr_idx,
-	.is_valid_msr = intel_is_valid_msr,
-	.get_msr = intel_pmu_get_msr,
-	.set_msr = intel_pmu_set_msr,
-	.refresh = intel_pmu_refresh,
-	.init = intel_pmu_init,
-	.reset = intel_pmu_reset,
-};
arch/x86/kvm/vmx/vmx.c +0 -2
index 09e6a477e06f..ff46008dc514 100644
@@ -7779,8 +7779,6 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = {
 	.pre_block = vmx_pre_block,
 	.post_block = vmx_post_block,
 
-	.pmu_ops = &intel_pmu_ops,
-
 	.update_pi_irte = vmx_update_pi_irte,
 
 #ifdef CONFIG_X86_64
maintainer
#30 lore raw

Re: [PATCH 13/17] KVM: monolithic: x86: drop the kvm_pmu_ops structure

Quoted 11 lines · Andrea Arcangeli

Cleanup after this was finally left fully unused.

---
 arch/x86/include/asm/kvm_host.h |  3 ---
 arch/x86/kvm/pmu.h              | 19 -------------------
 arch/x86/kvm/pmu_amd.c          | 15 ---------------
 arch/x86/kvm/svm.c              |  1 -
 arch/x86/kvm/vmx/pmu_intel.c    | 15 ---------------
 arch/x86/kvm/vmx/vmx.c          |  2 --
 6 files changed, 55 deletions(-)

Is there any reason not to do the same for kvm_x86_ops?

(As an aside, patch 2 is not copying over the comments in the struct kvm_x86_ops declarations. Granted there aren't many, but we should not lose the few that exist).

Paolo

#31 lore raw

Re: [PATCH 13/17] KVM: monolithic: x86: drop the kvm_pmu_ops structure

Quoted 13 lines · Paolo Bonzini
Quoted 11 lines · Andrea Arcangeli

Cleanup after this was finally left fully unused.

---
 arch/x86/include/asm/kvm_host.h |  3 ---
 arch/x86/kvm/pmu.h              | 19 -------------------
 arch/x86/kvm/pmu_amd.c          | 15 ---------------
 arch/x86/kvm/svm.c              |  1 -
 arch/x86/kvm/vmx/pmu_intel.c    | 15 ---------------
 arch/x86/kvm/vmx/vmx.c          |  2 --
 6 files changed, 55 deletions(-)
Quoted 1 line

Is there any reason not to do the same for kvm_x86_ops?

This was covered in the commit header of patch 2:

To reduce the rejecting parts while tracking upstream, this doesn't attempt to entirely remove the kvm_x86_ops structure yet, that is meant for a later cleanup. The pmu ops have been already cleaned up in this patchset because it was left completely unused right after the conversion from pointer to functions to external functions.

Lot more patches are needed to get rid of kvm_x86_ops entirely because there are lots of places checking the actual value of the method before making the indirect call. I tried to start that, but then it got into potentially heavily rejecting territory, so I thought it was simpler to start with what I had, considering from a performance standpoint it's optimal already as far as retpolines are concerned.

Quoted 3 lines

(As an aside, patch 2 is not copying over the comments in the struct kvm_x86_ops declarations. Granted there aren't many, but we should not lose the few that exist).

Yes sorry, this was actually unintentional and the comment need to be retained in the header declaration.

Thanks,
Andrea

maintainer
#32 lore raw

Re: [PATCH 13/17] KVM: monolithic: x86: drop the kvm_pmu_ops structure

Quoted 1 line · Andrea Arcangeli

This was covered in the commit header of patch 2:

Oops, sorry.

Quoted 6 lines

Lot more patches are needed to get rid of kvm_x86_ops entirely because there are lots of places checking the actual value of the method before making the indirect call. I tried to start that, but then it got into potentially heavily rejecting territory, so I thought it was simpler to start with what I had, considering from a performance standpoint it's optimal already as far as retpolines are concerned.

The performance may be good enough, but the maintainability is bad. Let's make a list of function pointers that are checked, and function pointers that are written at init time.

For the former, it should be possible to make them __weak symbols so that they are NULL if undeclared. For the latter, module parameters can be made extern and then you can have checks like kvm_x86_has_...() in inline functions in a header file.

Paolo

#33 lore raw

PATCH 14/17 KVM: monolithic: x86: inline more exit handlers in vmx.c

In reply to Andrea Arcangeli

They can be called directly more efficiently, so we can as well mark some of them inline in case gcc doesn't decide to inline them.

---
 arch/x86/kvm/vmx/vmx.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/vmx.c +3 -3
index ff46008dc514..a6e597025011 100644
@@ -4588,7 +4588,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-static int handle_external_interrupt(struct kvm_vcpu *vcpu)
+static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu)
 {
 	++vcpu->stat.irq_exits;
 	return 1;
@@ -4860,7 +4860,7 @@ static void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val)
 	vmcs_writel(GUEST_DR7, val);
 }
 
-static int handle_cpuid(struct kvm_vcpu *vcpu)
+static __always_inline int handle_cpuid(struct kvm_vcpu *vcpu)
 {
 	return kvm_emulate_cpuid(vcpu);
 }
@@ -4891,7 +4891,7 @@ static int handle_interrupt_window(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
-static int handle_halt(struct kvm_vcpu *vcpu)
+static __always_inline int handle_halt(struct kvm_vcpu *vcpu)
 {
 	return kvm_emulate_halt(vcpu);
 }
maintainer
#34 lore raw

Re: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c

Quoted 2 lines · Andrea Arcangeli

They can be called directly more efficiently, so we can as well mark some of them inline in case gcc doesn't decide to inline them.

What is the output of size(1) before and after?

Paolo

Quoted 37 lines
---
 arch/x86/kvm/vmx/vmx.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/vmx.c +3 -3
index ff46008dc514..a6e597025011 100644
@@ -4588,7 +4588,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-static int handle_external_interrupt(struct kvm_vcpu *vcpu)
+static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu)
 {
 	++vcpu->stat.irq_exits;
 	return 1;
@@ -4860,7 +4860,7 @@ static void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val)
 	vmcs_writel(GUEST_DR7, val);
 }
 
-static int handle_cpuid(struct kvm_vcpu *vcpu)
+static __always_inline int handle_cpuid(struct kvm_vcpu *vcpu)
 {
 	return kvm_emulate_cpuid(vcpu);
 }
@@ -4891,7 +4891,7 @@ static int handle_interrupt_window(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
-static int handle_halt(struct kvm_vcpu *vcpu)
+static __always_inline int handle_halt(struct kvm_vcpu *vcpu)
 {
 	return kvm_emulate_halt(vcpu);
 }
#35 lore raw

Re: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c

Quoted 5 lines · Paolo Bonzini
Quoted 2 lines · Andrea Arcangeli

They can be called directly more efficiently, so we can as well mark some of them inline in case gcc doesn't decide to inline them.

What is the output of size(1) before and after?

Before and after this specific commit there is a difference with gcc 8.3.

full patchset applied

753699 87971 9616 851286 cfd56 build/arch/x86/kvm/kvm-intel.ko

git revert

753739 87971 9616 851326 cfd7e build/arch/x86/kvm/kvm-intel.ko

git reset --hard HEAD^

753699 87971 9616 851286 cfd56 build/arch/x86/kvm/kvm-intel.ko

git revert

753739 87971 9616 851326 cfd7e build/arch/x86/kvm/kvm-intel.ko

maintainer
#36 lore raw

Re: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c

Quoted 10 lines · Andrea Arcangeli

Before and after this specific commit there is a difference with gcc 8.3.

full patchset applied

753699 87971 9616 851286 cfd56 build/arch/x86/kvm/kvm-intel.ko

git revert

753739 87971 9616 851326 cfd7e build/arch/x86/kvm/kvm-intel.ko

Quoted 7 lines

git reset --hard HEAD^

753699 87971 9616 851286 cfd56 build/arch/x86/kvm/kvm-intel.ko

git revert

753739 87971 9616 851326 cfd7e build/arch/x86/kvm/kvm-intel.ko

So it's forty bytes. I think we can leave this out.

Paolo

#37 lore raw

Re: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c

Quoted 13 lines · Paolo Bonzini
Quoted 12 lines · Andrea Arcangeli

Before and after this specific commit there is a difference with gcc 8.3.

full patchset applied

753699 87971 9616 851286 cfd56 build/arch/x86/kvm/kvm-intel.ko

git revert

753739 87971 9616 851326 cfd7e build/arch/x86/kvm/kvm-intel.ko

git reset --hard HEAD^

Quoted 7 lines
Quoted 5 lines

753699 87971 9616 851286 cfd56 build/arch/x86/kvm/kvm-intel.ko

git revert

753739 87971 9616 851326 cfd7e build/arch/x86/kvm/kvm-intel.ko

So it's forty bytes. I think we can leave this out.

This commit I reverted adds literally 3 inlines called by 3 functions, in a very fast path, how many bytes of .text difference did you expect by dropping some call/ret from a very fast path when you asked me to test it? I mean it's just a couple of insn each.

I thought the question was if gcc was already inlining without the hint or not or if it actually grew in size in case I got it wrong and there were many callers and it was better off not inline, so now I don't get what was the point of this test if with the result that confirms it's needed, the patch should be dropped.

It's possible that this patch may not be relevant anymore with the rename in place of the vmx/svm functions, but if this patch is to be dropped with the optimal result, then I recommend you to go ahead and submit a patch to drop __always_inline from the whole kernel because if it's not good to use it here in a extreme fast path like handle_external_interrupt and handle_halt, then I don't know what __always_inline is good for anywhere else in the kernel.

Thanks,
Andrea

#38 lore raw

Re: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c

Quoted 11 lines · Andrea Arcangeli

This commit I reverted adds literally 3 inlines called by 3 functions, in a very fast path, how many bytes of .text difference did you expect by dropping some call/ret from a very fast path when you asked me to test it? I mean it's just a couple of insn each.

I thought the question was if gcc was already inlining without the hint or not or if it actually grew in size in case I got it wrong and there were many callers and it was better off not inline, so now I don't get what was the point of this test if with the result that confirms it's needed, the patch should be dropped.

Quoted 7 lines

It's possible that this patch may not be relevant anymore with the rename in place of the vmx/svm functions, but if this patch is to be dropped with the optimal result, then I recommend you to go ahead and submit a patch to drop __always_inline from the whole kernel because if it's not good to use it here in a extreme fast path like handle_external_interrupt and handle_halt, then I don't know what __always_inline is good for anywhere else in the kernel.

Thinking more at this I don't think the result of the size check was nearly enough to come to any conclusion. The only probably conclusion one can take is that if the size didn't change it was a fail, because there would be high probability that it wouldn't be beneficial because it was a noop (even that wouldn't be 100% certain).

One needs to look at why it changed to take any conclusion, and the reason it got smaller is that all dynamic tracing for ftrace was dropped, the functions were already inlined fine in the RETPOLINE case.

Those are tiny functions so it looks a positive thing to make them as small as possible, there are already proper tracepoints in kvm_exit/kvm_entry for bpf tracing all all KVM events so there's no need of the fentry in those tiny functions with just an instruction that are only ever compiled as static because of the pointer to function.

This however also means it helps the CONFIG_RETPOLINE=n case and it doesn't help at all the CONFIG_RETPOLINE=y case, so it's fully orthogonal to the patchset and can be splitted off but I think it's worth it.

Thanks,
Andrea

maintainer
#39 lore raw

Re: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c

In reply to Andrea Arcangeli
Quoted 5 lines · Andrea Arcangeli
Quoted 1 line

So it's forty bytes. I think we can leave this out.

This commit I reverted adds literally 3 inlines called by 3 functions, in a very fast path, how many bytes of .text difference did you expect by dropping some call/ret from a very fast path when you asked me to test it? I mean it's just a couple of insn each.

Actually I was either expecting the difference to be zero, meaning GCC was already inlining them.

I think it is not inlining the functions because they are still referenced by vmx_exit_handlers. After patch 15 you could drop them from the array, and then GCC should inline them.

Paolo

#40 lore raw

PATCH 15/17 KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Andrea Arcangeli

It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.

---
 arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +22 -2
index a6e597025011..9aa73e216df2 100644
@@ -5866,9 +5866,29 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)
 	}
 
 	if (exit_reason < kvm_vmx_max_exit_handlers
-	    && kvm_vmx_exit_handlers[exit_reason])
+	    && kvm_vmx_exit_handlers[exit_reason]) {
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);
-	else {
+	} else {
 		vcpu_unimpl(vcpu, "vmx: unexpected exit reason 0x%x\n",
 				exit_reason);
 		dump_vmcs();
maintainer
#41 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 29 lines · Andrea Arcangeli

It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.

---
 arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +12 -1
index a6e597025011..9aa73e216df2 100644
@@ -5866,9 +5866,29 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)
 	}
 
 	if (exit_reason < kvm_vmx_max_exit_handlers
-	    && kvm_vmx_exit_handlers[exit_reason])
+	    && kvm_vmx_exit_handlers[exit_reason]) {
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5890

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +9 -0
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

I agree with the identified set of most common vmexits, however, this still looks a bit random. Would it be too much if we get rid of kvm_vmx_exit_handlers completely replacing this code with one switch()?

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
-	else {
+	} else {
 		vcpu_unimpl(vcpu, "vmx: unexpected exit reason 0x%x\n",
 				exit_reason);
 		dump_vmcs();
Signature · 1 line
Vitaly
maintainer
#42 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 11 lines · Vitaly Kuznetsov
arch/x86/kvm/vmx/vmx.c +11 -0
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5890

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +9 -0
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

Most of these, while frequent, are already part of slow paths.

I would keep only EXIT_REASON_MSR_WRITE, EXIT_REASON_PREEMPTION_TIMER, EXIT_REASON_EPT_MISCONFIG and add EXIT_REASON_IO_INSTRUCTION.

If you make kvm_vmx_exit_handlers const, can the compiler substitute for instance kvm_vmx_exit_handlers[EXIT_REASON_MSR_WRITE] with handle_wrmsr?
Just thinking out loud, not sure if it's an improvement code-wise.

Paolo

Paolo

#43 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 25 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov
arch/x86/kvm/vmx/vmx.c +11 -0
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5890

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +9 -0
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

Most of these, while frequent, are already part of slow paths.

Quoted 2 lines

I would keep only EXIT_REASON_MSR_WRITE, EXIT_REASON_PREEMPTION_TIMER, EXIT_REASON_EPT_MISCONFIG and add EXIT_REASON_IO_INSTRUCTION.

Intuition doesn't work great when it comes to CPU speculative execution runtime. I can however run additional benchmarks to verify your theory that keeping around frequent retpolines will still perform ok.

Quoted 3 lines

If you make kvm_vmx_exit_handlers const, can the compiler substitute for instance kvm_vmx_exit_handlers[EXIT_REASON_MSR_WRITE] with handle_wrmsr?
Just thinking out loud, not sure if it's an improvement code-wise.

gcc gets right if you make it const, it calls kvm_emulate_wrmsr in fact. However I don't think const will fly with_vmx_hardware_setup()... in fact at runtime testing nested I just got:

BUG: unable to handle page fault for address: ffffffffa00751e0
#PF: supervisor write access in kernel mode
#PF: error_code(0x0003) - permissions violation
PGD 2424067 P4D 2424067 PUD 2425063 PMD 7cc09067 PTE 80000000741cb161
Oops: 0003 [#1] SMP NOPTI
CPU: 1 PID: 4458 Comm: insmod Not tainted 5.3.0+ #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS ?-20190711_202441-buildvm-armv7-10.arm.fedoraproject.or4
RIP: 0010:nested_vmx_hardware_setup+0x29a/0x37a [kvm_intel]
Code: 41 ff c5 66 89 2c 85 20 92 0b a0 66 44 89 34 85 22 92 0b a0 49 ff c7 e9 e6 fe ff ff 44 89 2d 28 24 fc ff 48
RSP: 0018:ffffc90000257c18 EFLAGS: 00010246
RAX: ffffffffa001e0b0 RBX: ffffffffa0075140 RCX: 0000000000000000
RDX: ffff888078f60000 RSI: 0000000000002401 RDI: 0000000000000018
RBP: 0000000000006c08 R08: 0000000000001000 R09: 000000000007ffdc
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000006c08
R13: 0000000000000017 R14: 0000000000000268 R15: 0000000000000018
FS:  00007f7fb7ef0b80(0000) GS:ffff88807da40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffa00751e0 CR3: 0000000079620001 CR4: 0000000000160ee0
Call Trace:
 hardware_setup+0x4df/0x5b2 [kvm_intel]
 kvm_arch_hardware_setup+0x2f/0x27b [kvm_intel]
 kvm_init+0x5d/0x26d [kvm_intel]
#44 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 57 lines · Andrea Arcangeli
Quoted 25 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov
(quoted diff) +11 -0
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);

Review

Quoted 10 lines
(quoted diff) +9 -0
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

Most of these, while frequent, are already part of slow paths.

Quoted 2 lines

I would keep only EXIT_REASON_MSR_WRITE, EXIT_REASON_PREEMPTION_TIMER, EXIT_REASON_EPT_MISCONFIG and add EXIT_REASON_IO_INSTRUCTION.

Intuition doesn't work great when it comes to CPU speculative execution runtime. I can however run additional benchmarks to verify your theory that keeping around frequent retpolines will still perform ok.

Quoted 3 lines

If you make kvm_vmx_exit_handlers const, can the compiler substitute for instance kvm_vmx_exit_handlers[EXIT_REASON_MSR_WRITE] with handle_wrmsr?
Just thinking out loud, not sure if it's an improvement code-wise.

gcc gets right if you make it const, it calls kvm_emulate_wrmsr in fact. However I don't think const will fly with_vmx_hardware_setup()... in fact at runtime testing nested I just got:

BUG: unable to handle page fault for address: ffffffffa00751e0
#PF: supervisor write access in kernel mode
#PF: error_code(0x0003) - permissions violation
PGD 2424067 P4D 2424067 PUD 2425063 PMD 7cc09067 PTE 80000000741cb161
Oops: 0003 [#1] SMP NOPTI
CPU: 1 PID: 4458 Comm: insmod Not tainted 5.3.0+ #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS ?-20190711_202441-buildvm-armv7-10.arm.fedoraproject.or4
RIP: 0010:nested_vmx_hardware_setup+0x29a/0x37a [kvm_intel]
Code: 41 ff c5 66 89 2c 85 20 92 0b a0 66 44 89 34 85 22 92 0b a0 49 ff c7 e9 e6 fe ff ff 44 89 2d 28 24 fc ff 48
RSP: 0018:ffffc90000257c18 EFLAGS: 00010246
RAX: ffffffffa001e0b0 RBX: ffffffffa0075140 RCX: 0000000000000000
RDX: ffff888078f60000 RSI: 0000000000002401 RDI: 0000000000000018
RBP: 0000000000006c08 R08: 0000000000001000 R09: 000000000007ffdc
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000006c08
Quoted 8 lines
R13: 0000000000000017 R14: 0000000000000268 R15: 0000000000000018
FS:  00007f7fb7ef0b80(0000) GS:ffff88807da40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffa00751e0 CR3: 0000000079620001 CR4: 0000000000160ee0
Call Trace:
 hardware_setup+0x4df/0x5b2 [kvm_intel]
 kvm_arch_hardware_setup+0x2f/0x27b [kvm_intel]
 kvm_init+0x5d/0x26d [kvm_intel]

The attached patch should do the trick.

From 4e0c2d73d796eae03aa289f77bef5f4a7acef655 Mon Sep 17 00:00:00 2001
From: Sean Christopherson <[email protected]>
Date: Mon, 23 Sep 2019 13:19:43 -0700
Subject: [PATCH] KVM: nVMX: Do not dynamically set VMX instruction exit
handlers

Handle VMX instructions via a dedicated function and a switch statement provided by the nVMX code instead of overwriting kvm_vmx_exit_handlers when nested support is enabled. This will allow a future patch to make kvm_vmx_exit_handlers a const, which in turn allows for better compiler optimizations, e.g. direct calls instead of retpolined indirect calls.

---
 arch/x86/kvm/vmx/nested.c | 52 ++++++++++++++++++++++++++++-----------
 arch/x86/kvm/vmx/nested.h |  3 ++-
 arch/x86/kvm/vmx/vmx.c    |  5 +++-
 3 files changed, 44 insertions(+), 16 deletions(-)
arch/x86/kvm/vmx/nested.c +38 -14
index 6ce83c602e7f..41c7fcf28ab6 100644
@@ -5072,6 +5072,43 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+int nested_vmx_handle_vmx_instruction(struct kvm_vcpu *vcpu)
+{
+	switch (to_vmx(vcpu)->exit_reason) {
+	case EXIT_REASON_VMCLEAR:
+		return handle_vmclear(vcpu);
+	case EXIT_REASON_VMLAUNCH:
+		return handle_vmlaunch(vcpu);
+	case EXIT_REASON_VMPTRLD:
+		return handle_vmptrld(vcpu);
+	case EXIT_REASON_VMPTRST:
+		return handle_vmptrst(vcpu);
+	case EXIT_REASON_VMREAD:
+		return handle_vmread(vcpu);
+	case EXIT_REASON_VMRESUME:
+		return handle_vmresume(vcpu);
+	case EXIT_REASON_VMWRITE:
+		return handle_vmwrite(vcpu);
+	case EXIT_REASON_VMOFF:
+		return handle_vmoff(vcpu);
+	case EXIT_REASON_VMON:
+		return handle_vmon(vcpu);
+	case EXIT_REASON_INVEPT:
+		return handle_invept(vcpu);
+	case EXIT_REASON_INVVPID:
+		return handle_invvpid(vcpu);
+	case EXIT_REASON_VMFUNC:
+		return handle_vmfunc(vcpu);
+	}
+
+	WARN_ON_ONCE(1);
+	vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+	vcpu->run->internal.suberror =
+	KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON;
+	vcpu->run->internal.ndata = 1;
+	vcpu->run->internal.data[0] = to_vmx(vcpu)->exit_reason;
+	return 0;
+}
 
 static bool nested_vmx_exit_handled_io(struct kvm_vcpu *vcpu,
 				       struct vmcs12 *vmcs12)
@@ -5972,7 +6009,7 @@ void nested_vmx_hardware_unsetup(void)
 	}
 }
 
-__init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *))
+__init int nested_vmx_hardware_setup(void)
 {
 	int i;
 
@@ -5995,19 +6032,6 @@ __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *))
 		init_vmcs_shadow_fields();
 	}
 
-	exit_handlers[EXIT_REASON_VMCLEAR]	= handle_vmclear,
-	exit_handlers[EXIT_REASON_VMLAUNCH]	= handle_vmlaunch,
-	exit_handlers[EXIT_REASON_VMPTRLD]	= handle_vmptrld,
-	exit_handlers[EXIT_REASON_VMPTRST]	= handle_vmptrst,
-	exit_handlers[EXIT_REASON_VMREAD]	= handle_vmread,
-	exit_handlers[EXIT_REASON_VMRESUME]	= handle_vmresume,
-	exit_handlers[EXIT_REASON_VMWRITE]	= handle_vmwrite,
-	exit_handlers[EXIT_REASON_VMOFF]	= handle_vmoff,
-	exit_handlers[EXIT_REASON_VMON]		= handle_vmon,
-	exit_handlers[EXIT_REASON_INVEPT]	= handle_invept,
-	exit_handlers[EXIT_REASON_INVVPID]	= handle_invvpid,
-	exit_handlers[EXIT_REASON_VMFUNC]	= handle_vmfunc,
-
 	kvm_x86_ops->check_nested_events = vmx_check_nested_events;
 	kvm_x86_ops->get_nested_state = vmx_get_nested_state;
 	kvm_x86_ops->set_nested_state = vmx_set_nested_state;
arch/x86/kvm/vmx/nested.h +2 -1
index 187d39bf0bf1..0da48c83cccf 100644
@@ -10,9 +10,10 @@ void vmx_leave_nested(struct kvm_vcpu *vcpu);
 void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps,
 				bool apicv);
 void nested_vmx_hardware_unsetup(void);
-__init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *));
+__init int nested_vmx_hardware_setup(void);
 void nested_vmx_vcpu_setup(void);
 void nested_vmx_free_vcpu(struct kvm_vcpu *vcpu);
+int nested_vmx_handle_vmx_instruction(struct kvm_vcpu *vcpu);
 int nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, bool from_vmentry);
 bool nested_vmx_exit_reflected(struct kvm_vcpu *vcpu, u32 exit_reason);
 void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason,
arch/x86/kvm/vmx/vmx.c +4 -1
index 73bf9a2e6fb6..229b3a5e0695 100644
@@ -5459,6 +5459,9 @@ static int handle_preemption_timer(struct kvm_vcpu *vcpu)
  */
 static int handle_vmx_instruction(struct kvm_vcpu *vcpu)
 {
+	if (nested)
+		return nested_vmx_handle_vmx_instruction(vcpu);
+
 	kvm_queue_exception(vcpu, UD_VECTOR);
 	return 1;
 }
@@ -7631,7 +7634,7 @@ static __init int hardware_setup(void)
 		nested_vmx_setup_ctls_msrs(&vmcs_config.nested,
 					   vmx_capability.ept, enable_apicv);
 
-		r = nested_vmx_hardware_setup(kvm_vmx_exit_handlers);
+		r = nested_vmx_hardware_setup();
 		if (r)
 			return r;
 	}
Signature · 1 line
2.22.0
#45 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Hello,

Quoted 1 line · Sean Christopherson

The attached patch should do the trick.

The two most attractive options to me remains what I already have implemented under #ifdef CONFIG_RETPOLINE with direct calls (optionally replacing the "if" with a small "switch" still under CONFIG_RETPOLINE if we give up the prioritization of the checks), or the replacement of kvm_vmx_exit_handlers with a switch() as suggested by Vitaly which would cleanup some code.

The intermediate solution that makes "const" work, has the cons of forcing to parse EXIT_REASON_VMCLEAR and the other vmx exit reasons twice, first through a pointer to function (or another if or switch statement) then with a second switch() statement.

If we'd use a single switch statement per Vitaly's suggestion, the "if nested" would better be more simply implemented as:

	switch (exit_reason) {
		case EXIT_REASON_VMCLEAR:
			if (nested)
				return handle_vmclear(vcpu);
			else
				return handle_vmx_instruction(vcpu);
		case EXIT_REASON_VMCLEAR:
			if (nested)
		[..]

This also removes the compiler dependency to auto inline handle_vmclear in the added nested_vmx_handle_vmx_instruction extern call.

VMREAD/WRITE/RESUME are the most frequent vmexit in l0 while nested runs in l2.

Thanks,
Andrea

#46 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 20 lines · Andrea Arcangeli

Hello,

Quoted 1 line · Sean Christopherson

The attached patch should do the trick.

The two most attractive options to me remains what I already have implemented under #ifdef CONFIG_RETPOLINE with direct calls (optionally replacing the "if" with a small "switch" still under CONFIG_RETPOLINE if we give up the prioritization of the checks), or the replacement of kvm_vmx_exit_handlers with a switch() as suggested by Vitaly which would cleanup some code.

The intermediate solution that makes "const" work, has the cons of forcing to parse EXIT_REASON_VMCLEAR and the other vmx exit reasons twice, first through a pointer to function (or another if or switch statement) then with a second switch() statement.

If we'd use a single switch statement per Vitaly's suggestion, the "if nested" would better be more simply implemented as:

Quoted 9 lines
	switch (exit_reason) {
		case EXIT_REASON_VMCLEAR:
			if (nested)
				return handle_vmclear(vcpu);
			else
				return handle_vmx_instruction(vcpu);
		case EXIT_REASON_VMCLEAR:
			if (nested)
		[..]

Combing if/case statements would mitigate this to some degree, e.g.:

	if (exit_reason >= EXIT_REASON_VMCALL &&
	    exit_reason <= EXIT_REASON_VMON)
		return handle_vmx_instruction(vcpu);

or

	case EXIT_REASON_VMCALL ... EXIT_REASON_VMON:
		return handle_vmx_instruction(vcpu);
Quoted 3 lines

This also removes the compiler dependency to auto inline handle_vmclear in the added nested_vmx_handle_vmx_instruction extern call.

I like the idea of routing through handle_vmx_instruction() because it allows the individual handlers to be wholly encapsulated in nested.c, e.g. handle_vmclear() and company don't have to be exposed.

An extra CALL+RET isn't going to be noticeable, especially on modern hardware as the high frequency VMWRITE/VMREAD fields should hit the shadow VMCS.

Quoted 5 lines

VMREAD/WRITE/RESUME are the most frequent vmexit in l0 while nested runs in l2.

Thanks,
Andrea

#47 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 3 lines · Sean Christopherson

An extra CALL+RET isn't going to be noticeable, especially on modern hardware as the high frequency VMWRITE/VMREAD fields should hit the shadow VMCS.

In your last email with regard to the inlining optimizations made possible by the monolithic KVM model you said "That'd likely save a few CALL/RET/JMP instructions", that kind of directly contradicts the above. I think neither one if taken at face value can be possibly measured. However the above only is relevant for nested KVM so I'm fine if there's an agreement that it's better to hide the nested vmx handlers in nested.c at the cost of some call/ret.

From my part I'm dropping 15/16/17 in the short term, perhaps Vitaly or you or Paolo if he has time, want to work on that part in parallel to the orthogonal KVM monolithic changes?

Thanks,
Andrea

#48 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 5 lines · Andrea Arcangeli
Quoted 3 lines · Sean Christopherson

An extra CALL+RET isn't going to be noticeable, especially on modern hardware as the high frequency VMWRITE/VMREAD fields should hit the shadow VMCS.

Quoted 7 lines

In your last email with regard to the inlining optimizations made possible by the monolithic KVM model you said "That'd likely save a few CALL/RET/JMP instructions", that kind of directly contradicts the above. I think neither one if taken at face value can be possibly measured. However the above only is relevant for nested KVM so I'm fine if there's an agreement that it's better to hide the nested vmx handlers in nested.c at the cost of some call/ret.

For the immediate exit case, eliminating the CALL/RET/JMP instructions is a bonus. The real goal was to eliminate the oddity of bouncing through vendor code to invoke a one-line x86 function. Having a separate __kvm_request_immediate_exit() made sense when overwriting kvm_ops_x86, but not so much when using direct calls.

maintainer
#49 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Andrea Arcangeli
Quoted 11 lines · Andrea Arcangeli

The two most attractive options to me remains what I already have implemented under #ifdef CONFIG_RETPOLINE with direct calls (optionally replacing the "if" with a small "switch" still under CONFIG_RETPOLINE if we give up the prioritization of the checks), or the replacement of kvm_vmx_exit_handlers with a switch() as suggested by Vitaly which would cleanup some code.

The intermediate solution that makes "const" work, has the cons of forcing to parse EXIT_REASON_VMCLEAR and the other vmx exit reasons twice, first through a pointer to function (or another if or switch statement) then with a second switch() statement.

I agree. I think the way Andrea did it in his patch may not the nicest but is (a bit surprisingly) the easiest and most maintainable.

Paolo

#50 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 8 lines · Paolo Bonzini
Quoted 7 lines · Andrea Arcangeli

The two most attractive options to me remains what I already have implemented under #ifdef CONFIG_RETPOLINE with direct calls (optionally replacing the "if" with a small "switch" still under CONFIG_RETPOLINE if we give up the prioritization of the checks), or the replacement of kvm_vmx_exit_handlers with a switch() as suggested by Vitaly which would cleanup some code.

Quoted 7 lines
Quoted 4 lines

The intermediate solution that makes "const" work, has the cons of forcing to parse EXIT_REASON_VMCLEAR and the other vmx exit reasons twice, first through a pointer to function (or another if or switch statement) then with a second switch() statement.

I agree. I think the way Andrea did it in his patch may not the nicest but is (a bit surprisingly) the easiest and most maintainable.

Heh, which patch? The original patch of special casing the high priority exits?

maintainer
#51 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 4 lines · Sean Christopherson
Quoted 2 lines

I agree. I think the way Andrea did it in his patch may not the nicest but is (a bit surprisingly) the easiest and most maintainable.

Heh, which patch? The original patch of special casing the high priority exits?

Yes.

Paolo

maintainer
#52 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Sean Christopherson
Quoted 20 lines · Sean Christopherson
arch/x86/kvm/vmx/nested.c +19 -0
+int nested_vmx_handle_vmx_instruction(struct kvm_vcpu *vcpu)
+{
+	switch (to_vmx(vcpu)->exit_reason) {
+	case EXIT_REASON_VMCLEAR:
+		return handle_vmclear(vcpu);
+	case EXIT_REASON_VMLAUNCH:
+		return handle_vmlaunch(vcpu);
+	case EXIT_REASON_VMPTRLD:
+		return handle_vmptrld(vcpu);
+	case EXIT_REASON_VMPTRST:
+		return handle_vmptrst(vcpu);
+	case EXIT_REASON_VMREAD:
+		return handle_vmread(vcpu);
+	case EXIT_REASON_VMRESUME:
+		return handle_vmresume(vcpu);
+	case EXIT_REASON_VMWRITE:
+		return handle_vmwrite(vcpu);
+	case EXIT_REASON_VMOFF:
+		return handle_vmoff(vcpu);

arch/x86/kvm/vmx/nested.c static int handle_vmfunc(struct kvm_vcpu *vcpu) · line 5103

Quoted 10 lines
arch/x86/kvm/vmx/nested.c +10 -0
+	case EXIT_REASON_VMON:
+		return handle_vmon(vcpu);
+	case EXIT_REASON_INVEPT:
+		return handle_invept(vcpu);
+	case EXIT_REASON_INVVPID:
+		return handle_invvpid(vcpu);
+	case EXIT_REASON_VMFUNC:
+		return handle_vmfunc(vcpu);
+	}
+

Do you really need that? Why couldn't the handle_* functions simply be exported from nested.c to vmx.c?

Paolo

#53 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Hi Paolo,

Quoted 2 lines · Paolo Bonzini

Do you really need that? Why couldn't the handle_* functions simply be exported from nested.c to vmx.c?

I prefer the direct call too indeed.

If Sean doesn't want to export those generic names to the whole kernel it would be enough to #include "nested.c" from vmx.c, and you'd still have it private but with no additional checks and no additional extern call. It's not like kvm-intel can be built without linking nested.o anyway.

This overall is a C shortcoming of some sort if you've to resort to #include "nested.c" to keep the function hidden in kvm-intel.o despite it's implemented in two different object files. One should be able to limit the scope of an extern function declaration per a group of object files and to drop the declaration before linking that object beyond that initial group.

Thanks,
Andrea

#54 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Paolo Bonzini
Quoted 26 lines · Paolo Bonzini
Quoted 25 lines · Sean Christopherson
arch/x86/kvm/vmx/nested.c +24 -0
+int nested_vmx_handle_vmx_instruction(struct kvm_vcpu *vcpu)
+{
+	switch (to_vmx(vcpu)->exit_reason) {
+	case EXIT_REASON_VMCLEAR:
+		return handle_vmclear(vcpu);
+	case EXIT_REASON_VMLAUNCH:
+		return handle_vmlaunch(vcpu);
+	case EXIT_REASON_VMPTRLD:
+		return handle_vmptrld(vcpu);
+	case EXIT_REASON_VMPTRST:
+		return handle_vmptrst(vcpu);
+	case EXIT_REASON_VMREAD:
+		return handle_vmread(vcpu);
+	case EXIT_REASON_VMRESUME:
+		return handle_vmresume(vcpu);
+	case EXIT_REASON_VMWRITE:
+		return handle_vmwrite(vcpu);
+	case EXIT_REASON_VMOFF:
+		return handle_vmoff(vcpu);
+	case EXIT_REASON_VMON:
+		return handle_vmon(vcpu);
+	case EXIT_REASON_INVEPT:
+		return handle_invept(vcpu);
+	case EXIT_REASON_INVVPID:
Quoted 8 lines

arch/x86/kvm/vmx/nested.c static int handle_vmfunc(struct kvm_vcpu *vcpu) · line 5103

Quoted 5 lines
arch/x86/kvm/vmx/nested.c +5 -0
+		return handle_invvpid(vcpu);
+	case EXIT_REASON_VMFUNC:
+		return handle_vmfunc(vcpu);
+	}
+

Do you really need that? Why couldn't the handle_* functions simply be exported from nested.c to vmx.c?

Nope, just personal preference to keep the nested code as isolated as possible. We use a similar approach for vmx_{g,s}et_vmx_msr().

Though if we do want to go this route, it'd be better to simply move handle_vmx_instruction() to nested.c instead of bouncing through that and nested_vmx_handle_vmx_instruction().

#55 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Andrea Arcangeli
Quoted 26 lines · Andrea Arcangeli
Quoted 25 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov
arch/x86/kvm/vmx/vmx.c +11 -0
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5890

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +9 -0
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

Most of these, while frequent, are already part of slow paths.

Quoted 7 lines
Quoted 2 lines

I would keep only EXIT_REASON_MSR_WRITE, EXIT_REASON_PREEMPTION_TIMER, EXIT_REASON_EPT_MISCONFIG and add EXIT_REASON_IO_INSTRUCTION.

Intuition doesn't work great when it comes to CPU speculative execution runtime. I can however run additional benchmarks to verify your theory that keeping around frequent retpolines will still perform ok.

On one most recent CPU model there's no measurable difference with your list or my list with a hrtimer workload (no cpuid). It's challenging to measure any difference below 0.5%.

An artificial cpuid loop takes 1.5% longer to compute if I don't add CPUID to the list, but that's just the measurement of the cost of hitting a frequent retpoline in the exit reason handling code.

Thanks,
Andrea

maintainer
#56 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 10 lines · Andrea Arcangeli
Quoted 7 lines
Quoted 3 lines

I would keep only EXIT_REASON_MSR_WRITE, EXIT_REASON_PREEMPTION_TIMER, EXIT_REASON_EPT_MISCONFIG and add EXIT_REASON_IO_INSTRUCTION.

Intuition doesn't work great when it comes to CPU speculative execution runtime. I can however run additional benchmarks to verify your theory that keeping around frequent retpolines will still perform ok.

On one most recent CPU model there's no measurable difference with your list or my list with a hrtimer workload (no cpuid). It's challenging to measure any difference below 0.5%.

Let's keep the short list then.

Paolo

#57 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Vitaly Kuznetsov
Quoted 37 lines · Vitaly Kuznetsov
Quoted 35 lines · Andrea Arcangeli

It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.

---
 arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +18 -1
index a6e597025011..9aa73e216df2 100644
@@ -5866,9 +5866,29 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)
 	}
 
 	if (exit_reason < kvm_vmx_max_exit_handlers
-	    && kvm_vmx_exit_handlers[exit_reason])
+	    && kvm_vmx_exit_handlers[exit_reason]) {
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5890

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

I agree with the identified set of most common vmexits, however, this still looks a bit random. Would it be too much if we get rid of kvm_vmx_exit_handlers completely replacing this code with one switch()?

Hmm, that'd require redirects for nVMX functions since they are set at runtime. That isn't necessarily a bad thing. The approach could also be used if Paolo's idea of making kvm_vmx_max_exit_handlers const allows the compiler to avoid retpoline.

E.g.:

static int handle_vmx_instruction(struct kvm_vcpu *vcpu)

{
	if (nested)
		return nested_vmx_handle_exit(vcpu);

	kvm_queue_exception(vcpu, UD_VECTOR);
	return 1;
}
maintainer
#58 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 6 lines · Sean Christopherson
Quoted 2 lines

Would it be too much if we get rid of
kvm_vmx_exit_handlers completely replacing this code with one switch()?

Hmm, that'd require redirects for nVMX functions since they are set at runtime. That isn't necessarily a bad thing. The approach could also be used if Paolo's idea of making kvm_vmx_max_exit_handlers const allows the compiler to avoid retpoline.

But aren't switch statements also retpolin-ized if they use a jump table?

Paolo

#59 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 9 lines · Paolo Bonzini
Quoted 6 lines · Sean Christopherson
Quoted 2 lines

Would it be too much if we get rid of
kvm_vmx_exit_handlers completely replacing this code with one switch()?

Hmm, that'd require redirects for nVMX functions since they are set at runtime. That isn't necessarily a bad thing. The approach could also be used if Paolo's idea of making kvm_vmx_max_exit_handlers const allows the compiler to avoid retpoline.

But aren't switch statements also retpolin-ized if they use a jump table?

See commit a9d57ef15cbe327fe54416dd194ee0ea66ae53a4.

We disabled that feature or the kernel would potentially suffer the downsides of the exit handlers through pointer to functions for every switch statement in the kernel.

In turn you can't make it run any faster by converting my "if" to a "switch" at least the "if" can deterministic control the order of what is more likely that we should also re-review, but the order of secondary effect, the important thing is to reduce the retpolines to zero during normal hrtimer guest runtime.

#60 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 17 lines · Andrea Arcangeli
Quoted 9 lines · Paolo Bonzini
Quoted 6 lines · Sean Christopherson
Quoted 2 lines

Would it be too much if we get rid of
kvm_vmx_exit_handlers completely replacing this code with one switch()?

Hmm, that'd require redirects for nVMX functions since they are set at runtime. That isn't necessarily a bad thing. The approach could also be used if Paolo's idea of making kvm_vmx_max_exit_handlers const allows the compiler to avoid retpoline.

But aren't switch statements also retpolin-ized if they use a jump table?

See commit a9d57ef15cbe327fe54416dd194ee0ea66ae53a4.

We disabled that feature or the kernel would potentially suffer the downsides of the exit handlers through pointer to functions for every switch statement in the kernel.

Quoted 5 lines

In turn you can't make it run any faster by converting my "if" to a "switch" at least the "if" can deterministic control the order of what is more likely that we should also re-review, but the order of secondary effect, the important thing is to reduce the retpolines to zero during normal hrtimer guest runtime.

On the flip side, using a switch for the fast-path handlers gives the compiler more flexibility to rearrange and combine checks. Of course that doesn't mean the compiler will actually generate faster code for our purposes :-)

Anyways, getting rid of the retpolines is much more important.

#61 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 6 lines · Sean Christopherson

On the flip side, using a switch for the fast-path handlers gives the compiler more flexibility to rearrange and combine checks. Of course that doesn't mean the compiler will actually generate faster code for our purposes :-)

Anyways, getting rid of the retpolines is much more important.

Precisely because of your last point, if we throw away the deterministic priority, then we could drop the whole structure like Vitaly suggested and we'd rely on gcc to add the indirect jump on a non-retpoline build.

Solving the nested if we drop the structure and we don't pretend to make it const, isn't tricky: it only requires one more check if nested is enabled. The same variable that will have to be checked is also the variable that needs to be checked in the kvm_x86_ops->check_nested_events replacement later to drop the kvm_x86_ops struct as a whole like kvm_pmu_ops was dropped clean.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#63 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

Quoted 50 lines · Christophe de Dinechin
Quoted 41 lines · Vitaly Kuznetsov
Quoted 35 lines · Andrea Arcangeli <[email protected] <mailto:[email protected]>> writes:

It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.

---
arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++++++++++++--

1 file changed, 22 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +18 -1
index a6e597025011..9aa73e216df2 100644
@@ -5866,9 +5866,29 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)
 	}
 
 	if (exit_reason < kvm_vmx_max_exit_handlers
-	    && kvm_vmx_exit_handlers[exit_reason])
+	    && kvm_vmx_exit_handlers[exit_reason]) {
+#ifdef CONFIG_RETPOLINE
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)
+			return handle_halt(vcpu);
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5890

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);

I agree with the identified set of most common vmexits, however, this still looks a bit random. Would it be too much if we get rid of kvm_vmx_exit_handlers completely replacing this code with one switch()?

Quoted 4 lines

Not sure, but if you do that, won’t the compiler generate a table and bring you back to square one? Or is there a reason why the mitigation is not needed for tables and indirect branches generated from switch statements?

When the kernel is built with retpolines the compiler is forbidden to use a table for any switch. I pointed out the relevant commit earlier in this thread. Instead the compiler will still try to bisect the exit_reason trying to make the cost more equal for all exit_reason and to reduce the number of checks, but we know the most likely exits so it should be better to prioritize the most frequent exit reasons.

Thanks,
Andrea

#64 lore raw

Re: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers

In reply to Andrea Arcangeli

Subject should be something like:

KVM: VMX: Make direct calls to fast path VM-Exit handlers

Quoted 13 lines · Andrea Arcangeli

It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.

---
 arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +0 -0
index a6e597025011..9aa73e216df2 100644

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5870

Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +2 -1
@@ -5866,9 +5866,29 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)
 	}
 
 	if (exit_reason < kvm_vmx_max_exit_handlers
-	    && kvm_vmx_exit_handlers[exit_reason])
+	    && kvm_vmx_exit_handlers[exit_reason]) {
+#ifdef CONFIG_RETPOLINE

I'd strongly prefer to make any optimization of this type unconditional, i.e. not dependent on CONFIG_RETPOLINE. Today, I'm comfortable testing KVM only with CONFIG_RETPOLINE=y since the only KVM-specific difference is additive code in vmx_vmexit. That would no longer be the case if KVM has non-trivial code differences for retpoline.

Quoted 9 lines
arch/x86/kvm/vmx/vmx.c +9 -0
+		if (exit_reason == EXIT_REASON_MSR_WRITE)
+			return handle_wrmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER)
+			return handle_preemption_timer(vcpu);
+		else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT)
+			return handle_interrupt_window(vcpu);
+		else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT)
+			return handle_external_interrupt(vcpu);
+		else if (exit_reason == EXIT_REASON_HLT)

arch/x86/kvm/vmx/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5889

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+			return handle_halt(vcpu);
+		else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION)
+			return handle_pause(vcpu);
+		else if (exit_reason == EXIT_REASON_MSR_READ)
+			return handle_rdmsr(vcpu);
+		else if (exit_reason == EXIT_REASON_CPUID)
+			return handle_cpuid(vcpu);
+		else if (exit_reason == EXIT_REASON_EPT_MISCONFIG)
+			return handle_ept_misconfig(vcpu);
+#endif

This can be hoisted above the if statement.

Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +1 -1
 		return kvm_vmx_exit_handlers[exit_reason](vcpu);
-	else {
+	} else {
 		vcpu_unimpl(vcpu, "vmx: unexpected exit reason 0x%x\n",
 				exit_reason);
 		dump_vmcs();
#65 lore raw

PATCH 16/17 KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers

In reply to Andrea Arcangeli

It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons.

After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a SVM host:

[..]
@[

    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get_update_offsets_now+70
    hrtimer_interrupt+131
    smp_apic_timer_interrupt+106
    apic_timer_interrupt+15
    start_sw_timer+359
    restart_apic_timer+85
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 1940
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_r12+33
    force_qs_rnp+217
    rcu_gp_kthread+1270
    kthread+268
    ret_from_fork+34
]: 4644
@[]: 25095
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    lapic_next_event+28
    clockevents_program_event+148
    hrtimer_start_range_ns+528
    start_sw_timer+356
    restart_apic_timer+85
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 41474
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    clockevents_program_event+148
    hrtimer_start_range_ns+528
    start_sw_timer+356
    restart_apic_timer+85
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 41474
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get+58
    clockevents_program_event+84
    hrtimer_start_range_ns+528
    start_sw_timer+356
    restart_apic_timer+85
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 41887
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    lapic_next_event+28
    clockevents_program_event+148
    hrtimer_try_to_cancel+168
    hrtimer_cancel+21
    kvm_set_lapic_tscdeadline_msr+43
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 42723
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    clockevents_program_event+148
    hrtimer_try_to_cancel+168
    hrtimer_cancel+21
    kvm_set_lapic_tscdeadline_msr+43
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 42766
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get+58
    clockevents_program_event+84
    hrtimer_try_to_cancel+168
    hrtimer_cancel+21
    kvm_set_lapic_tscdeadline_msr+43
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 42848
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    ktime_get+58
    start_sw_timer+279
    restart_apic_timer+85
    kvm_set_msr_common+1497
    msr_interception+142
    vcpu_enter_guest+684
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68

]: 499845

@total: 1780243

SVM has no TSC based programmable preemption timer so it is invoking ktime_get() frequently.

---
 arch/x86/kvm/svm.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/kvm/svm.c +14 -0
index ca9fd1762519..7324243b0a82 100644
@@ -4991,6 +4991,20 @@ static int handle_exit(struct kvm_vcpu *vcpu)
 		return 0;
 	}
 
+#ifdef CONFIG_RETPOLINE
+	if (exit_code == SVM_EXIT_MSR)
+		return msr_interception(svm);
+	else if (exit_code == SVM_EXIT_VINTR)
+		return interrupt_window_interception(svm);
+	else if (exit_code == SVM_EXIT_INTR)
+		return intr_interception(svm);
+	else if (exit_code == SVM_EXIT_HLT)
+		return halt_interception(svm);
+	else if (exit_code == SVM_EXIT_NPF)
+		return npf_interception(svm);
+	else if (exit_code == SVM_EXIT_CPUID)
+		return cpuid_interception(svm);
+#endif
 	return svm_exit_handlers[exit_code](svm);
 }
 
maintainer
#66 lore raw

Re: [PATCH 16/17] KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers

Quoted 4 lines · Andrea Arcangeli
arch/x86/kvm/svm.c +4 -0
+#ifdef CONFIG_RETPOLINE
+	if (exit_code == SVM_EXIT_MSR)
+		return msr_interception(svm);
+	else if (exit_code == SVM_EXIT_VINTR)

arch/x86/kvm/svm.c static int handle_exit(struct kvm_vcpu *vcpu) · line 5007

Quoted 10 lines
arch/x86/kvm/svm.c +10 -0
+		return interrupt_window_interception(svm);
+	else if (exit_code == SVM_EXIT_INTR)
+		return intr_interception(svm);
+	else if (exit_code == SVM_EXIT_HLT)
+		return halt_interception(svm);
+	else if (exit_code == SVM_EXIT_NPF)
+		return npf_interception(svm);
+	else if (exit_code == SVM_EXIT_CPUID)
+		return cpuid_interception(svm);
+#endif

Same here; msr_interception and npf_interception are the main ones we care about, plus io_interception which isn't listed probably because it depends on the virtual hardware.

Paolo

#67 lore raw

PATCH 17/17 x86: retpolines: eliminate retpoline from msr event handlers

In reply to Andrea Arcangeli

It's enough to check the value and issue the direct call.

After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a VMX host:

[..]
@[

    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68
]: 267
@[]: 2256
@[
    trace_retpoline+1
    __trace_retpoline+30
    __x86_indirect_thunk_rax+33
    __kvm_wait_lapic_expire+284
    vmx_vcpu_run.part.97+1091
    vcpu_enter_guest+377
    kvm_arch_vcpu_ioctl_run+261
    kvm_vcpu_ioctl+559
    do_vfs_ioctl+164
    ksys_ioctl+96
    __x64_sys_ioctl+22
    do_syscall_64+89
    entry_SYSCALL_64_after_hwframe+68

]: 2390
@[]: 33410

@total: 315707

Note the highest hit above is __delay so probably not worth optimizing even if it would be more frequent than 2k hits per sec.

---
 arch/x86/events/intel/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/events/intel/core.c +11 -0
index 27ee47a7be66..65b383d5e062 100644
@@ -3323,8 +3323,19 @@ static int intel_pmu_hw_config(struct perf_event *event)
 	return 0;
 }
 
+#ifdef CONFIG_RETPOLINE
+static struct perf_guest_switch_msr *core_guest_get_msrs(int *nr);
+static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr);
+#endif
+
 struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr)
 {
+#ifdef CONFIG_RETPOLINE
+	if (x86_pmu.guest_get_msrs == intel_guest_get_msrs)
+		return intel_guest_get_msrs(nr);
+	else if (x86_pmu.guest_get_msrs == core_guest_get_msrs)
+		return core_guest_get_msrs(nr);
+#endif
 	if (x86_pmu.guest_get_msrs)
 		return x86_pmu.guest_get_msrs(nr);
 	*nr = 0;
#68 lore raw
In reply to Andrea Arcangeli
Quoted 4 lines · Andrea Arcangeli

Andrea Arcangeli (17):
x86: spec_ctrl: fix SPEC_CTRL initialization after kexec
KVM: monolithic: x86: convert the kvm_x86_ops methods to external
functions

Quoted 8 lines

KVM: monolithic: x86: handle the request_immediate_exit variation
KVM: monolithic: x86: convert the kvm_pmu_ops methods to external
functions
KVM: monolithic: x86: enable the kvm_x86_ops external functions
KVM: monolithic: x86: enable the kvm_pmu_ops external functions
KVM: monolithic: x86: adjust the section prefixes
KVM: monolithic: adjust the section prefixes in the KVM common code
KVM: monolithic: x86: remove kvm.ko

IMO, the conversion to a monolithic module should occur immediately, i.e. "KVM: monolithic: x86: remove kvm.ko" should be patch 01/nn.

Removing kvm_x86_ops and kvm_pmu_ops isn't a preqrequisite to making KVM a monolothic module, rather they're enhancements that are made possible *because* KVM is a monolithic module.

With that ordering, I suspect the convert->enable->use of kvm_x86_ops can be collapsed into a single patch.

Quoted 9 lines

KVM: monolithic: x86: use the external functions instead of kvm_x86_ops
KVM: monolithic: x86: remove exports
KVM: monolithic: remove exports from KVM common code
KVM: monolithic: x86: drop the kvm_pmu_ops structure

KVM: monolithic: x86: inline more exit handlers in vmx.c
KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers
KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers
x86: retpolines: eliminate retpoline from msr event handlers