FAILED: patch "[PATCH] KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said" failed to apply to 7.2-stable tree
Subsystems (3)
[email protected]
maintainerThe patch below does not apply to the 7.2-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-7.2.y
git checkout FETCH_HEAD
git cherry-pick -x 10180a277549339020b08000206092c07e0bff5a
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2026092945-unzip-pluck-7c40@gregkh' --subject-prefix 'PATCH 7.2.y' 'HEAD^..'
Possible dependencies:
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 10180a277549339020b08000206092c07e0bff5a Mon Sep 17 00:00:00 2001
From: Sean Christopherson <[email protected]>
Date: Mon, 21 Sep 2026 14:16:07 -0700
Subject: [PATCH] KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said
pages fails
Re-pend GET_NESTED_STATE_PAGES before exiting to userspace if getting the nested pages fails in the KVM_RUN path. If userspace re-runs the vCPU, and vmcs02 holds valid PFNs from the *previous* run of L2, then KVM could re-enter L2 with stale, unpinned PFNs mapped into e.g. the vAPIC page.
Note, both SVM and VMX (as of commit 11722439fb20 ("KVM: nVMX: Ensure KVM_REQ_GET_NESTED_STATE_PAGES is cleared on VM-Exit") ensure the request is cleared on VM-Exit (including the "forced" case), i.e. there is no risk of double-mapping due to emulated VMLAUNCH/VMRESUME/VMRUN *and* the request trying to map the nested pages.
arch/x86/kvm/x86.c +1 -0
@@ -8072,6 +8072,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) {if (unlikely (!kvm_nested_call (get_nested_state_pages)(vcpu))) { +kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); r =0 ; goto out; }
Sasha Levin
maintainerPATCH 1/8 KVM: x86: Move IRQ-related helper declarations from kvm_host.h => irq.h
From: Sean Christopherson <[email protected]>
[ Upstream commit 0bdd2d6d7328a7debcf138a7d6ef95d01a26b1b8 ]
Move the function declaration for APIs to get/query pending IRQs from kvm_host.h to irq.h, as the APIs are only used by KVM x86 code.
No functional change intended.
Message-ID: <[email protected]>
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/include/asm/kvm_host.h | 5 ----- arch/x86/kvm/irq.h | 6 ++++++ arch/x86/kvm/svm/nested.c | 1 + arch/x86/kvm/vmx/nested.c | 1 + 4 files changed, 8 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -5
@@ -2462,12 +2462,7 @@ enum {# define kvm_memslots_for_spte_role (kvm, role) __kvm_memslots(kvm, 0)#endif -int kvm_cpu_has_injectable_intr (struct kvm_vcpu *v); -int kvm_cpu_has_interrupt (struct kvm_vcpu *vcpu); -int kvm_cpu_has_extint (struct kvm_vcpu *v);int kvm_arch_interrupt_allowed (struct kvm_vcpu *vcpu); -int kvm_cpu_get_extint (struct kvm_vcpu *v); -int kvm_cpu_get_interrupt (struct kvm_vcpu *v);void kvm_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low,
arch/x86/kvm/irq.h +6 -0
@@ -112,6 +112,12 @@ static inline int irqchip_in_kernel(struct kvm *kvm)return mode !=KVM_IRQCHIP_NONE ; } +int kvm_cpu_has_injectable_intr (struct kvm_vcpu *v); +int kvm_cpu_has_interrupt (struct kvm_vcpu *vcpu); +int kvm_cpu_has_extint (struct kvm_vcpu *v); +int kvm_cpu_get_extint (struct kvm_vcpu *v); +int kvm_cpu_get_interrupt (struct kvm_vcpu *v); +void kvm_inject_pending_timer_irqs (struct kvm_vcpu *vcpu);void kvm_inject_apic_timer_irqs (struct kvm_vcpu *vcpu);void kvm_apic_nmi_wd_deliver (struct kvm_vcpu *vcpu);
arch/x86/kvm/svm/nested.c +1 -0
@@ -23,6 +23,7 @@#include "kvm_emulate.h" #include "trace.h" +#include "irq.h" #include "mmu.h" #include "x86.h" #include "smm.h"
arch/x86/kvm/vmx/nested.c +1 -0
@@ -11,6 +11,7 @@#include "x86.h" #include "cpuid.h" #include "hyperv.h" +#include "irq.h" #include "mmu.h" #include "nested.h" #include "pmu.h"
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 2/8 KVM: x86: Rework kvm_arch_interrupt_allowed() into kvm_is_interrupt_allowed()
From: Sean Christopherson <[email protected]>
[ Upstream commit 5b9585bc5b883bb64f02bf171fd7c676caecd16f ]
Rename kvm_arch_interrupt_allowed() to kvm_is_interrupt_allowed() and change its return type to a boolean, as the purpose of the helper is purely to check if interrupts are architecturally allowed. I.e. whether or not interrupts are temporarily disallowed due a pending nested VM-Enter is irrelevant (and callers are most definitely not supposed to care).
Opportunistically bury the helper in x86.c, as it hasn't been referenced by arch-neutral code since commit a1b37100d9e2 ("KVM: Reduce runnability interface with arch support code"), and has long since gained _very_ x86-specific semantics (see above).
Opportunistically add a comment to call out that treating -EBUSY as "allowed" is intentional.
For all intents and purposes, no functional change intended (KVM treats -EBUSY as "allowed" before and after).
Message-ID: <[email protected]>
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/x86.c | 23 +++++++++++++++-------- 2 files changed, 15 insertions(+), 9 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -1
@@ -2462,7 +2462,6 @@ enum {# define kvm_memslots_for_spte_role (kvm, role) __kvm_memslots(kvm, 0)#endif -int kvm_arch_interrupt_allowed (struct kvm_vcpu *vcpu);void kvm_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low,
arch/x86/kvm/x86.c +15 -8
@@ -4911,6 +4911,18 @@ static int kvm_vcpu_ioctl_set_lapic(struct kvm_vcpu *vcpu,return 0 ; } +static bool kvm_is_interrupt_allowed (struct kvm_vcpu *vcpu) +{ +/* +* Note, .interrupt_allowed() returns -EBUSY if interrupts are allowed +* based on CPU state, but can't be immediately delivered due to a +* pending nested VM-Enter. Treat that case as "allowed", because +* the goal here is just to check if interrupts are architecturally +* allowed, not to check if they can be injected. +*/ +return kvm_x86_call (interrupt_allowed)(vcpu, false); +} +static int kvm_cpu_accept_dm_intr (struct kvm_vcpu *vcpu) {/*
@@ -4936,7 +4948,7 @@ static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)* or KVM_SET_SREGS. For that to work, we must be at an * instruction boundary and with no events half-injected. */ -return (kvm_arch_interrupt_allowed (vcpu) && +return (kvm_is_interrupt_allowed (vcpu) &&kvm_cpu_accept_dm_intr (vcpu) && !kvm_event_needs_reinjection (vcpu) && !kvm_is_exception_pending (vcpu));
@@ -11140,7 +11152,7 @@ bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)if (kvm_test_request (KVM_REQ_UPDATE_PROTECTED_GUEST_STATE , vcpu))return true; -if (kvm_arch_interrupt_allowed (vcpu) &&kvm_cpu_has_interrupt (vcpu)) +if (kvm_is_interrupt_allowed (vcpu) &&kvm_cpu_has_interrupt (vcpu))return true;if (kvm_hv_has_stimer_pending (vcpu))
@@ -13004,11 +13016,6 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)return kvm_vcpu_exiting_guest_mode (vcpu) ==IN_GUEST_MODE ; } -int kvm_arch_interrupt_allowed (struct kvm_vcpu *vcpu) -{ -return kvm_x86_call (interrupt_allowed)(vcpu, false); -} -static inline u32 kvm_async_pf_hash_fn (gfn_t gfn) {BUILD_BUG_ON (!is_power_of_2 (ASYNC_PF_PER_VCPU ));
@@ -13144,7 +13151,7 @@ bool kvm_can_do_async_pf(struct kvm_vcpu *vcpu)* If interrupts are off we cannot even use an artificial * halt state. */ -return kvm_arch_interrupt_allowed (vcpu); +return kvm_is_interrupt_allowed (vcpu); } bool kvm_arch_async_page_not_present(struct kvm_vcpu *vcpu,
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 3/8 KVM: x86/mmu: Move kvm_arch_async_page_ready() below kvm_tdp_page_fault()
In reply to Sasha LevinFrom: Sean Christopherson <[email protected]>
[ Upstream commit 31a2cf735cc00c01526bcb676f111d5c0b711d17 ]
Move the implementation of kvm_arch_async_page_ready() "down" in mmu.c so that it lives below kvm_tdp_page_fault(). This will allow moving kvm_mmu_do_page_fault() into mmu.c without needing a forward declaration.
No functional change intended.
Message-ID: <[email protected]>
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/kvm/mmu/mmu.c | 62 +++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 31 deletions(-)
arch/x86/kvm/mmu/mmu.c +31 -31
@@ -4600,37 +4600,6 @@ static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, kvm_vcpu_gfn_to_hva(vcpu, fault->gfn), &arch); } -void kvm_arch_async_page_ready (struct kvm_vcpu *vcpu,struct kvm_async_pf *work) -{ -int r; - -if (WARN_ON_ONCE (work->arch.error_code &PFERR_PRIVATE_ACCESS )) -return ; - -if ((vcpu->arch.mmu->root_role.direct != work->arch.direct_map) || - work->wakeup_all) -return ; - - r =kvm_mmu_reload (vcpu); -if (unlikely (r)) -return ; - -if (!vcpu->arch.mmu->root_role.direct && - work->arch.cr3 !=kvm_mmu_get_guest_pgd (vcpu, vcpu->arch.mmu)) -return ; - - r =kvm_mmu_do_page_fault (vcpu, work->cr2_or_gpa, work->arch.error_code, - true,NULL ,NULL ); - -/* -* Account fixed page faults, otherwise they'll never be counted, but -* ignore stats for all other return times. Page-ready "faults" aren't -* truly spurious and never trigger emulation -*/ -if (r ==RET_PF_FIXED ) - vcpu->stat.pf_fixed++; -} -static void kvm_mmu_finish_page_fault (struct kvm_vcpu *vcpu,struct kvm_page_fault *fault,int r) {
@@ -5088,6 +5057,37 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,return min (range->size, end - range->gpa); } +void kvm_arch_async_page_ready (struct kvm_vcpu *vcpu,struct kvm_async_pf *work) +{ +int r; + +if (WARN_ON_ONCE (work->arch.error_code &PFERR_PRIVATE_ACCESS )) +return ; + +if ((vcpu->arch.mmu->root_role.direct != work->arch.direct_map) || + work->wakeup_all) +return ; + + r =kvm_mmu_reload (vcpu); +if (unlikely (r)) +return ; + +if (!vcpu->arch.mmu->root_role.direct && + work->arch.cr3 !=kvm_mmu_get_guest_pgd (vcpu, vcpu->arch.mmu)) +return ; + + r =kvm_mmu_do_page_fault (vcpu, work->cr2_or_gpa, work->arch.error_code, + true,NULL ,NULL ); + +/* +* Account fixed page faults, otherwise they'll never be counted, but +* ignore stats for all other return times. Page-ready "faults" aren't +* truly spurious and never trigger emulation +*/ +if (r ==RET_PF_FIXED ) + vcpu->stat.pf_fixed++; +} +#ifdef CONFIG_KVM_GUEST_MEMFD static void kvm_assert_gmem_invalidate_lock_held (struct kvm_memory_slot *slot) {
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 4/8 KVM: x86/mmu: Move kvm_mmu_do_page_fault() from mmu_internal.h => mmu.c
In reply to Sasha LevinFrom: Sean Christopherson <[email protected]>
[ Upstream commit 20fe9252460524f9028b515f0f672442ffe6779b ]
Move kvm_mmu_do_page_fault() into mmu.c, as there are no users outside of mmu.c, and the function typically isn't inlined by the compiler anyways. This will allow moving the EMULTYPE_xxx definitions into x86.h without having to include x86.h in mmu_internal.h, i.e. will help preserve the goal of making x86.h KVM x86's "top-level" include.
No functional change intended.
Message-ID: <[email protected]>
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/kvm/mmu/mmu.c | 67 ++++++++++++++++++++++++++++++++- arch/x86/kvm/mmu/mmu_internal.h | 66 -------------------------------- 2 files changed, 66 insertions(+), 67 deletions(-)
arch/x86/kvm/mmu/mmu.c +66 -1
@@ -4957,7 +4957,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, }#endif -int kvm_tdp_page_fault (struct kvm_vcpu *vcpu,struct kvm_page_fault *fault) +static int kvm_tdp_page_fault (struct kvm_vcpu *vcpu,struct kvm_page_fault *fault) {#ifdef CONFIG_X86_64 if (tdp_mmu_enabled)
@@ -4967,6 +4967,71 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)return direct_page_fault (vcpu, fault); } +static int kvm_mmu_do_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa, +u64 err,bool prefetch,int *emulation_type, +u8 *level) +{ +struct kvm_page_fault fault = { + .addr = cr2_or_gpa, + .error_code = err, + .exec = err &PFERR_FETCH_MASK , + .write = err &PFERR_WRITE_MASK , + .present = err &PFERR_PRESENT_MASK , + .rsvd = err &PFERR_RSVD_MASK , + .user = err &PFERR_USER_MASK , + .prefetch = prefetch, + .is_tdp =likely (vcpu->arch.mmu->page_fault == kvm_tdp_page_fault), + .nx_huge_page_workaround_enabled = +is_nx_huge_page_enabled (vcpu->kvm), + + .max_level =KVM_MAX_HUGEPAGE_LEVEL , + .req_level =PG_LEVEL_4K , + .goal_level =PG_LEVEL_4K , + .is_private = err &PFERR_PRIVATE_ACCESS , + + .pfn =KVM_PFN_ERR_FAULT , + }; +int r; + +if (vcpu->arch.mmu->root_role.direct) { +/* +* Things like memslots don't understand the concept of a shared +* bit. Strip it so that the GFN can be used like normal, and the +* fault.addr can be used when the shared bit is needed. +*/ + fault.gfn =gpa_to_gfn (fault.addr) & ~kvm_gfn_direct_bits (vcpu->kvm); + fault.slot =kvm_vcpu_gfn_to_memslot (vcpu, fault.gfn); + } + +/* +* With retpoline being active an indirect call is rather expensive, +* so do a direct call in the most common case. +*/ +if (IS_ENABLED (CONFIG_MITIGATION_RETPOLINE ) && fault.is_tdp) + r =kvm_tdp_page_fault (vcpu, &fault); +else + r = vcpu->arch.mmu->page_fault (vcpu, &fault); + +/* +* Not sure what's happening, but punt to userspace and hope that +* they can fix it by changing memory to shared, or they can +* provide a better error. +*/ +if (r ==RET_PF_EMULATE && fault.is_private) { +pr_warn_ratelimited ("kvm: unexpected emulation request on private memory\n" ); +kvm_mmu_prepare_memory_fault_exit (vcpu, &fault); +return -EFAULT ; + } + +if (fault.write_fault_to_shadow_pgtable && emulation_type) + *emulation_type |=EMULTYPE_WRITE_PF_TO_SP ; +if (level) + *level = fault.goal_level; + +return r; +} + +static int kvm_tdp_page_prefault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code,u8 *level) {
arch/x86/kvm/mmu/mmu_internal.h +0 -66
78 lines not on this page · raw message
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 5/8 KVM: move TSS constants from kvm_host.h to tss.h
In reply to Sasha LevinFrom: Paolo Bonzini <[email protected]>
[ Upstream commit 35fdfa632e7cf8271189c7ae6f97e7c4b55f7f80 ]
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/include/asm/kvm_host.h | 7 ------- arch/x86/kvm/tss.h | 7 +++++++ arch/x86/kvm/vmx/vmx.c | 1 + 3 files changed, 8 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -7
@@ -2433,13 +2433,6 @@ static inline void kvm_inject_gp(struct kvm_vcpu *vcpu, u32 error_code)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code); } -#define TSS_IOPB_BASE_OFFSET 0x66 -#define TSS_BASE_SIZE 0x68 -#define TSS_IOPB_SIZE (65536 / 8) -#define TSS_REDIRECTION_SIZE (256 / 8) -#define RMODE_TSS_SIZE \ - (TSS_BASE_SIZE + TSS_REDIRECTION_SIZE + TSS_IOPB_SIZE + 1) -enum {TASK_SWITCH_CALL =0 ,TASK_SWITCH_IRET =1 ,
arch/x86/kvm/tss.h +7 -0
@@ -57,4 +57,11 @@ struct tss_segment_16 { u16 ldt; }; +#define TSS_IOPB_BASE_OFFSET 0x66 +#define TSS_BASE_SIZE 0x68 +#define TSS_IOPB_SIZE (65536 / 8) +#define TSS_REDIRECTION_SIZE (256 / 8) +#define RMODE_TSS_SIZE \ + (TSS_BASE_SIZE + TSS_REDIRECTION_SIZE + TSS_IOPB_SIZE + 1) +#endif
arch/x86/kvm/vmx/vmx.c +1 -0
@@ -72,6 +72,7 @@#include "x86.h" #include "x86_ops.h" #include "smm.h" +#include "tss.h" #include "vmx_onhyperv.h" #include "vmenter.h" #include "posted_intr.h"
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 6/8 KVM: x86: Reject nested CAP enablement if nested virtualization is disabled
In reply to Sasha LevinFrom: Sean Christopherson <[email protected]>
[ Upstream commit b65699be2c606d2687593516d93e66d16a713b61 ]
Add a flag to explicitly track if nested virtualization is enabled, and use it enumerate that various nested CAPs are unsupported, and to reject enablement of said CAPs. When the nested ops hooks were moved to their own structure, KVM's NULL-by-default behavior was deliberately dropped, with the changelog asserting that all was well. That wasn't quite true; there is no danger to KVM, but now KVM is over-reporting support for KVM_CAP_NESTED_STATE and KVM_CAP_HYPERV_ENLIGHTENED_VMCS.
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/hyperv.c | 3 ++- arch/x86/kvm/svm/svm.c | 1 + arch/x86/kvm/vmx/vmx.c | 1 + arch/x86/kvm/x86.c | 12 +++++++----- 5 files changed, 13 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -2019,6 +2019,8 @@ struct kvm_x86_ops { };struct kvm_x86_nested_ops { +bool enabled; +void (*leave_nested)(struct kvm_vcpu *vcpu);bool (*is_exception_vmexit)(struct kvm_vcpu *vcpu,u8 vector,u32 error_code);
arch/x86/kvm/hyperv.c +2 -1
@@ -2800,7 +2800,8 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, };int i, nent =ARRAY_SIZE (cpuid_entries); -if (kvm_x86_ops.nested_ops->get_evmcs_version) +if (kvm_x86_ops.nested_ops->enabled && + kvm_x86_ops.nested_ops->get_evmcs_version) evmcs_ver = kvm_x86_ops.nested_ops->get_evmcs_version (vcpu);if (cpuid->nent < nent)
arch/x86/kvm/svm/svm.c +1 -0
@@ -5662,6 +5662,7 @@ static __init int svm_hardware_setup(void)if (r)return r; } + svm_nested_ops.enabled = nested;/* * KVM's MMU doesn't support using 2-level paging for itself, and thus
arch/x86/kvm/vmx/vmx.c +1 -0
@@ -8771,6 +8771,7 @@ __init int vmx_hardware_setup(void)if (r)return r; } + vmx_nested_ops.enabled = nested;kvm_set_posted_intr_wakeup_handler (pi_wakeup_handler);
arch/x86/kvm/x86.c +7 -5
@@ -4517,7 +4517,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r &= ~KVM_X2APIC_ENABLE_SUPPRESS_EOI_BROADCAST ;break ;case KVM_CAP_NESTED_STATE : - r = kvm_x86_ops.nested_ops->get_state ? + r = kvm_x86_ops.nested_ops->enabled ? kvm_x86_ops.nested_ops->get_state (NULL ,NULL ,0 ) :0 ;break ;#ifdef CONFIG_KVM_HYPERV
@@ -4525,7 +4525,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = kvm_x86_ops.enable_l2_tlb_flush !=NULL ;break ;case KVM_CAP_HYPERV_ENLIGHTENED_VMCS : - r = kvm_x86_ops.nested_ops->enable_evmcs !=NULL ; + r = kvm_x86_ops.nested_ops->enabled && + kvm_x86_ops.nested_ops->enable_evmcs !=NULL ;break ;#endif case KVM_CAP_SMALLER_MAXPHYADDR :
@@ -5579,7 +5580,8 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,uint16_t vmcs_version;void __user *user_ptr; -if (!kvm_x86_ops.nested_ops->enable_evmcs) +if (!kvm_x86_ops.nested_ops->enabled || + !kvm_x86_ops.nested_ops->enable_evmcs)return -ENOTTY ; r = kvm_x86_ops.nested_ops->enable_evmcs (vcpu, &vmcs_version);if (!r) {
@@ -6079,7 +6081,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,u32 user_data_size; r = -EINVAL ; -if (!kvm_x86_ops.nested_ops->get_state) +if (!kvm_x86_ops.nested_ops->enabled)break ;BUILD_BUG_ON (sizeof (user_data_size) !=sizeof (user_kvm_nested_state->size));
@@ -6109,7 +6111,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,int idx; r = -EINVAL ; -if (!kvm_x86_ops.nested_ops->set_state) +if (!kvm_x86_ops.nested_ops->enabled)break ; r = -EFAULT ;
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 7/8 KVM: x86: Add static calls for nested virtualization ops
In reply to Sasha LevinFrom: Sean Christopherson <[email protected]>
[ Upstream commit 4b9819a50674dd40d176033f9ea2e23a70889dc6 ]
Use static calls to invoke nested virtualization ops, as many of the calls are in relatively hot paths when L2 is active, e.g. checking for events, and because there's no reason not use static calls these days.
Opportunistically use a RET0 static call for get_evmcs_version() instead of manually checking for a non-NULL vendor hook.
Backport notes:
Preserve the stable tree's kvm_x86_vendor_init()/exit() declarations and ignore_msrs/report_ignored_msrs module parameters at the conflicting insertion points. Initialize nested static calls in the existing kvm_ops_update() instead of adding kvm_nested_ops_update(), preserving the same mandatory, optional, and return-zero hook behavior without adding a function. Keep the get_nested_state_pages call conversion so target 10180a277549339020b08000206092c07e0bff5a applies unchanged.
Stable-dep-of: 10180a277549 ("KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails")
--- arch/x86/include/asm/kvm-x86-nested-ops.h | 36 ++++++++++++++++ arch/x86/include/asm/kvm_host.h | 8 ++++ arch/x86/kvm/hyperv.c | 7 ++-- arch/x86/kvm/mmu.h | 5 +-- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/x86.c | 50 +++++++++++++++-------- arch/x86/kvm/x86.h | 2 +- 7 files changed, 83 insertions(+), 27 deletions(-) create mode 100644 arch/x86/include/asm/kvm-x86-nested-ops.h
arch/x86/include/asm/kvm-x86-nested-ops.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(KVM_X86_NESTED_OP ) || \ + !defined(KVM_X86_NESTED_OP_OPTIONAL ) || \ + !defined(KVM_X86_NESTED_OP_OPTIONAL_RET0 ) +#error Missing one or more KVM_X86_NESTED_OP #defines +#else +/* +* KVM_X86_NESTED_OP() and KVM_X86_NESTED_OP_OPTIONAL() are used to help +* generate both DECLARE/DEFINE_STATIC_CALL() invocations and +* "static_call_update()" calls. +* +* KVM_X86_NESTED_OP_OPTIONAL() can be used for those functions that can have +* a NULL definition. KVM_X86_NESTED_OP_OPTIONAL_RET0() can be used likewise +* to make a definition optional, but in this case the default will +* be __static_call_return0. +*/ +KVM_X86_NESTED_OP(leave_nested) +KVM_X86_NESTED_OP(is_exception_vmexit) +KVM_X86_NESTED_OP(check_events) +KVM_X86_NESTED_OP_OPTIONAL_RET0(has_events) +KVM_X86_NESTED_OP(triple_fault) +KVM_X86_NESTED_OP(get_state) +KVM_X86_NESTED_OP(set_state) +KVM_X86_NESTED_OP(get_nested_state_pages) +KVM_X86_NESTED_OP_OPTIONAL_RET0(write_log_dirty) +KVM_X86_NESTED_OP(translate_nested_gpa) +#ifdef CONFIG_KVM_HYPERV +KVM_X86_NESTED_OP_OPTIONAL(enable_evmcs) +KVM_X86_NESTED_OP_OPTIONAL_RET0(get_evmcs_version) +KVM_X86_NESTED_OP(hv_inject_synthetic_vmexit_post_tlb_flush) +#endif +#endif + +#undef KVM_X86_NESTED_OP +#undef KVM_X86_NESTED_OP_OPTIONAL +#undef KVM_X86_NESTED_OP_OPTIONAL_RET0
arch/x86/include/asm/kvm_host.h +8 -0
@@ -2077,6 +2077,14 @@ extern struct kvm_x86_ops kvm_x86_ops;#define KVM_X86_OP_OPTIONAL_RET0 KVM_X86_OP#include <asm/kvm-x86-ops.h> +#define kvm_nested_call (func) static_call(kvm_x86_nested_##func) + +#define KVM_X86_NESTED_OP (func) \ + DECLARE_STATIC_CALL(kvm_x86_nested_##func, *(((struct kvm_x86_nested_ops *)0)->func)); +#define KVM_X86_NESTED_OP_OPTIONAL KVM_X86_NESTED_OP +#define KVM_X86_NESTED_OP_OPTIONAL_RET0 KVM_X86_NESTED_OP +#include <asm/kvm-x86-nested-ops.h> +int kvm_x86_vendor_init (struct kvm_x86_init_ops *ops);void kvm_x86_vendor_exit (void );
arch/x86/kvm/hyperv.c +3 -4
@@ -2419,7 +2419,7 @@ static int kvm_hv_hypercall_complete(struct kvm_vcpu *vcpu, u64 result) ret =kvm_skip_emulated_instruction (vcpu);if (tlb_lock_count) - kvm_x86_ops.nested_ops->hv_inject_synthetic_vmexit_post_tlb_flush (vcpu); +kvm_nested_call (hv_inject_synthetic_vmexit_post_tlb_flush)(vcpu);return ret; }
@@ -2800,9 +2800,8 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, };int i, nent =ARRAY_SIZE (cpuid_entries); -if (kvm_x86_ops.nested_ops->enabled && - kvm_x86_ops.nested_ops->get_evmcs_version) - evmcs_ver = kvm_x86_ops.nested_ops->get_evmcs_version (vcpu); +if (kvm_x86_ops.nested_ops->enabled) + evmcs_ver =kvm_nested_call (get_evmcs_version)(vcpu);if (cpuid->nent < nent)return -E2BIG ;
arch/x86/kvm/mmu.h +2 -3
@@ -308,9 +308,8 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, {if (mmu != &vcpu->arch.nested_mmu)return gpa; -return kvm_x86_ops.nested_ops->translate_nested_gpa (vcpu, gpa, access, - exception, - pte_access); +return kvm_nested_call (translate_nested_gpa)(vcpu, gpa, access, + exception, pte_access); }static inline bool kvm_has_mirrored_tdp (const struct kvm *kvm)
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -233,7 +233,7 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, !(pte & PT_GUEST_DIRTY_MASK)) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte));#if PTTYPE ==PTTYPE_EPT -if (kvm_x86_ops.nested_ops->write_log_dirty (vcpu, addr)) +if (kvm_nested_call (write_log_dirty)(vcpu, addr))return -EINVAL ;#endif pte |=PT_GUEST_DIRTY_MASK ;
arch/x86/kvm/x86.c +32 -18
137 lines not on this page · raw message
arch/x86/kvm/x86.h +1 -1
@@ -151,7 +151,7 @@ int kvm_check_nested_events(struct kvm_vcpu *vcpu);/* Forcibly leave the nested mode in cases like a vCPU reset */ static inline void kvm_leave_nested (struct kvm_vcpu *vcpu) { - kvm_x86_ops.nested_ops->leave_nested (vcpu); +kvm_nested_call (leave_nested)(vcpu); }/*
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 8/8 KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails
In reply to Sasha LevinFrom: Sean Christopherson <[email protected]>
[ Upstream commit 10180a277549339020b08000206092c07e0bff5a ]
Re-pend GET_NESTED_STATE_PAGES before exiting to userspace if getting the nested pages fails in the KVM_RUN path. If userspace re-runs the vCPU, and vmcs02 holds valid PFNs from the *previous* run of L2, then KVM could re-enter L2 with stale, unpinned PFNs mapped into e.g. the vAPIC page.
Note, both SVM and VMX (as of commit 11722439fb20 ("KVM: nVMX: Ensure KVM_REQ_GET_NESTED_STATE_PAGES is cleared on VM-Exit") ensure the request is cleared on VM-Exit (including the "forced" case), i.e. there is no risk of double-mapping due to emulated VMLAUNCH/VMRESUME/VMRUN *and* the request trying to map the nested pages.
--- arch/x86/kvm/x86.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/x86.c +1 -0
@@ -10706,6 +10706,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) {if (unlikely (!kvm_nested_call (get_nested_state_pages)(vcpu))) { +kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); r =0 ; goto out; }
Signature · 1 line
2.53.0